2 mzpp

mzpp is a simple preprocessor that allows mixing Scheme code with text files in a similar way to PHP or BRL. Processing of input files works by translating the input file to Scheme code that prints the contents, except for marked portions that contain Scheme code. The Scheme parts of a file are marked with << and >> tokens by default. The Scheme code is then passed through a read-eval-print loop that is similar to a normal REPL with a few differences in how values are printed.

2.1 Invoking mzpp

Use the --h flag to get the available flags. See above for an explanation of the --run flag.

2.2 mzpp files

Here is a sample file that mzpp can process, using the default beginning and ending markers:

<< (define bar "BAR") >>

foo1

foo2 << bar newline* bar >> baz

foo3

First, this file is converted to the following Scheme code:

(thunk (cd "tmp/") (current-file "foo"))

(thunk (push-indentation ""))

(define bar "BAR") (thunk (pop-indentation))

newline*

"foo1"

newline*

"foo2 "

(thunk (push-indentation "     "))

bar newline* bar (thunk (pop-indentation))

" baz"

newline*

"foo3"

newline*

(thunk (cd "/home/eli") (current-file #f))

which is then fed to the REPL, resulting in the following output:

foo1

foo2 BAR

BAR baz

foo3

To see the processed input that the REPL receives, use the --debug flag. Note that the processed code contains expressions that have no side-effects, only values – see below for an explanation of the REPL printing behavior. Some expressions produce values that change the REPL environment, for example, the indentation commands are used to keep track of the column where the Scheme marker was found, and cd is used to switch to the directory where the file is (here it was in "/home/foo/tmp") so including a relative file works. Also, note that the first newline* did not generate a newline, and that the one in the embedded Scheme code added the appropriate spaces for indentation.

It is possible to temporarily switch from Scheme to text-mode and back in a way that does not respect a complete Scheme expression, but you should be aware that text is converted to a sequence of side-effect free expressions (not to a single string, and not expression that uses side effects). For example:

<< (if (zero? (random 2))

(list >>foo1<<)

(list >>foo2<<))

>>

<< (if (zero? (random 2)) (list >>

foo1

<<) (list >>

foo2

<<)) >>

will print two lines, each containing foo1 or foo (the first approach plays better with the smart space handling). The show function can be used instead of list with the same results, since it will print out the values in the same way the REPL does. The conversion process does not transform every continuous piece of text into a single Scheme string because doing this:

• the Scheme process will need to allocating big strings which makes this unfeasible for big files,

• it will not play well with “interactive” input feeding, for example, piping in the output of some process will show results only on Scheme marker boundaries,

• special treatment for newlines in these strings will become expensive.

(Note that this is different from the BRL approach.)

2.3 Raw preprocessing directives

Some preprocessing directives happen at the "raw level" – the stage where text is transformed into Scheme expressions. These directives cannot be changed from withing transformed text because they change the way this transformation happens. Some of these transformation

• Skipping input:

  First, the processing can be modified by specifying a skip-to string that disables any output until a certain line is seen. This is useful for script files that use themselves for input. For example, the following script:

  #!/bin/sh

  echo shell output

  exec mzpp -s "---TEXT-START---" "$0"

  exit 1

  ---TEXT-START---

  Some preprocessed text

  123*456*789 = << (* 123 456 789) >>

  will produce this output:

  shell output

  Some preprocessed text

  123*456*789 = 44253432

• Quoting the markers:

  In case you need to use the actual text of the markers, you can quote them. A backslash before a beginning or an ending marker will make the marker treated as text, it can also quote a sequence of backslashes and a marker. For example, using the default markers, \<<\>> will output <<>>, \\<<\\\>> will output \<<\\>> and \a\b\<< will output \a\b<<.

• Modifying the markers:

  Finally, if the markers collide with a certain file contents, it is possible to change them. This is done by a line with a special structure – if the current Scheme markers are <beg1> and <end1> then a line that contains exactly:

  <beg1><beg2><beg1><end1><end2><end1>

  will change the markers to <beg2> and <end2>. It is possible to change the markers from the Scheme side (see below), but this will not change already-transformed text, which is the reason for this special format.

2.4 The mzpp read-eval-print loop

The REPL is initialized by requiring preprocessor/mzpp, so the same module provides both the preprocessor functionality as well as bindings for embedded Scheme code in processed files. The REPL is then fed the transformed Scheme code that is generated from the source text (the same code that --debug shows). Each expression is evaluated and its result is printed using the show function (multiple values are all printed), where show works in the following way:

• #<void> and #f values are ignored.

• Structures of pairs are recursively scanned and their parts printed (no spaces are used, so to produce Scheme code as output you must use format strings – again, this is not intended for preprocessing Scheme code).

• Procedures are applied to zero arguments (so a procedure that doesn’t accept zero arguments will cause an error) and the result is sent back to show. This is useful for using thunks to wrap side-effects as values (e.g, the thunk wraps shown by the debug output above).

• Promises are forced and the result is sent again to show.

• All other values are printed with display. No newlines are used after printing values.

2.5 Provided bindings

(require preprocessor/mzpp)

First, bindings that are mainly useful for invoking the preprocessor:

All of the above are accessible in preprocessed texts, but the only one that might make any sense to use is preprocess and include is a better choice. When include is used, it can be wrapped with parameter settings, which is why they are available. Note in particular that these parameters change the way that the text transformation works and have no effect over the current preprocessed document (for example, the Scheme marks are used in a different thread, and skip-to cannot be re-set when processing has already began). The only one that could be used is no-spaces? but even that makes little sense on selected parts.

The following are bindings that are used in preprocessed texts:

Note that when a sequence of files are processed (through command-line arguments or through a single include expression), then they are all taken as one textual unit – so changes to the markers, working directory etc in one file can modify the way sequential files are processed. This means that including two files in a single include expression can be different than using two expressions.