12.4 Syntax Transformers

The property value must be an exact integer or procedure of one or two arguments. In the former case, the integer designates a field within the structure that should contain a procedure; the integer must be between 0 (inclusive) and the number of non-automatic fields in the structure type (exclusive, not counting supertype fields), and the designated field must also be specified as immutable.

If the property value is a procedure of one argument, then the procedure serves as a syntax transformer and for set! transformations. If the property value is a procedure of two arguments, then the first argument is the structure whose type has prop:set!-transformer property, and the second argument is a syntax object as for a syntax transformer and for set! transformations; set!-transformer-procedure applied to the structure produces a new function that accepts just the syntax object and calls the procedure associated through the property. Finally, if the property value is an integer, the target identifier is extracted from the structure instance; if the field value is not a procedure of one argument, then a procedure that always calls raise-syntax-error is used, instead.

If a value has both the prop:set!-transformer and prop:rename-transformer properties, then the latter takes precedence. If a structure type has the prop:set!-transformer and prop:procedure properties, then the former takes precedence for the purposes of macro expansion.

Such a transformer could be written manually, but the one created by make-rename-transformer triggers special cooperation with the parser and other syntactic forms when id is bound to the rename transformer:

• The parser installs a free-identifier=? and identifier-binding equivalence between id and id-stx, as long as id-stx does not have a true value for the 'not-free-identifier=? syntax property.

• A provide of id provides the binding indicated by id-stx instead of id, as long as id-stx does not have a true value for the 'not-free-identifier=? syntax property and as long as id-stx has a binding.

• If provide exports id, it uses a symbol-valued 'nominal-id property of id-stx to specify the “nominal source identifier” of the binding as reported by identifier-binding.

• If id-stx has a true value for the 'not-provide-all-defined syntax property, then id (or its target) is not exported by all-defined-out.

• The syntax-local-value function recognizes rename-transformer bindings and consult their targets.

Changed in version 6.3 of package base: Removed an optional second argument.
Changed in version 7.4.0.10: Adjusted rename-transformer expansion to add a macro-introduction scope, the same as regular macro expansion.

The property value must be an exact integer, an identifier syntax object, or a procedure that takes one argument. In the former case, the integer designates a field within the structure that should contain an identifier; the integer must be between 0 (inclusive) and the number of non-automatic fields in the structure type (exclusive, not counting supertype fields), and the designated field must also be specified as immutable.

If the property value is an identifier, the identifier serves as the target for renaming, just like the first argument to make-rename-transformer. If the property value is an integer, the target identifier is extracted from the structure instance; if the field value is not an identifier, then an identifier ? with an empty context is used, instead.

If the property value is a procedure that takes one argument, then the procedure is called to obtain the identifier that the rename transformer will use as a target identifier. The returned identifier should probably have the 'not-free-identifier=? syntax property. If the procedure returns any value that is not an identifier, the exn:fail:contract exception is raised.

Changed in version 6.3 of package base: the property now accepts a procedure of one argument.

The stop-ids argument controls how far local-expand expands stx:

• If stop-ids is an empty list, then stx is recursively expanded (i.e. expansion proceeds to sub-expressions). The result is guaranteed to be a fully-expanded form, which can include the bindings listed in Fully Expanded Programs, plus #%expression in any expression position.

• If stop-ids is a list containing just module*, then expansion proceeds as if stop-ids were an empty list, except that expansion does not recur to submodules defined with module* (which are left unexpanded in the result).

• If stop-ids is any other list, then begin, quote, set!, #%plain-lambda, case-lambda, let-values, letrec-values, if, begin0, with-continuation-mark, letrec-syntaxes+values, #%plain-app, #%expression, #%top, and #%variable-reference are implicitly added to stop-ids. Expansion proceeds recursively, stopping when the expander encounters any of the forms in stop-ids, and the result is the partially-expanded form.

  When the expander would normally implicitly introduce a #%app, #%datum, or #%top identifier as described in Expansion Steps, it checks to see if an identifier with the same binding as the one to be introduced appears in stop-ids. If so, the identifier is not introduced; the result of expansion is the bare application, literal data expression, or unbound identifier rather than one wrapped in the respective explicit form.

  When #%plain-module-begin is not in stop-ids, the #%plain-module-begin transformer detects and expands sub-forms (such as define-values) regardless of the identifiers presence in stop-ids.

  Expansion does not replace the scopes in a local-variable reference to match the binding identifier.

• If stop-ids is #f instead of a list, then stx is expanded only as long as the outermost form of stx is a macro (i.e. expansion does not proceed to sub-expressions, and it does not replace the scopes in a local-variable reference to match the binding identifier). The #%app, #%datum, and #%top identifiers are never introduced.

Independent of stop-ids, when local-expand encounters an identifier that has a local binding but no binding in the current expansion context, the variable is left as-is (as opposed to triggering an “out of context” syntax error).

When context-v is 'module-begin, and the result of expansion is a #%plain-module-begin form, then a 'submodule syntax property is added to each enclosed module form (but not module* forms) in the same way as by module expansion.

If the intdef-ctx argument is an internal-definition context, its bindings and bindings from all parent internal-definition contexts are added to the local binding context during the dynamic extent of the call to local-expand. Additionally, unless #f was provided for the add-scope? argument to syntax-local-make-definition-context when the internal-definition context was created, its inside-edge scope (but not the scopes of any parent internal-definition contexts) is added to the lexical information for both stx prior to its expansion and the expansion result (because the expansion might introduce bindings or references to internal-definition bindings).

For backwards compatibility, when intdef-ctx is a list all bindings from all of the provided internal-definition contexts and their parents are added to the local binding context, and the inside-edge scope from each context for which add-scope? was not #f is added in the same way.

Expansion records use-site scopes for removal from definition bindings. When the intdef-ctx argument is an internal-definition context, use-site scopes are recorded with that context. When intdef-ctx is #f or (for backwards compatibility) a list, use-site scopes are recorded with the current expand context.

For a particular internal-definition context, generate a unique value and put it into a list for context-v. To allow liberal expansion of define forms, the generated value should be an instance of a structure with a true value for prop:liberal-define-context. If the internal-definition context is meant to be self-contained, the list for context-v should contain only the generated value; if the internal-definition context is meant to splice into an immediately enclosing context, then when syntax-local-context produces a list, cons the generated value onto that list.

When expressions are expanded via local-expand with an internal-definition context intdef-ctx, and when the expanded expressions are incorporated into an overall form new-stx, then typically internal-definition-context-track should be applied to intdef-ctx and new-stx to provide expansion history to external tools.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

This procedure’s binding is provided as protected in the sense of protect-out.

Changed in version 6.0.1.3 of package base: Changed treatment of #%top so that it is never introduced as an explicit wrapper.
Changed in version 6.0.90.27: Loosened the contract on the intdef-ctx argument to allow an empty list.
Changed in version 8.2.0.4: Changed binding to protected.

The latter can be used in place of the former (perhaps in a larger expression produced by a macro transformer), and when the macro expander encounters the opaque object, it substitutes the fully expanded expression without re-expanding it; the exn:fail:syntax exception is raised if the expansion context includes scopes that were not present for the original expansion, in which case re-expansion might produce different results. Consistent use of syntax-local-expand-expression and the opaque object thus avoids quadratic expansion times when local expansions are nested.

If opaque-only? is true, then the first result is #f instead of the expanded expression. Obtaining only the second, opaque result can be more efficient in some expansion contexts.

Unlike local-expand, syntax-local-expand-expression normally produces an expanded expression that contains no #%expression forms. However, if syntax-local-expand-expression is used within an expansion that is triggered by an enclosing local-expand call, then the result of syntax-local-expand-expression can include #%expression forms.

Changed in version 6.90.0.13 of package base: Added the opaque-only? argument.
Changed in version 8.2.0.4: Changed binding to protected.

Any lifted expressions—from calls to syntax-local-lift-expression during the expansion of stx—are captured in the result. If context-v is 'top-level, then lifts are captured in a begin form, otherwise lifts are captured in let-values forms. If no expressions are lifted during expansion, then no begin or let-values wrapper is added.

This procedure’s binding is provided as protected in the sense of protect-out.

Changed in version 6.5.0.3 of package base: Allowed and captured lifts in a 'top-level context.
Changed in version 8.2.0.4: Changed binding to protected.

If context-v is 'top-level or 'module, then module forms can appear in the result as added via syntax-local-lift-module. If context-v is 'module, then module* forms can appear, too.

This procedure’s binding is provided as protected in the sense of protect-out.

Changed in version 8.2.0.4 of package base: Changed binding to protected.

This procedure’s binding is provided as protected in the sense of protect-out.

Changed in version 8.2.0.4 of package base: Changed binding to protected.

The context-v argument is as in local-expand, and the intdef-ctx is an internal-definition context value or #f. The binding-id specifies a binding associated with the transformer, which the expander uses to determine whether to add use-site scopes and which code inspector to use during expansion.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

Added in version 8.2.0.7 of package base.

Before expanding forms whose lexical context should include the definitions, the transformer should use internal-definition-context-add-scopes to apply the context’s scopes to the syntax. Calls to procedures such as local-expand to expand the forms should provide the internal-definition context value as an argument.

After discovering an internal define-values or define-syntaxes form, use syntax-local-bind-syntaxes to add bindings to the context.

An internal-definition context internally creates an outside-edge scope and an inside-edge scope to represent the context. The inside-edge scope is added to any form that is expanded within the context or that appears as the result of a (partial) expansion within the context. For backward compatibility, providing #f for add-scope? disables this behavior.

If parent-ctx is not #f, then parent-ctx is made the parent internal-definition context for the new internal-definition context. Whenever the new context’s bindings are added to the local binding context (e.g. by providing the context to local-expand, syntax-local-bind-syntaxes, or syntax-local-value), then the bindings from parent-ctx are also added as well. If parent-ctx was also created with a parent internal-definition context, bindings from its parent are also added, and so on recursively. Note that the scopes of parent contexts are not added implicitly, only the bindings, even when the inside-edge scope of the child context would be implicitly added. If the scopes of parent definition contexts should be added, the parent contexts must be provided explicitly.

Additionally, if the created definition context is intended to be spliced into a surrounding definition context, the surrounding context should always be provided for the parent-ctx argument to ensure the necessary use-site scopes are added to macros expanded in the context. Otherwise, expansion of nested definitions can be inconsistent with the expansion of definitions in the surrounding context.

An internal-definition context also tracks use-site scopes created during expansion within the definition context, so that they can be removed from bindings created in the context, at syntax-local-identifier-as-binding, and at internal-definition-context-splice-binding-identifier.

The scopes associated with a new definition context are pruned from quote-syntax forms only when it is created during the dynamic extent of a syntax transformer application or in a begin-for-syntax form (potentially nested) within a module being expanded.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

Changed in version 6.3 of package base: Added the add-scope? argument, and made calling internal-definition-context-seal no longer necessary.
Changed in version 8.2.0.7: Added the outside-edge scope and use-site scope tracking behaviors.

Use this function to apply the definition context scopes to syntax that originates within the definition context before expansion.

Added in version 8.2.0.7 of package base.

Use when splicing a binding originating within the intdef-ctx into a surrounding context.

Added in version 8.2.0.7 of package base.

For backwards compatibility, the lexical information of each element of extra-intdef-ctxs is also added to each identifier in id-list before binding.

Supply #f for expr when the identifiers correspond to define-values bindings, and supply a compile-time expression when the identifiers correspond to define-syntaxes bindings. In the latter case, the number of values produced by the expression should match the number of identifiers, otherwise the exn:fail:contract:arity exception is raised.

When expr is not #f, it is expanded in an expression context and evaluated in the current transformer environment. In this case, the bindings and lexical information from both intdef-ctx and extra-intdef-ctxs are used to enrich expr’s lexical information and extend the local binding context in the same way as the fourth argument to local-expand. If expr is #f, the value provided for extra-intdef-ctxs is ignored.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

Changed in version 6.90.0.27 of package base: Added the extra-intdef-ctxs argument.
Changed in version 8.2.0.7: Changed the return value from #<void> to the list of bound identifiers.

Added in version 6.3.0.4 of package base.

This function is provided for backwards compatibility; internal-definition-context-add-scopes and internal-definition-context-splice-binding-identifier are preferred.

Added in version 6.3 of package base.

The identifier-remove-from-definition-context function is provided for backward compatibility; the internal-definition-context-splice-binding-identifier function is preferred.

Changed in version 6.3 of package base: Simplified the operation to scope removal.

The prop:expansion-contexts property is most useful in combination with prop:rename-transformer, since a general transformer procedure can use syntax-local-context. Furthermore, a prop:expansion-contexts property makes the most sense when a rename transformer’s identifier has the 'not-free-identifier=? property, otherwise a definition of the binding creates a binding alias that effectively routes around the prop:expansion-contexts property.

Added in version 6.3 of package base.

If id-stx is bound to a rename transformer created with make-rename-transformer, syntax-local-value effectively calls itself with the target of the rename and returns that result, instead of the rename transformer.

If id-stx has no transformer binding (via define-syntax, let-syntax, etc.) in that environment, the result is obtained by applying failure-thunk if not #f. If failure-thunk is false, the exn:fail:contract exception is raised.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

This procedure’s binding is provided as protected in the sense of protect-out.

Changed in version 6.90.0.27 of package base: Changed intdef-ctx to accept a list of internal-definition contexts in addition to a single internal-definition context or #f.
Changed in version 8.2.0.4: Changed binding to protected.

If id-stx has no transformer binding, then failure-thunk is called (and it can return any number of values), or an exception is raised if failure-thunk is #f.

This procedure’s binding is provided as protected in the sense of protect-out.

Changed in version 8.2.0.4 of package base: Changed binding to protected.

A run-time expression within a module is lifted to the module’s top level, just before the expression whose expansion requests the lift. Similarly, a run-time expression outside of a module is lifted to a top-level definition. A compile-time expression in a letrec-syntaxes+values or define-syntaxes binding is lifted to a let wrapper around the corresponding right-hand side of the binding. A compile-time expression within begin-for-syntax is lifted to a define declaration just before the requesting expression within the begin-for-syntax.

Other syntactic forms can capture lifts by using local-expand/capture-lifts or local-transformer-expand/capture-lifts.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

The module is not immediately declared when syntax-local-lift-module returns. Instead, the module declaration is recorded for processing when expansion returns to the enclosing module body or top-level sequence.

Added in version 6.3 of package base.

The resulting syntax object is the same as stx, except that a fresh scope is added if new-scope? is true. The same scope is added to the lifted #%require form, so that the #%require form can bind uses of imported identifiers in the resulting syntax object (assuming that the lexical information of stx includes the binding environment into which the #%require is lifted). If new-scope? is #f, then the result exactly stx, and no scope is added to the lifted #%require form; in that case, take care to ensure that the lifted require does not change the meaning of already-expanded identifiers in the module, otherwise re-expansion of the enclosing module will not produce the same result as the expanded module.

If raw-require-spec is part of the input to a transformer, then typically syntax-local-introduce should be applied before passing it to syntax-local-lift-require. Otherwise, marks added by the macro expander can prevent access to the new imports.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

Changed in version 6.90.0.27 of package base: Changed the scope added to inputs from a macro-introduction scope to one that does not affect whether or not the resulting syntax is considered original as reported by syntax-original?.
Changed in version 8.6.0.4: Added the new-scope? optional argument.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

The symbol results indicate that the expression is being expanded for an expression context, a top-level context, a module context, or a module-begin context.

A list result indicates expansion in an internal-definition context. The identity of the list’s first element (i.e., its eq?ness) reflects the identity of the internal-definition context; in particular two transformer expansions receive the same first value if and only if they are invoked for the same internal-definition context. Later values in the list similarly identify internal-definition contexts that are still being expanded, and that required the expansion of nested internal-definition contexts.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

Changed in version 8.2.0.3 of package base: Generalized result to phase–space combinations.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

The current implementation returns all symbols for reachable interned scopes, but that behavior may change in the future to return a less conservative list of symbols.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

Added in version 8.2.0.7 of package base.

This function is intended for the implementation of syntax-parameterize and local-require.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

Changed in version 6.3 of package base: Simplified to the minimal functionality needed for syntax-parameterize and local-require.

Changed in version 6.3 of package base: changed to raise exn:fail:supported.

Currently, (syntax-transforming?) implies (syntax-transforming-with-lifts?).

Added in version 6.3.0.9 of package base.

In a syntax transformer that runs in a non-expression context and forces the expansion of subforms with local-expand, use syntax-local-identifier-as-binding on an identifier from the expansion before moving it into a binding position or comparing it with bound-identifier=?. Otherwise, the results can be inconsistent with the way that define works in the same definition context.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

Added in version 6.3 of package base.
Changed in version 8.2.0.7: Added the optional intdef-ctx argument.

This procedure must be called during the dynamic extent of a syntax transformer application by the expander or while a module is visited (see syntax-transforming?), otherwise the exn:fail:contract exception is raised.

The action of the generated procedure can be 'flip (the default) to flip the presence of a scope in each part of a given syntax object, 'add to add the scope to each regardless of whether it is present already, or 'remove to remove the scope when it is currently present in any part.

Multiple applications of the same make-syntax-introducer result procedure use the same scope, and different result procedures use distinct scopes.

Changed in version 6.3 of package base: Added the optional as-use-site? argument, and added the optional operation argument in the result procedure.

This function is intended for the implementation of separate binding spaces within a single phase, for which the scope associated with each environment must be the same across modules.

Unlike make-syntax-introducer, the scope added by a procedure created with make-interned-syntax-introducer is always treated like a use-site scope, not a macro-introduction scope, so it does not affect originalness as reported by syntax-original?.

Added in version 6.90.0.28 of package base.
Changed in version 8.2.0.4: Added the constraint that key is interned.

• If the scopes of base-stx are a subset of the scopes of ext-stx, then the result of make-syntax-delta-introducer adds, removes, or flips scopes that are in the set for ext-stx and not in the set for base-stx.

• If the scopes of base-stx are not a subset of the scopes of ext-stx, but if it has a binding, then the set of scopes associated with the binding id subtracted from the set of scopes for ext-stx, and the result of make-syntax-delta-introducer adds, removes, or flips that difference.

A #f value for base-stx is equivalent to a syntax object with no scopes.

This procedure is potentially useful when some m-id has a transformer binding that records some orig-id, and a use of m-id introduces a binding of orig-id. In that case, the scopes one the use of m-id added since the binding of m-id should be transferred to the binding instance of orig-id, so that it captures uses with the same lexical context as the use of m-id.

If ext-stx is tainted, then an identifier result from the created procedure is tainted.

It returns a hash table mapping a phase-level number (such as 0) to a list of all definitions at that phase level within the module being expanded. This information is used for implementing provide sub-forms like all-defined-out.

Beware that the phase-level keys are absolute relative to the enclosing module, and not relative to the current transformer phase level as reported by syntax-local-phase-level.

It returns an association list mapping phase level and binding space combinations to lists of identifiers. Each list of identifiers includes all bindings imported (into the module being expanded) using the module path mod-path, or all modules if mod-path is #f. The association list includes all identifiers imported with a phase level and binding space shift as represented by shift, or all shifts if shift is #t. If shift is not #t, the result can be #f if no identifiers are imported at that shift.

When an identifier is renamed on import, the result association list includes the identifier by its internal name. Use identifier-binding to obtain more information about the identifier.

Beware that the phase-level shifts are absolute relative to the enclosing module, and not relative to the current transformer phase level as reported by syntax-local-phase-level.

Changed in version 8.2.0.3 of package base: Generalized shift and result to phase–space combinations.

The liberal-define-context? predicate returns #t if v is an instance of a structure with a true value for the prop:liberal-define-context property, #f otherwise.

12.4.1 require Transformers

A transformer binding whose value is a structure with the prop:require-transformer property implements a derived require-spec for require as a require transformer.

A require transformer is called with the syntax object representing its use as a require-spec within a require form, and the result must be two lists: a list of imports and a list of import-sources.

If the derived form contains a sub-form that is a require-spec, then it can call expand-import to transform the sub-require-spec to lists of imports and import sources.

See also define-require-syntax, which supports macro-style require transformers.