1.2 Syntax Model

The syntax of a Racket program is defined by

• a read pass that processes a character stream into a syntax object; and

• an expand pass that processes a syntax object to produce one that is fully parsed.

For details on the read pass, see The Reader. Source code is normally read in read-syntax mode, which produces a syntax object.

The expand pass recursively processes a syntax object to produce a complete parse of the program. Binding information in a syntax object drives the expansion process, and when the expansion process encounters a binding form, it extends syntax objects for sub-expressions with new binding information.

1.2.1 Identifiers, Binding, and Scopes

Identifiers and Binding in The Racket Guide introduces binding.

An identifier is a source-program entity. Parsing (i.e., expanding) a Racket program reveals that some identifiers correspond to variables, some refer to syntactic forms (such as lambda, which is the syntactic form for functions), some refer to transformers for macro expansion, and some are quoted to produce symbols or syntax objects. An identifier binds another (i.e., it is a binding) when the former is parsed as a variable or syntactic form and the latter is parsed as a reference to the former; the latter is bound.

For example, as a fragment of source, the text

(let ([x 5]) x)

includes two identifiers: let and x (which appears twice). When this source is parsed in a context where let has its usual meaning, the first x binds the second x.

Bindings and references are determined through scope sets. A scope corresponds to a region of the program that is either in part of the source or synthesized through elaboration of the source. Nested binding contexts (such as nested functions) create nested scopes, while macro expansion creates scopes that overlap in more complex ways. Conceptually, each scope is represented by a unique token, but the token is not directly accessible. Instead, each scope is represented by a value that is internal to the representation of a program.

A form is a fragment of a program, such as an identifier or a function call. A form is represented as a syntax object, and each syntax object has an associated set of scopes (i.e., a scope set). In the above example, the representations of the xs include the scope that corresponds to the let form.

When a form parses as the binding of a particular identifier, parsing updates a global table that maps a combination of an identifier’s symbol and scope set to its meaning: a variable, a syntactic form, or a transformer. An identifier refers to a particular binding when the reference’s symbol and the identifier’s symbol are the same, and when the reference’s scope set is a superset of the binding’s scope set. For a given identifier, multiple bindings may have scope sets that are subsets of the identifier’s; in that case, the identifier refers to the binding whose set is a superset of all others; if no such binding exists, the reference is ambiguous (and triggers a syntax error if it is parsed as an expression). A binding shadows any binding (i.e., it is shadowing any binding) with the same symbol but a subset of scopes.

For example, in

(let ([x 5]) x)

in a context where let corresponds to the usual syntactic form, the parsing of let introduces a new scope for the binding of x. Since the second x receives that scope as part of the let body, the first x binds the second x. In the more complex case

(let ([x 5])

(let ([x 6])

x))

the inner let creates a second scope for the second x, so its scope set is a superset of the first x’s scope set—which means that the binding for the second x shadows the one for the first x, and the third x refers to the binding created by the second one.

A top-level binding is a binding from a definition at the top-level; a module binding is a binding from a definition in a module; all other bindings are local bindings. Within a module, references to top-level bindings are disallowed. An identifier without a binding is unbound.

Throughout the documentation, identifiers are typeset to suggest the way that they are parsed. A hyperlinked identifier like lambda indicates a reference to a syntactic form or variable. A plain identifier like x is a variable or a reference to an unspecified top-level variable.

Every binding has a phase level in which it can be referenced, where a phase level normally corresponds to an integer (but the special label phase level does not correspond to an integer). Phase level 0 corresponds to the run time of the enclosing module (or the run time of top-level expressions). Bindings in phase level 0 constitute the base environment. Phase level 1 corresponds to the time during which the enclosing module (or top-level expression) is expanded; bindings in phase level 1 constitute the transformer environment. Phase level -1 corresponds to the run time of a different module for which the enclosing module is imported for use at phase level 1 (relative to the importing module); bindings in phase level -1 constitute the template environment. The label phase level does not correspond to any execution time; it is used to track bindings (e.g., to identifiers within documentation) without implying an execution dependency.

An identifier can have different bindings in different phase levels. More precisely, the scope set associated with a form can be different at different phase levels; a top-level or module context implies a distinct scope at every phase level, while scopes from macro expansion or other syntactic forms are added to a form’s scope sets at all phases. The context of each binding and reference determines the phase level whose scope set is relevant.

Changed in version 6.3 of package base: Changed local bindings to have a specific phase level, like top-level and module bindings.

1.2.2 Syntax Objects

A syntax object combines a simpler Racket value, such as a symbol or pair, with lexical information, source-location information, syntax properties, and tamper status. The lexical information of a syntax object comprises a set of scope sets, one for each phase level. In particular, an identifier is represented as a syntax object containing a symbol, and its lexical information can be combined with the global table of bindings to determine its binding (if any) at each phase level.

For example, a car identifier might have lexical information that designates it as the car from the racket/base language (i.e., the built-in car). Similarly, a lambda identifier’s lexical information may indicate that it represents a procedure form. Some other identifier’s lexical information may indicate that it references a top-level variable.

When a syntax object represents a more complex expression than an identifier or simple constant, its internal components can be extracted. Even for extracted identifiers, detailed information about binding is available mostly indirectly; two identifiers can be compared to determine whether they refer to the same binding (i.e., free-identifier=?), or whether the identifiers have the same scope set so that each identifier would bind the other if one were in a binding position and the other in an expression position (i.e., bound-identifier=?).

For example, when the program written as

(let ([x 5]) (+ x 6))

is represented as a syntax object, then two syntax objects can be extracted for the two xs. Both the free-identifier=? and bound-identifier=? predicates will indicate that the xs are the same. In contrast, the let identifier is not free-identifier=? or bound-identifier=? to either x.

The lexical information in a syntax object is independent of the rest of the syntax object, and it can be copied to a new syntax object in combination with an arbitrary other Racket value. Thus, identifier-binding information in a syntax object is predicated on the symbolic name of the identifier as well as the identifier’s lexical information; the same question with the same lexical information but different base value can produce a different answer.

For example, combining the lexical information from let in the program above to 'x would not produce an identifier that is free-identifier=? to either x, since it does not appear in the scope of the x binding. Combining the lexical context of the 6 with 'x, in contrast, would produce an identifier that is bound-identifier=? to both xs.

The quote-syntax form bridges the evaluation of a program and the representation of a program. Specifically, (quote-syntax datum #:local) produces a syntax object that preserves all of the lexical information that datum had when it was parsed as part of the quote-syntax form. Note that the (quote-syntax datum) form is similar, but it removes certain scopes from the datum’s scope sets; see quote-syntax for more information.

1.2.3 Expansion (Parsing)

Expansion recursively processes a syntax object in a particular phase level, starting with phase level 0. Bindings from the syntax object’s lexical information drive the expansion process, and cause new bindings to be introduced for the lexical information of sub-expressions. In some cases, a sub-expression is expanded in a phase deeper (having a bigger phase level number) than the enclosing expression.

1.2.3.1 Fully Expanded Programs

A complete expansion produces a syntax object matching the following grammar:

Beware that the symbolic names of identifiers in a fully expanded program may not match the symbolic names in the grammar. Only the binding (according to free-identifier=?) matters.

A fully-expanded syntax object corresponds to a parse of a program (i.e., a parsed program), and lexical information on its identifiers indicates the parse.

More specifically, the typesetting of identifiers in the above grammar is significant. For example, the second case for expr is a syntax-object list whose first element is an identifier, where the identifier’s lexical information specifies a binding to the #%plain-lambda of the racket/base language (i.e., the identifier is free-identifier=? to one whose binding is #%plain-lambda). In all cases, identifiers above typeset as syntactic-form names refer to the bindings defined in Syntactic Forms.

In a fully expanded program for a namespace whose base phase is 0, the relevant phase level for a binding in the program is N if the binding has N surrounding begin-for-syntax and/or define-syntaxes forms—not counting any begin-for-syntax forms that wrap a module or module* form for the body of the module or module*, unless a module* form has #f in place of a module-path after the id. The datum in a quote-syntax form preserves its information for all phase levels.

A reference to a local binding in a fully expanded program has a scope set that matches its binding identifier exactly. Additional scopes, if any, are removed. As a result, bound-identifier=? can be used to correlate local binding identifiers with reference identifiers, while free-identifier=? must be used to relate references to module bindings or top-level bindings.

In addition to the grammar above, #%expression can appear in a fully local-expanded expression position. For example, #%expression can appear in the result from local-expand when the stop list is empty. Reference-identifier scope sets are reduced in local-expanded expressions only when the local-expand stop list is empty.

Changed in version 6.3 of package base: Added the #:local variant of quote-syntax; removed letrec-syntaxes+values from possibly appearing in a fully local-expanded form.

1.2.3.2 Expansion Steps

In a recursive expansion, each single step in expanding a syntax object at a particular phase level depends on the immediate shape of the syntax object being expanded:

• If it is an identifier (i.e., a syntax-object symbol), then a binding is determined by the identifier’s lexical information. If the identifier has a binding, that binding is used to continue. If the identifier is unbound, a new syntax-object symbol '#%top is created using the lexical information of the identifier with implicit-made-explicit properties; if this #%top identifier has no binding, then parsing fails with an exn:fail:syntax exception. Otherwise, the new identifier is combined with the original identifier in a new syntax-object pair (also using the same lexical information as the original identifier), and the #%top binding is used to continue.

  Changed in version 6.3 of package base: Changed the introduction of #%top in a top-level context to unbound identifiers only.

• If it is a syntax-object pair whose first element is an identifier, and if the identifier has a binding other than as a top-level variable, then the identifier’s binding is used to continue.

• If it is a syntax-object pair of any other form, then a new syntax-object symbol '#%app is created using the lexical information of the pair with implicit-made-explicit properties. If the resulting #%app identifier has no binding, parsing fails with an exn:fail:syntax exception. Otherwise, the new identifier is combined with the original pair to form a new syntax-object pair (also using the same lexical information as the original pair), and the #%app binding is used to continue.

• If it is any other syntax object, then a new syntax-object symbol '#%datum is created using the lexical information of the original syntax object with implicit-made-explicit properties. If the resulting #%datum identifier has no binding, parsing fails with an exn:fail:syntax exception. Otherwise, the new identifier is combined with the original syntax object in a new syntax-object pair (using the same lexical information as the original pair), and the #%datum binding is used to continue.

Thus, the possibilities that do not fail lead to an identifier with a particular binding. This binding refers to one of three things:

• A transformer, such as introduced by define-syntax or let-syntax. If the associated value is a procedure of one argument, the procedure is called as a syntax transformer (described below), and parsing starts again with the syntax-object result. If the transformer binding is to any other kind of value, parsing fails with an exn:fail:syntax exception. The call to the syntax transformer is parameterized to set current-namespace to a namespace that shares bindings and variables with the namespace being used to expand, except that its base phase is one greater.

• A variable binding, such as introduced by a module-level define or by let. In this case, if the form being parsed is just an identifier, then it is parsed as a reference to the corresponding variable. If the form being parsed is a syntax-object pair, then an #%app is added to the front of the syntax-object pair in the same way as when the first item in the syntax-object pair is not an identifier (third case in the previous enumeration), and parsing continues.

• A core syntactic form, which is parsed as described for each form in Syntactic Forms. Parsing a core syntactic form typically involves recursive parsing of sub-forms, and may introduce bindings that determine the parsing of sub-forms.

When a #%top, #%app, or #%datum identifier is added by the expander, it is given implicit-made-explicit properties: an 'implicit-made-explicit syntax property whose value is #t, and a hidden property to indicate that the implicit identifier is original in the sense of syntax-original? if the syntax object that gives the identifier its lexical information has that property.

Changed in version 7.9.0.13 of package base: Added implicit-made-explicit properties.

1.2.3.3 Expansion Context

Each expansion step occurs in a particular context, and transformers and core syntactic forms may expand differently for different contexts. For example, a module form is allowed only in a top-level context or module context, and it fails in other contexts. The possible contexts are as follows:

• top-level context : outside of any module, definition, or expression, except that sub-expressions of a top-level begin form are also expanded as top-level forms.

• module-begin context : inside the body of a module, as the only form within the module.

• module context : in the body of a module (inside the module-begin layer).

• internal-definition context : in a nested context that allows both definitions and expressions.

• expression context : in a context where only expressions are allowed.

Different core syntactic forms parse sub-forms using different contexts. For example, a let form always parses the right-hand expressions of a binding in an expression context, but it starts parsing the body in an internal-definition context.

1.2.3.4 Introducing Bindings

Bindings are introduced during expansion when certain core syntactic forms are encountered:

• When a require form is encountered at the top level or module level, each symbol specified by the form is paired with the scope set of the specification to introduce new bindings. If not otherwise indicated in the require form, bindings are introduced at the phase levels specified by the exporting modules: phase level 0 for each normal provide, phase level 1 for each for-syntax provide, and so on. The for-meta provide form allows exports at an arbitrary phase level (as long as a binding exists within the module at the phase level).

  A for-syntax sub-form within require imports similarly, but the resulting bindings have a phase level that is one more than the exported phase levels, when exports for the label phase level are still imported at the label phase level. More generally, a for-meta sub-form within require imports with the specified phase level shift; if the specified shift is #f, or if for-label is used to import, then all bindings are imported into the label phase level.

• When a define, define-values, define-syntax, or define-syntaxes form is encountered at the top level or module level, a binding is added to phase level 0 (i.e., the base environment is extended) for each defined identifier.

• When a begin-for-syntax form is encountered at the top level or module level, bindings are introduced as for define-values and define-syntaxes, but at phase level 1 (i.e., the transformer environment is extended). More generally, begin-for-syntax forms can be nested, and each begin-for-syntax shifts its body by one phase level.

• When a let-values form is encountered, the body of the let-values form is extended (by creating new syntax objects) with a fresh scope. The scope is added to the identifiers themselves, so that the identifiers in binding position are bound-identifier=? to uses in the fully expanded form, and so they are not bound-identifier=? to other identifiers. The new bindings are at the phase level at which the let-values form is expanded.

• When a letrec-values or letrec-syntaxes+values form is encountered, bindings are added as for let-values, except that the right-hand-side expressions are also extended with the new scope.

• Definitions in internal-definition contexts introduce new scopes and bindings as described in Internal Definitions.

For example, in

(let-values ([(x) 10]) (+ x y))

the binding introduced for x applies to the x in the body, because a fresh scope is created and added to both the binding x and reference x. The same scope is added to the y, but since it has a different symbol than the binding x, it does not refer to the new binding. Any x outside of this let-values form does not receive the fresh scope and therefore does not refer to the new binding.

1.2.3.5 Transformer Bindings

In a top-level context or module context, when the expander encounters a define-syntaxes form, the binding that it introduces for the defined identifiers is a transformer binding. The value of the binding exists at expansion time, rather than run time (though the two times can overlap), though the binding itself is introduced with phase level 0 (i.e., in the base environment).

The value for the binding is obtained by evaluating the expression in the define-syntaxes form. This expression must be expanded (i.e., parsed) before it can be evaluated, and it is expanded at phase level 1 (i.e., in the transformer environment) instead of phase level 0.

If the resulting value is a procedure of one argument or the result of make-set!-transformer on a procedure, then it is used as a syntax transformer (a.k.a. macro). The procedure is expected to accept a syntax object and return a syntax object. A use of the binding (at phase level 0) triggers a call of the syntax transformer by the expander; see Expansion Steps.

Before the expander passes a syntax object to a transformer, the syntax object is extended with a fresh macro-introduction scope (that applies to all sub-syntax objects) to distinguish syntax objects at the macro’s use site from syntax objects that are introduced by the macro; in the result of the transformer the presence of the scope is flipped, so that introduced syntax objects retain the scope, and use-site syntax objects do not have it. In addition, if the use of a transformer is in the same definition context as its binding, the use-site syntax object is extended with an additional fresh use-site scope that is not flipped in the transformer’s result, so that only use-site syntax objects have the use-site scope.

The scope-introduction process for macro expansion helps keep binding in an expanded program consistent with the lexical structure of the source program. For example, the expanded form of the program

(define x 12)

(define-syntax m

(syntax-rules ()

[(_ id) (let ([x 10]) id)]))

(m x)

is

(define x 12)

(define-syntax m ....)

(let ([x 10]) x)

However, the result of the last expression is 12, not 10. The reason is that the transformer bound to m introduces the binding x, but the referencing x is present in the argument to the transformer. The introduced x is left with one fresh scope, while the reference x has a different fresh scope, so the binding x is not bound-identifier=? to the body x.

A use-site scope on a binding identifier is ignored when the definition is in the same context where the use-site scope was introduced. This special treatment of use-site scopes allows a macro to expand to a visible definition. For example, the expanded form of the program

(define-syntax m

(syntax-rules ()

[(_ id) (define id 5)]))

(m x)

x

is

(define-syntax m ....)

(define x 5)

x

where the x in the define form has a use-site scope that is not present on the final x. The final x nevertheless refers to the definition, because the use-site scope is effectively removed before installing the definition’s binding. In contrast, the expansion of

(define-syntax m

(syntax-rules ()

[(_ id) (let ([x 4])

(let ([id 5])

x))]))

(m x)

is

(define-syntax m ....)

(let ([x 4])

(let ([x 5])

x))

where the second x has a use-site scope that prevents it from binding the final x. The use-site scope is not ignored in this case, because the binding is not part of the definition context where (m x) was expanded.

The set! form works with the make-set!-transformer and prop:set!-transformer property to support assignment transformers that transform set! expressions. An assignment transformer contains a procedure that is applied by set! in the same way as a normal transformer by the expander.

The make-rename-transformer procedure or prop:rename-transformer property creates a value that is also handled specially by the expander and by set! as a transformer binding’s value. When id is bound to a rename transformer produced by make-rename-transformer, it is replaced with the target identifier passed to make-rename-transformer. In addition, as long as the target identifier does not have a true value for the 'not-free-identifier=? syntax property, the binding table is extended to indicate that id is an alias for the identifier in the rename transformer. The free-identifier=? function follows aliasing chains to determine equality of bindings, the identifier-binding function similarly follows aliasing chains, and the provide form exports id as the target identifier. Finally, the syntax-local-value function follows rename transformer chains even when binding aliases are not installed.

In addition to using scopes to track introduced identifiers, the expander tracks the expansion history of a form through syntax properties such as 'origin. See Syntax Object Properties for more information.

Finally, the expander uses a tamper status to control the way that unexported and protected module bindings are used. See Syntax Taints for more information on a tamper status.

The expander’s handling of letrec-syntaxes+values is similar to its handling of define-syntaxes. A letrec-syntaxes+values can be expanded in an arbitrary phase level n (not just 0), in which case the expression for the transformer binding is expanded at phase level n+1.

The expressions in a begin-for-syntax form are expanded and evaluated in the same way as for define-syntaxes. However, any introduced bindings from definition within begin-for-syntax are at phase level 1 (not a transformer binding at phase level 0).

1.2.3.6 Local Binding Context

Although the binding of an identifier can be uniquely determined from the combination of its lexical information and the global binding table, the expander also maintains a local binding context that records additional information about local bindings to ensure they are not used outside of the lexical region in which they are bound.

Due to the way local binding forms like let add a fresh scope to both bound identifiers and body forms, it isn’t ordinarily possible for an identifier to reference a local binding without appearing in the body of the let. However, if macros use compile-time state to stash bound identifiers, or use local-expand to extract identifiers from an expanded binding form, they can violate this constraint. For example, the following stash-id and unstash-id macros cooperate to move a reference to a locally-bound x identifier outside of the lexical region in which it is bound:

> (begin-for-syntax     (define stashed-id #f))

> (define-syntax (stash-id stx)     (syntax-case stx ()       [(_ id)        (begin          (set! stashed-id #'id)          #'(void))]))

> (define-syntax (unstash-id stx)     stashed-id)

> (let ([x 42])     (stash-id x)     (unstash-id))

42

> (unstash-id)

identifier used out of context: #<syntax:eval:5:0 x>

In general, an identifier’s lexical information is not sufficient to know whether or not its binding is available in the enclosing context, since the scope set for the identifier stored in stashed-id unambiguously refers to a binding in the global binding table. This can be observed by the fact that identifier-binding produces 'lexical, not #f:

> (define-syntax (stashed-id-binding stx)     #`'#,(identifier-binding stashed-id))

> (stashed-id-binding)

'lexical

However, the reference produced by (unstash-id) in the above program is still illegal, even if it isn’t technically unbound. To record the fact that x’s binding is in scope only within the body of its corresponding let form, the expander adds x’s binding to the local binding context while expanding the let body. More generally, the expander adds all local variable bindings to the local binding context while expanding expressions in which a reference to the variable would be legal. When the expander encounters an identifier bound to a local variable, and the associated binding is not in the current local binding context, it raises a syntax error.

The local binding context also tracks local transformer bindings (i.e. bindings bound by forms like let-syntax) in a similar way, except that the context also stores the compile-time value associated with the transformer. When an identifier that is locally bound as a transformer is used in application position as a syntax transformer, or its compile-time value is looked up using syntax-local-value, the local binding context is consulted to retrieve the value. If the binding is in scope, its associated compile-time value is used; otherwise, the expander raises a syntax error.

1.2.3.7 Partial Expansion

In certain contexts, such as an internal-definition context or module context, partial expansion is used to determine whether forms represent definitions, expressions, or other declaration forms. Partial expansion works by cutting off the normal recursive expansion when the relevant binding is for a primitive syntactic form.

As a special case, when expansion would otherwise add an #%app, #%datum, or #%top identifier to an expression, and when the binding turns out to be the primitive #%app, #%datum, or #%top form, then expansion stops without adding the identifier.

1.2.3.8 Internal Definitions

An internal-definition context supports local definitions mixed with expressions. Forms that allow internal definitions document such positions using the body meta-variable. Definitions in an internal-definition context are equivalent to local binding via letrec-syntaxes+values; macro expansion converts internal definitions to a letrec-syntaxes+values form.

Expansion relies on partial expansion of each body in an internal-definition sequence. Partial expansion of each body produces a form matching one of the following cases:

• A define-values form: The binding table is immediately enriched with bindings for the define-values form. Further expansion of the definition is deferred, and partial expansion continues with the rest of the body.

• A define-syntaxes form: The right-hand side is expanded and evaluated (as for a letrec-syntaxes+values form), and a transformer binding is installed for the body sequence before partial expansion continues with the rest of the body.

• A primitive expression form other than begin: Further expansion of the expression is deferred, and partial expansion continues with the rest of the body.

• A begin form: The sub-forms of the begin are spliced into the internal-definition sequence, and partial expansion continues with the first of the newly-spliced forms (or the next form, if the begin had no sub-forms).

After all body forms are partially expanded, if no definitions were encountered, then the expressions are collected into a begin form as the internal-definition context’s expansion. Otherwise, at least one expression must appear after the last definition, and any expr that appears between definitions is converted to (define-values () (begin expr (values))); the definitions are then converted to bindings in a letrec-syntaxes+values form, and all expressions after the last definition become the body of the letrec-syntaxes+values form.

Before partial expansion begins, expansion of an internal-definition context begins with the introduction of a fresh outside-edge scope on the content of the internal-definition context. This outside-edge scope effectively identifies syntax objects that are present in the original form. An inside-edge scope is also created and added to the original content; furthermore, the inside-edge scope is added to the result of any partial expansion. This inside-edge scope ensures that all bindings introduced by the internal-definition context have a particular scope in common.

1.2.3.9 Module Expansion, Phases, and Visits

Expansion of a module form proceeds in a similar way to expansion of an internal-definition context: an outside-edge scope is created for the original module content, and an inside-edge scope is added to both the original module and any form that appears during a partial expansion of the module’s top-level forms to uncover definitions and imports.

A require form not only introduces bindings at expansion time, but also visits the referenced module when it is encountered by the expander. That is, the expander instantiates any variables defined in the module within begin-for-syntax, and it also evaluates all expressions for define-syntaxes transformer bindings.

Module visits propagate through requires in the same way as module instantiation. Moreover, when a module is visited at phase 0, any module that it requires for-syntax is instantiated at phase 1, while further requires for-template leading back to phase 0 causes the required module to be visited at phase 0 (i.e., not instantiated).

During compilation, the top-level of module context is itself implicitly visited. Thus, when the expander encounters (require (for-syntax ....)), it immediately instantiates the required module at phase 1, in addition to adding bindings at phase level 1 (i.e., the transformer environment). Similarly, the expander immediately evaluates any form that it encounters within begin-for-syntax.

Phases beyond 0 are visited on demand. For example, when the right-hand side of a phase-0 let-syntax is to be expanded, then modules that are available at phase 1 are visited. More generally, initiating expansion at phase n visits modules at phase n, which in turn instantiates modules at phase n+1. These visits and instantiations apply to available modules in the enclosing namespace’s module registry; a per-registry lock prevents multiple threads from concurrently instantiating and visiting available modules. On-demand instantiation of available modules uses the same reentrant lock as namespace-call-with-registry-lock.

When the expander encounters require and (require (for-syntax ....)) within a module context, the resulting visits and instantiations are specific to the expansion of the enclosing module, and are kept separate from visits and instantiations triggered from a top-level context or from the expansion of a different module. Along the same lines, when a module is attached to a namespace through namespace-attach-module, modules that it requires are transitively attached, but instances are attached only at phases at or below the namespace’s base phase.

1.2.3.10 Macro-Introduced Bindings

When a top-level definition binds an identifier that originates from a macro expansion, the definition captures only uses of the identifier that are generated by the same expansion due to the fresh scope that is generated for the expansion.

For a top-level definition (outside of a module), the order of evaluation affects the binding of a generated definition for a generated identifier use. If the use precedes the definition, then the use is resolved with the bindings that are in place at that point, which will not include the binding from the subsequently macro-generated definition. (No such dependency on order occurs within a module, since a module binding covers the entire module body.) To support the declaration of an identifier before its use, the define-syntaxes form avoids binding an identifier if the body of the define-syntaxes declaration produces zero results.

Macro-generated require and provide clauses also introduce and reference generation-specific bindings (due to the added scope) with the same ordering effects as for definitions. The bindings depend on the scope set attached to specific parts of the form:

• In require, for a require-spec of the form (rename-in [orig-id bind-id]) or (only-in .... [orig-id bind-id]), the bind-id supplies the scope set for the binding. In require for other require-specs, the generator of the require-spec determines the scope set.

• In provide, for a provide-spec of the form id, the exported identifier is the one that binds id, but the external name is the plain, symbolic part of id. The exceptions for all-except-out are similarly determined, as is the orig-id binding of a rename-out form, and plain symbols are used for the external names. For all-defined-out, only identifiers with definitions having only the scopes of (all-defined-out) form are exported; the external name is the plain symbol from the definition.

1.2.4 Compilation

Before expanded code is evaluated, it is first compiled. A compiled form has essentially the same information as the corresponding expanded form, though the internal representation naturally dispenses with identifiers for syntactic forms and local bindings. One significant difference is that a compiled form is almost entirely opaque, so the information that it contains cannot be accessed directly (which is why some identifiers can be dropped). At the same time, a compiled form can be marshaled to and from a byte string, so it is suitable for saving and re-loading code.

Although individual read, expand, compile, and evaluate operations are available, the operations are often combined automatically. For example, the eval procedure takes a syntax object and expands it, compiles it, and evaluates it.

1.2.5 Namespaces

See Namespaces for functions that manipulate namespaces.

A namespace is both a starting point for parsing and a starting point for running compiled code. A namespace also has a module registry that maps module names to module declarations (see Modules and Module-Level Variables). This registry is shared by all phase levels, and it applies both to parsing and to running compiled code.

As a starting point for parsing, a namespace provides scopes (one per phase level, plus one that spans all phase levels). Operations such as namespace-require create initial bindings using the namespace’s scopes, and the further expansion and evaluation in the namespace can create additional bindings. Evaluation of a form with a namespace always adds the namespace’s phase-specific scopes to the form and to the result of expanding a top-level form; as a consequence, every binding identifier has at least one scope. The namespace’s additional scope is added only on request (e.g., by using eval as opposed to eval-syntax); if requested, the additional scope is added at all phase levels. Except for namespaces generated by a module (see module->namespace), every namespace uses the same scope as the one added to all phase levels, while the scopes specific to a phase level are always distinct.

As a starting point for evaluating compiled code, each namespace encapsulates a distinct set of top-level variables at various phases, as well as a potentially distinct set of module instances in each phase. That is, even though module declarations are shared for all phase levels, module instances are distinct for each phase. Each namespace has a base phase, which corresponds to the phase used by reflective operations such as eval and dynamic-require. In particular, using eval on a require form instantiates a module in the namespace’s base phase.

After a namespace is created, module instances from existing namespaces can be attached to the new namespace. In terms of the evaluation model, top-level variables from different namespaces essentially correspond to definitions with different prefixes, but attaching a module uses the same prefix for the module’s definitions in namespaces where it is attached. The first step in evaluating any compiled expression is to link its top-level variable and module-level variable references to specific variables in the namespace.

At all times during evaluation, some namespace is designated as the current namespace. The current namespace has no particular relationship, however, with the namespace that was used to expand the code that is executing, or with the namespace that was used to link the compiled form of the currently evaluating code. In particular, changing the current namespace during evaluation does not change the variables to which executing expressions refer. The current namespace only determines the behavior of reflective operations to expand code and to start evaluating expanded/compiled code.

If an identifier is bound to syntax or to an import, then defining the identifier as a variable shadows the syntax or import in future uses of the environment. Similarly, if an identifier is bound to a top-level variable, then binding the identifier to syntax or an import shadows the variable; the variable’s value remains unchanged, however, and may be accessible through previously evaluated expressions.

Like a top-level namespace, each module form has an associated scope to span all phase levels of the module’s content, plus a scope at each phase level. The latter is added to every form, original or appearing through partial macro expansion, within the module’s immediate body. Those same scopes are propagated to a namespace created by module->namespace for the module. Meanwhile, parsing of a module form begins by removing the all scopes that correspond to the enclosing top-level or (in the case of submodules) module and module* forms.

1.2.6 Inferred Value Names

To improve error reporting, names are inferred at compile-time for certain kinds of values, such as procedures. For example, evaluating the following expression:

(let ([f (lambda () 0)]) (f 1 2 3))

produces an error message because too many arguments are provided to the procedure. The error message is able to report f as the name of the procedure. In this case, Racket decides, at compile-time, to name as 'f all procedures created by the let-bound lambda.

(define my-f

(let ([f (lambda () 0)]) f))

the procedure bound to my-f will have the inferred name 'f.

When an 'inferred-name property is attached to a syntax object for an expression (see Syntax Object Properties), the property value is used for naming the expression, and it overrides any name that was inferred from the expression’s context. Normally, the property value should be a symbol. A 'inferred-name property value of #<void> hides a name that would otherwise be inferred from context (perhaps to avoid exposing an identifier from an automatically generated binding).

To support the propagation and merging of consistent properties during expansions, the value of the 'inferred-name property can be a tree formed with cons where all of the leaves are the same. For example, (cons 'name 'name) is equivalent to 'name, and (cons (void) (void)) is equivalent to #<void>.

When an inferred name is not available, but a source location is available, a name is constructed using the source location information. Inferred and property-assigned names are also available to syntax transformers, via syntax-local-name.

1.2.7 Cross-Phase Persistent Module Declarations

A module is cross-phase persistent only if it fits the following grammar, which uses non-terminals from Fully Expanded Programs, only if it includes (#%declare #:cross-phase-persistent), only it includes no uses of quote-syntax or #%variable-reference, and only if no module-level binding is set!ed.