14.14 Linklets and the Core Compiler

A linklet is a primitive element of compilation, bytecode marshaling, and evaluation. Racket’s implementations of modules, macros, and top-level evaluation are all built on linklets. Racket programmers generally do not encounter linklets directly, but the racket/linklet library provides access to linklet facilities.

A single Racket module (or collection of top-level forms) is typically implemented by multiple linklets. For example, each phase of evaluation that exists in a module is implemented in a separate linklet. A linklet is also used for metadata such as the module path indexes for a module’s requires. These linklets, plus some other metadata, are combined to form a linklet bundle. Information in a linklet bundle is keyed by either a symbol or a fixnum. A linklet bundle containing linklets can be marshaled to and from a byte stream by write and (with read-accept-compiled is enabled) read.

When a Racket module has submodules, the linklet bundles for the module and the submodules are grouped together in a linklet directory. A linklet directory can have nested linklet directories. Information in a linklet directory is keyed by #f or a symbol, where #f must be mapped to a linklet bundle (if anything) and each symbol must be mapped to a linklet directory. A linklet directory can be equivalently viewed as a mapping from a lists of symbols to a linklet bundle. Like linklet bundles, a linklet directory can be marshaled to and from a byte stream by write and read; the marshaled form allows individual linklet bundles to be loaded independently.

A linklet consists of a set of variable definitions and expressions, an exported subset of the defined variable names, a set of variables to export from the linklet despite having no corresponding definition, and a set of imports that provide other variables for the linklet to use. To run a linklet, it is instantiated as as linklet instance (or just instance, for short). When a linklet is instantiated, it receives other linklet instances for its imports, and it extracts a specified set of variables that are exported from each of the given instances. The newly created linklet instance provides its exported variables for use by other linklets or for direct access via instance-variable-value. A linklet instance can be synthesized directly with make-instance.

A linklet is created by compiling an enriched S-expression representation of its source. Since linklets exist below the layer of macros and syntax objects, linklet compilation does not use syntax objects. Instead, linklet compilation uses correlated objects, which are like syntax objects without lexical-context information and without the constraint that content is coerced to correlated objects. Using an S-expression or correlated object, the grammar of a linklet as recognized by compile-linklet is

(linklet [[imported-id/renamed ...] ...]          [exported-id/renamed ...]   defn-or-expr ...)

imported-id/renamed   =   imported-id     |   (external-imported-id internal-imported-id)           exported-id/renamed   =   exported-id     |   (internal-exported-id external-exported-id)

Each import set [imported-id/renamed ...] refers to a single imported instance, and each import-id/renamed corresponds to a variable from that instance. If separate external-imported-id and internal-imported-id are specified, then external-imported-id is the name of the variable as exported by the instance, and internal-imported-id is the name used to refer to the variable in the defn-or-exprs. For exports, separate internal-exported-id and external-exported-id names corresponds to the variable name as exported as referenced in the defn-or-exprs, respectively.

The grammar of an defn-or-expr is similar to the expander’s grammar of fully expanded expressions (see Fully Expanded Programs) with some exceptions: quote-syntax and #%top are not allowed; #%plain-lambda is spelled lambda; #%plain-app is omitted (i.e., application is implicit); lambda, case-lambda, let-values, and letrec-values can have only a single body expression; and numbers, booleans, strings, and byte strings are self-quoting. Primitives are accessed directly by name, and shadowing is not allowed within a linklet form for primitive names, imported variables, defined variables, or local variables.

When an exported-id/renamed has no corresponding definition among the defn-or-exprs, then the variable is effectively defined as uninitialized; referencing the variable will trigger exn:fail:contract:variable, the same as referencing a variable before it is defined. When a target instance is provided to instantiate-linklet, any existing variable with the same name will be left as-is, instead of set to undefined. This treatment of uninitialized variables provides core support for top-level evaluation where variables may be referenced and then defined in a separate element of compilation.

Added in version 6.90.0.1 of package base.

The optional name is associated to the linklet for debugging purposes and as the default name of the linklet’s instance.

The optional import-keys and get-import arguments support cross-linklet optimization. If import-keys is a vector, it must have as many elements as sets of imports in form. If the compiler becomes interested in optimizing a reference to an imported variable, it passes back to get-import (if non-#f) the element of import-keys that corresponds to the variable’s import set. The get-import function can then return a linklet or instance that represents an instance to be provided to the compiled linklet when it is eventually instantiated; ensuring consistency between reported linklet or instance and the eventual instance is up to the caller of compile-linklet. If get-import returns #f as its first value, the compiler will be prevented from making any assumptions about the imported instance. The second result from get-import is an optional vector of keys to provide transitive information on a returned linklet’s imports (and is not allowed for a returned instance); the returned vector must have the same number of elements as the linklet has imports. When vector elements are eq? and non-#f, the compiler can assume that they correspond to the same run-time instance. A #f value for get-import is equivalent to a function that always returns two #f results.

When import-keys is not #f, then the compiler is allowed to grow or shrink the set of imported instances for the linklet. The result vector specifies the keys of the imports for the returned linklet. Any key that is #f or a linklet instance must be preserved intact, however.

If 'unsafe is included in options, then the linklet is compiled in unsafe mode: uses of safe operations within the linklet can be converted to unsafe operations on the assumption that the relevant contracts are satisfied. For example, car is converted to unsafe-car. Some substituted unsafe operations may not have directly accessible names, such as the unsafe variant of in-list that can be substituted in unsafe mode. An unsafe operation is substituted only if its (unchecked) contract is subsumed by the safe operation’s contract. The fact that the linklet is compiled in unsafe mode can be exposed through variable-reference-from-unsafe? using a variable reference produced by a #%variable-reference form within the module body.

If 'static is included in options, then the linklet must be instantiated only once; if the linklet is serialized, then any individual instance read from the serialized form must be instantiated at most once. Compilation with 'static is intended to improve the performance of references within the linklet to defined and imported variables.

If 'use-prompt is included in options, then instantiating resulting linklet always wraps a prompt around each definition and immediate expression in the linklet. Otherwise, supplying #t as the use-prompt? argument to instantiate-linklet may only wrap a prompt around the entire instantiation.

If 'uninterned-literal is included in options, then literals in form will not necessarily be interned via datum-intern-literal when compiling or loading the linklet. Disabling the use of datum-intern-literal can be especially useful of the linklet includes a large string or byte string constant that is not meant to be shared.

The symbols in options must be distinct, otherwise exn:fail:contract exception is raised.

Changed in version 7.1.0.8 of package base: Added the 'use-prompt option.
Changed in version 7.1.0.10: Added the 'uninterned-literal option.

Changed in version 7.1.0.6 of package base: Added the options argument.
Changed in version 7.1.0.8: Added the 'use-prompt option.
Changed in version 7.1.0.10: Added the 'uninterned-literal option.

If target-instance is #f or not provided, the result is a fresh instance for the linklet. If target-instance is an instance, then the instance is used and modified for the linklet definitions and expressions, and the result is the value of the last expression in the linklet.

The linklet’s exported variables are accessible in the result instance or in target-instance using the linklet’s external name for each export. If target-instance is provided as non-#f, its existing variables remain intact if they are not modified by a linklet definition.

If use-prompt? is true, then a a prompt is wrapped around the linklet instantiation in same ways as an expression in a module body. If the linklet contains multiple definitions or immediate expressions, then a prompt may or may not be wrapped around each definition or expression; supply 'use-prompt to compile-linklet to ensure that a prompt is used around each definition and expression.

The optional data and mode arguments must be provided if any variable-name and variable-value arguments are provided. The mode argument is used as in instance-set-variable-value! for every variable-name.

If mode is 'constant or 'consistent, then the variable is created or changed to be constant. Furthermore, when the instance is reported for a linklet’s import though a get-import callback to compile-linklet, the compiler can assume that the variable will be constant in all future instances that are used to satisfy a linklet’s imports.

If mode is 'consistent, when the instance is reported though a callback to compile-linklet, the compiler can further assume that the variable’s value will be the same for future instances. For compilation purposes, “the same” can mean that a procedure value will have the same arity and implementation details, a structure type value will have the same configuration, a marshalable constant will be equal? to the current value, and so on.

• `(procedure ,arity-mask) — the value is always a procedure that is not impersonated and not a structure, and its arity in the style of procedure-arity-mask is arity-mask.

• `(procedure/succeeds ,arity-mask) — like `(procedure ,arity-mask), but for a procedure that never raises an exception of otherwise captures or escapes the calling context.

• `(procedure/pure ,arity-mask) — like `(procedure/succeeds ,arity-mask), but with no observable side effects, so a call to the procedure can be reordered.

Added in version 7.1.0.8 of package base.

When ref-site? is #f, the result is #f when varref is from (#%variable-reference) with no identifier. The result is a symbol if varref refers to a primitive.

Unlike datum->syntax, datum->correlated does not recur through the given S-expression and convert pieces to correlated objects. Instead, a correlated object is simply wrapped around the immediate value. In contrast, correlated->datum recurs through its argument (which is not necessarily a correlated object) to discover any correlated objects and convert them to plain S-expressions.

Changed in version 7.6.0.6 of package base: Added the prop argument to datum->correlated.