1 raco make: Compiling Source to Bytecode
The raco make command accept filenames for Racket modules to be compiled to bytecode format. Modules are re-compiled only if the source Racket file is newer than the bytecode file and has a different SHA-1 hash, or if any imported module is recompiled or has a different SHA-1 hash for its compiled form plus dependencies.
1.1 Bytecode Files
A file "‹name›.‹ext›" is compiled to bytecode that is saved as "compiled/‹name›_‹ext›.zo" relative to the file. As a result, the bytecode file is normally used automatically when "‹name›.‹ext›" is required as a module, since the underlying load/use-compiled operation detects such a bytecode file.
For example, in a directory that contains the following files:
#lang racket (require "b.rkt" "c.rkt") (+ b c)
#lang racket (provide b) (define b 1)
#lang racket (provide c) (define c 1)
raco make a.rkt
triggers the creation of "compiled/a_rkt.zo", "compiled/b_rkt.zo", and "compiled/c_rkt.zo". A subsequent
loads bytecode from the generated ".zo" files, paying attention to the ".rkt" sources only to confirm that each ".zo" file has a later timestamp.
racket b.rkt c.rkt
would create only "compiled/b_rkt.zo" and "compiled/c_rkt.zo", since neither "b.rkt" nor "c.rkt" imports "a.rkt".
1.2 Dependency Files
In addition to a bytecode file, raco make creates a file "compiled/‹name›_‹ext›.dep" that records dependencies of the compiled module on other module files and the source file’s SHA-1 hash. Using this dependency information, a re-compilation request via raco make can consult both the source file’s timestamp/hash and the timestamps/hashes for the bytecode of imported modules. Furthermore, imported modules are themselves compiled as necessary, including updating the bytecode and dependency files for the imported modules, transitively.
Continuing the raco make a.rkt example from the previous section, the raco make command creates "compiled/a_rkt.dep", "compiled/b_rkt.dep", and "compiled/c_rkt.dep" at the same time as the ".zo" files. The "compiled/a_rkt.dep" file records the dependency of "a.rkt" on "b.rkt", "c.rkt" and the racket library. If the "b.rkt" file is modified (so that its timestamp and SHA-1 hash changes), then running
raco make a.rkt
again rebuilds "compiled/a_rkt.zo" and "compiled/b_rkt.zo".
For module files that are within library collections, raco setup uses the same ".zo" and ".dep" conventions and files as raco make, so the two tools can be used together.
1.3 API for Making Bytecode
(make-compilation-manager-load/use-compiled-handler [ delete-zos-when-rkt-file-does-not-exist? #:security-guard security-guard]) → (path? (or/c symbol? false/c) . -> . any) delete-zos-when-rkt-file-does-not-exist? : any/c = #f security-guard : (or/c security-guard? #f) = #f
the file is expected to contain a module (i.e., the second argument to the handler is a symbol);
the value of each of (current-eval), (current-load), and (namespace-module-registry (current-namespace)) is the same as when make-compilation-manager-load/use-compiled-handler was called;
the value of use-compiled-file-paths contains the first path that was present when make-compilation-manager-load/use-compiled-handler was called;
the value of current-load/use-compiled is the result of this procedure; and
one of the following holds:
the source file is newer than the ".zo" file in the first sub-directory listed in use-compiled-file-paths (at the time that make-compilation-manager-load/use-compiled-handler was called), and either no ".dep" file exists or it records a source-file SHA-1 hash that differs from the current version and source-file SHA-1 hash;
no ".dep" file exists next to the ".zo" file;
the version recorded in the ".dep" file does not match the result of (version);
one of the files listed in the ".dep" file has a ".zo" timestamp newer than the target ".zo", and the combined hashes of the dependencies recorded in the ".dep" file does not match the combined hash recorded in the ".dep" file.
If SHA-1 hashes override a timestamp-based decision to recompile the file, then the target ".zo" file’s timestamp is updated to the current time.
After the handler procedure compiles a ".zo" file, it creates a corresponding ".dep" file that lists the current version and the identification of every file that is directly required by the module in the compiled file. Additional dependencies can be installed during compilation via compiler/cm-accomplice. The ".dep" file also records the SHA-1 hash of the module’s source, and it records a combined SHA-1 hash of all of the dependencies that includes their recursive dependencies.
The handler caches timestamps when it checks ".dep" files, and the cache is maintained across calls to the same handler. The cache is not consulted to compare the immediate source file to its ".zo" file, which means that the caching behavior is consistent with the caching of the default module name resolver (see current-module-name-resolver).
If use-compiled-file-paths contains an empty list when make-compilation-manager-load/use-compiled-handler is called, then exn:fail:contract exception is raised.
If the delete-zos-when-rkt-file-does-not-exist? argument is a true value, then the returned handler will delete ".zo" files when there is no corresponding original source file.
If the security-guard argument is supplied, it is used when creating ".zo" files, ".dep" files, and "compiled/" directories, and when it adjusts the timestamps for existing files. If it is #f, then the security guard in the current-security-guard when the files are created is used (not the security guard at the point make-compilation-manager-load/use-compiled-handler is called).
Do not install the result of
make-compilation-manager-load/use-compiled-handler when the
current namespace contains already-loaded versions of modules that may
need to be recompiled—
(managed-compile-zo file [ read-src-syntax #:security-guard security-guard]) → void? file : path-string?
read-src-syntax : (any/c input-port? . -> . syntax?) = read-syntax security-guard : (or/c security-guard? #f) = #f
If file is compiled from source, then read-src-syntax is used in the same way as read-syntax to read the source module. The normal read-syntax is used for any required files, however.
If security-guard is not #f, then the provided security guard is used when creating the "compiled/" directories, ".dep" and ".zo" files, and when it adjusts the timestamps of existing files. If it is #f, then the security guard in the current-security-guard when the files are created is used (not the security guard at the point managed-compile-zo is called).
(trust-existing-zos) → boolean? (trust-existing-zos trust?) → void? trust? : any/c
(make-caching-managed-compile-zo read-src-syntax [ #:security-guard security-guard]) → (path-string? . -> . void?) read-src-syntax : (any/c input-port? . -> . syntax?) security-guard : (or/c security-guard? #f) = #f
(manager-compile-notify-handler) → (path? . -> . any) (manager-compile-notify-handler notify) → void? notify : (path? . -> . any)
(manager-trace-handler) → (string? . -> . any) (manager-trace-handler notify) → void? notify : (string? . -> . any)
(manager-skip-file-handler) → (-> path? (or/c (cons/c number? promise?) #f)) (manager-skip-file-handler proc) → void? proc : (-> path? (or/c (cons/c number? promise?) #f))
(file-stamp-in-collection p) → (or/c (cons/c number? promise?) #f) p : path?
(file-stamp-in-paths p paths) → (or/c (cons/c number? promise?) #f) p : path? paths : (listof path?)
This function is intended for use with manager-skip-file-handler.
(get-file-sha1 p) → (or/c string? #f) p : path?
(get-compiled-file-sha1 p) → (or/c string? #f) p : path?
(with-compile-output p proc) → any p : path-string? proc : ([port input-port?] [tmp-path path?] . -> . any)
Windows prevents programs from overwriting files that are open. As a result, with-compile-output calls to rename-file-or-directory will fail if the destination file argument is an open file. Windows, however, does allow you to rename an open file. To avoid overwriting open files windows, with-compile-output creates a second temporary file tmp-path2, renames p to tmp-path2, renames tmp-path to p, and finally deletes tmp-path2.
(or/c #f (->i ([command (or/c 'lock 'unlock)] [file bytes?]) [res (command) (if (eq? command 'lock) boolean? void?)])) (parallel-lock-client proc) → void?
(or/c #f (->i ([command (or/c 'lock 'unlock)] [file bytes?]) [res (command) (if (eq? command 'lock) boolean? void?)]))
When proc is #f (the default), no checking for parallel compilation is done (and thus multiple threads or places running compilations via make-compilation-manager-load/use-compiled-handler will potentially corrupt each other’s ".zo" files).
When proc is a function, its first argument is a command, indicating if it wants to lock or unlock the path specified in the second argument.
When the proc 'lock command returns #t, the current builder has obtained the lock for zo-path. Once compilation of zo-path is complete, the builder process must release the lock by calling proc 'unlock with the exact same zo-path.
When the proc 'lock command returns #f, another parallel builder obtained the lock first and has already compiled the zo. The parallel builder should continue without compiling zo-path. (In this case, make-compilation-manager-load/use-compiled-handler’s result will not call proc with 'unlock.)
(compile-lock->parallel-lock-client pc [ cust]) → (-> (or/c 'lock 'unlock) bytes? boolean?) pc : place-channel? cust : (or/c #f custodian?) = #f
This communication protocol implementation is not kill safe. To make it kill safe, it needs a sufficiently powerful custodian, i.e., one that is not subject to termination (unless all of the participants in the compilation are also terminated). It uses this custodian to create a thread that monitors the threads that are doing the compilation. If one of them is terminated, the presence of the custodian lets another one continue. (The custodian is also used to create a thread that manages a thread safe table.)
(make-compile-lock) → place-channel?
1.4 API for Parallel-Build
(parallel-compile-files list-of-files [ #:worker-count worker-count #:handler handler]) → void? list-of-files : (listof path?) worker-count : non-negative-integer? = (processor-count)
(->i ([handler-type symbol?] [path path-string?] [msg string?] [out string?] [err string?]) void?) = void
(parallel-compile-files source-files #:worker-count 4 #:handler (lambda (type work msg out err) (match type ['done (when (verbose) (printf " Made ~a\n" work))] ['output (printf " Output from: ~a\n~a~a" work out err)] [else (printf " Error compiling ~a\n~a\n~a~a" work msg out err)])))
(parallel-compile worker-count setup-fprintf append-error collects-tree) → (void) worker-count : non-negative-integer?
(->* ([stage string?] [format string?]) () #:rest (listof any/c) void)
(-> cc? [prefix string?] [exn (or/c exn? null?)] [out string?] [err srtring?] [message string?] void?) collects-tree : (listof any/c)
1.5 Compilation Manager Hook for Syntax Transformers
(register-external-file file) → void? file : (and path? complete-path?)
A compilation manager implemented by compiler/cm looks for such messages to register an external dependency. The compilation manager records (in a ".dep" file) the path as contributing to the implementation of the module currently being compiled. Afterward, if the registered file is modified, the compilation manager will know to recompile the module.
The include macro, for example, calls this procedure with the path of an included file as it expands an include form.
1.6 Compiling to Raw Bytecode
The --no-deps mode for raco make is an improverished form of the compilation, because it does not track import dependencies. It does, however, support compilation of non-module source in an namespace that initially imports scheme.
Outside of a module, top-level define-syntaxes, module, #%require, define-values-for-syntax, and begin expressions are handled specially by raco make --no-deps: the compile-time portion of the expression is evaluated, because it might affect later expressions.
For example, when compiling the file containing
(require racket/class) (define f (class object% (super-new)))
the class form from the racket/class library must be bound in the compilation namespace at compile time. Thus, the require expression is both compiled (to appear in the output code) and evaluated (for further computation).
Many definition forms expand to define-syntaxes. For example, define-signature expands to define-syntaxes. In --no-deps mode, raco make --no-deps detects define-syntaxes and other expressions after expansion, so top-level define-signature expressions affect the compilation of later expressions, as a programmer would expect.
In contrast, a load or eval expression in a source
file is compiled—
In general, a better solution is to put all code to compile into a module and use raco make in its default mode.