13 raco test: Run tests
The raco test command requires and runs the (by default) test submodule associated with each path given on the command line. Command-line flag can control which submodule is run, whether to run the main module if no submodule is found, and whether to run tests directly, in separate processes (the default), or in separate places. The current directory is set to a test file’s directory before running the file.
When an argument path refers to a directory, the tool recursively discovers and runs all files within the directory that end in ".rkt", end in ".scrbl", or have a (possibly empty) list of arguments-line provided by test-command-line-arguments in an "info.rkt" file.
A test is counted as failing if it logs a failing test code via test-log!, causes Racket to exit with a non-zero exit code, or (when -e or --check-stderr is specified) if it produces output on the error port.
The raco test command accepts several flags:
-c or --collection —
Intreprets the arguments as collections where whose files should be tested. -p or --package —
Intreprets the arguments as packages whose files should be tested. (All package scopes are searched for the first, most specific package.) -m or --modules —
Not only interprets the arguments as paths (which is the default mode), but treats them the same as a path found in a directory, which means ignoring a file argument that does not have the extension ".rkt", have the extension ".scrbl", or is enabled explicitly via test-command-line-arguments in an "info.rkt" file. --drdr —
Configures defaults to imitate the DrDr continuous testing system: ignore non-modules, use as many jobs as available processors, set the default timeout to 90 seconds, count stderr output as a test failure, quiet program output, and print a table of results. -s ‹name› or --submodule ‹name› —
Requires the submodule ‹name› rather than test. Supply -s or --submodule to run multiple submodules, or combine multiple submodules with --first-avail to run the first available of the listed modules. -r or --run-if-absent —
Requires the top-level module of a file if a relevant submodule is not present. This is the default mode. -x or --no-run-if-absent —
Ignores a file if the relevant submodule is not present. --first-avail —
When multiple submodule names are provided with -s or --submodule, runs only the first available submodule. --direct —
Runs each test in a thread. This mode is the default if a single file is specified. Multiple tests can interfere with each other and the overall test run by exiting, unsafe operations that block (and thus prevent timeout), and so on. --process —
Runs each test in a separate operating-system process. This mode is the default if multiple files are specified or if a directory, collection, or package is specified. --place —
Runs each test in a place, instead of in an operating-system process. -j ‹n› or --jobs ‹n› —
Runs up to ‹n› tests in parallel. --timeout ‹seconds› —
Sets the default timeout (after which a test counts as failed) to ‹seconds›. Use +inf.0 to allow tests to run without limit but allow timeout sub-submodule configuration. If any test fails due to a timeout, the exit status of raco test is 2 (as opposed to 1 for only non-timeout failures or 0 for success). -Q or --quiet-program —
suppresses output from each test program. -e or --check-stderr —
count any stderr output as a test failure. -q or --quiet —
suppresses output of progress information, responsible parties, and varying output (see Responsible-Party and Varying-Output Logging). --table or -t —
Print a summary table after all tests. If a test uses rackunit, or if a test at least uses test-log! from rackunit/log to log successes and failures, the table reports test and failure counts based on the log.
13.1 Test Configuration by Submodule
When raco test runs a test in a submodule, a config sub-submodule can provide additional configuration for running the test. The config sub-submodule should use the info module language to define the following identifiers:
timeout —
a real number to override the default timeout for the test, which applies only when timeouts are enabled. responsible —
a string, symbol, or list of symbols and strings identifying a responsible party that should be notified when the test fails. See Responsible-Party and Varying-Output Logging. lock-name —
a string that names a lock file that is used to serialize tests (i.e., tests that have the same lock name do not run concurrently). The lock file’s location is determined by the PLTLOCKDIR enviornment varible or defaults to (find-system-path 'temp-dir). The maximum time to wait on the lock file is determined by the PLTLOCKTIME environment variable or defaults to 4 hours. random? —
if true, indicates that the test’s output is expected to vary. See Responsible-Party and Varying-Output Logging.
13.2 Test Configuration by "info.rkt"
Submodule-based test configuration is preferred (see Test Configuration by Submodule). In particular, to prevent raco test from running a particular file, normally the file should contain a submodule that takes no action.
In some cases, however, adding a submodule is inconvenient or impossible (e.g., because the file will not always compile). Thus, raco test also consults any "info.rkt" file in the candidate test file’s directory. In the case of a file within a collection, "info.rkt" files from any enclosing collection directories are also consulted for test-omit-paths. Finally, for a file within a package, the package’s "info.rkt" is consulted for pkg-authors to set the default responsible parties (see Responsible-Party and Varying-Output Logging) for all files in the package.
The following "info.rkt" fields are recognized:
test-omit-paths —
a list of path strings (relative to the enclosing directory) or 'all to omit all files within the enclosing directory. When a path string refers to a directory, all files within the directory are omitted. test-command-line-arguments —
a list of (list module-path-string (list argument-path-string ...)), where current-command-line-arguments is set to a vector that contains the argument-path-string when running module-path-string. test-timeouts —
a list of (list module-path-string real-number) to override the default timeout for module-path-string. test-responsibles —
a list of (list module-path-string party) or (list 'all party) to override the default responsible party for module-path-string or all files within the directory (except as overridden), respectively. Each party is a string, symbol, or list of symbols and strings. See Responsible-Party and Varying-Output Logging. test-lock-names —
a list of (list module-path-string lock-string) to declare a lock file name for module-path-string. See lock-name in Test Configuration by Submodule. test-randoms —
a list of path strings (relative to the enclosing directory) for modules whose output varies. See Responsible-Party and Varying-Output Logging.
13.3 Responsible-Party and Varying-Output Logging
When a test has a declared responsible party, then the test’s output is prefixed with a
raco test:‹which› @(test-responsible '‹responsible›) |
line, where ‹which› is a space followed by an exact non-negative number indicating a parallel task when parallelism is enabled (or empty otherwise), and ‹responsible› is a string, symbol, or list datum.
When a test’s output (as written to stdout) is expected to vary across
runs—
raco test:‹which› @(test-random #t) |
line.