3 RackUnit API

3.1 Overview of RackUnit

There are three basic concepts in RackUnit:

• A check is the basic unit of a test. As the name suggests, it checks some condition is true.

• A test case is a group of checks that form one conceptual unit. If any check within the case fails, the entire case fails.

• A test suite is a group of test cases and test suites that has a name.

3.2 Checks

Checks are the basic building block of RackUnit. A check checks some condition and always evaluates to (void). If the condition doesn’t hold, the check will report the failure (see current-check-handler for customizing how failures are handled).

Although checks are implemented as macros, which is necessary to grab source location, they are conceptually functions (with the exception of check-match below). This means, for instance, checks always evaluate their arguments. You can use checks as first class functions, though you will lose precision in the reported source locations if you do so.

The following are the basic checks RackUnit provides. You can create your own checks using define-check.

For example, the following checks all fail:

> (check-eq? (list 1) (list 1) "allocated data not eq?")

-------------------- FAILURE name:       check-eq? location:   eval:3:0 message:    "allocated data not eq?" actual:     '(1) expected:   '(1) --------------------

> (check-not-eq? 1 1 "fixnums are eq?")

-------------------- FAILURE name:       check-not-eq? location:   eval:4:0 params:     '(1 1) message:    "fixnums are eq?" --------------------

> (check-eqv? 1 1.0 "not eqv?")

-------------------- FAILURE name:       check-eqv? location:   eval:5:0 message:    "not eqv?" actual:     1 expected:   1.0 --------------------

> (check-not-eqv? 1 1 "integers are eqv?")

-------------------- FAILURE name:       check-not-eqv? location:   eval:6:0 params:     '(1 1) message:    "integers are eqv?" --------------------

> (check-equal? 1 1.0 "not equal?")

-------------------- FAILURE name:       check-equal? location:   eval:7:0 message:    "not equal?" actual:     1 expected:   1.0 --------------------

> (check-not-equal? (list 1) (list 1) "equal?")

-------------------- FAILURE name:       check-not-equal? location:   eval:8:0 params:     '((1) (1)) message:    "equal?" --------------------

For example, the following check passes:

For example, the following checks all fail:

> (check-true 1)

-------------------- FAILURE name:       check-true location:   eval:13:0 params:     '(1) --------------------

> (check-false 1)

-------------------- FAILURE name:       check-false location:   eval:14:0 params:     '(1) --------------------

> (check-not-false #f)

-------------------- FAILURE name:       check-not-false location:   eval:15:0 params:     '(#f) --------------------

For example, the following checks succeed:

> (check-exn    exn:fail?    (lambda ()      (raise (make-exn:fail "Hi there"                            (current-continuation-marks)))))

> (check-exn    exn:fail?    (lambda ()      (error 'hi "there")))

The following check fails:

> (check-exn exn:fail?              (lambda ()                (break-thread (current-thread))))

-------------------- ERROR user break --------------------

The following example is a common mistake. The call to error is not within a lambda, so it bypasses check-exn entirely.

; Forgot to wrap the expression in a thunk. Don't do this!

> (check-exn exn:fail?              (error 'hi "there"))

hi: there

> (check-not-exn (λ () 1))

> (check-not-exn (λ () (car '())))

-------------------- FAILURE name:               check-not-exn location:           eval:21:0 params:             '(#<procedure:temp81>) message:            "Exception raised" exception-message:  "car: contract violation\n  expected: pair?\n  given: '()" exception:          #(struct:exn:fail:contract "car: contract violation\n  expected: pair?\n  given: '()" #<continuation-mark-set>) --------------------

> (check-not-exn (λ () (/ 1 0)) "don't divide by 0")

-------------------- FAILURE name:               check-not-exn location:           eval:22:0 params:             '(#<procedure:temp84>) message:            "don't divide by 0" message:            "Exception raised" exception-message:  "/: division by zero" exception:          #(struct:exn:fail:contract:divide-by-zero "/: division by zero" #<continuation-mark-set>) --------------------

For example, the following check succeeds:

> (check-regexp-match "a+bba" "aaaaaabba")

The following check fails:

> (check-regexp-match "a+bba" "aaaabbba")

-------------------- FAILURE name:       check-regexp-match location:   eval:24:0 params:     '("a+bba" "aaaabbba") --------------------

> (check-match (list 1 2 3) (list _ _ 3))

This check fails to match:

> (check-match (list 1 2 3) (list _ _ 4))

-------------------- FAILURE name:       check-match location:   eval:26:0 actual:     '(1 2 3) expected:   '(list _ _ 4) name:       check-true location:   /home/scheme/pltbuild/racket/build/user/6.10/pkgs/rackunit-lib/rackunit/private/check.rkt:224:9 --------------------

If pred is provided, it is evaluated with the bindings from the match pattern. If it produces #t, the entire check succeeds, otherwise it fails. For example, this use succeeds, binding x in the predicate:

> (check-match (list 1 (list 3)) (list x (list _)) (odd? x))

This check fails because the pred fails:

> (check-match 6 x (odd? x))

-------------------- FAILURE name:       check-match location:   eval:28:0 actual:     6 expected:   'x name:       check-true location:   /home/scheme/pltbuild/racket/build/user/6.10/pkgs/rackunit-lib/rackunit/private/check.rkt:224:9 --------------------

This check fails because of a failure to match:

> (check-match (list 1 2) (list x) (odd? x))

-------------------- FAILURE name:       check-match location:   eval:29:0 actual:     '(1 2) expected:   '(list x) name:       check-true location:   /home/scheme/pltbuild/racket/build/user/6.10/pkgs/rackunit-lib/rackunit/private/check.rkt:224:9 --------------------

For example, the following check succeeds:

> (check < 2 3)

The following check fails:

> (check memq 'pine '(apple orange pear))

-------------------- FAILURE name:       check location:   eval:31:0 params:     '(#<procedure:memq> pine (apple orange pear)) --------------------

3.2.1 Augmenting Information on Check Failure

When a check fails it stores information including the name of the check, the location and message (if available), the expression the check is called with, and the parameters to the check. Additional information can be stored by using the with-check-info* function, and the with-check-info macro.

The are several predefined functions that create check information structures with predefined names. This avoids misspelling errors:

procedure (make-check-name name) → check-info?

name : string?

(make-check-params params) → check-info?

params : (listof any)

(make-check-location loc) → check-info?

loc   :   (list/c any (or/c number? #f) (or/c number? #f)             (or/c number? #f) (or/c number? #f))

(make-check-expression msg) → check-info?

msg : any

(make-check-message msg) → check-info?

msg : string?

(make-check-actual param) → check-info?

param : any

(make-check-expected param) → check-info?

param : any

> (with-check-info*    (list (make-check-info 'time (current-seconds)))    (lambda () (check = 1 2)))

-------------------- FAILURE time:       1501516560 name:       check location:   eval:36:0 params:     '(#<procedure:=> 1 2) --------------------

When this check fails the message

is printed along with the usual information on an check failure.

> (for-each    (lambda (elt)      (with-check-info       (('current-element elt))       (check-pred odd? elt)))    (list 1 3 5 7 8))

-------------------- FAILURE current-element:  8 name:             check-pred location:         eval:37:0 params:           '(#<procedure:odd?> 8) --------------------

When this test fails the message

is displayed along with the usual information on an check failure.

3.2.2 Custom Checks

Custom checks can be defined using define-check and its variants. To effectively use these macros it is useful to understand a few details about a check’s evaluation model.

Firstly, a check should be considered a function, even though most uses are actually macros. In particular, checks always evaluate their arguments exactly once before executing any expressions in the body of the checks. Hence if you wish to write checks that evaluate user defined code that code must be wrapped in a thunk (a function of no arguments) by the user. The predefined check-exn is an example of this type of check.

It is also useful to understand how the check information stack operates. The stack is stored in a parameter and the with-check-info forms evaluate to calls to parameterize. For this reason simple checks (see below) cannot usefully contain calls to with-check-info to report additional information. All checks created using define-simple-check or define-check grab some information by default: the name of the checks and the values of the parameters. Additionally the macro forms of checks grab location information and the expressions passed as parameters.

For example, the following code defines a check check-odd?

> (define-simple-check (check-odd? number)     (odd? number))

We can use these checks in the usual way:

> (check-odd? 3)

> (check-odd? 2)

-------------------- FAILURE name:       check-odd? location:   eval:40:0 params:     '(2) --------------------

Here’s the first form, where we use a predefined predicate to construct a binary check:

> (define-binary-check (check-char=? char=? actual expected))

In use:

> (check-char=? (read-char (open-input-string "a")) #\a)

If the expression is more complicated the second form should be used. For example, below we define a binary check that tests a number if within 0.01 of the expected value:

> (define-binary-check (check-in-tolerance actual expected)     (< (abs (- actual expected)) 0.01))

3.3 Compound Testing Forms

3.3.1 Test Cases

As programs increase in complexity the unit of testing grows beyond a single check. For example, it may be the case that if one check fails it doesn’t make sense to run another. To solve this problem compound testing forms can be used to group expressions. If any expression in a group fails (by raising an exception) the remaining expressions will not be evaluated.

For example, in the following code the world is not destroyed as the preceding check fails:

(test-begin

(check-eq? 'a 'b)

; This line won't be run

(destroy-the-world))

Here’s the above example rewritten to use test-case so the test can be named.

(test-case

"Example test"

(check-eq? 'a 'b)

; This line won't be run

(destroy-the-world))

3.3.1.1 Shortcuts for Defining Test Cases

3.3.2 Test Suites

Test cases can themselves be grouped into test suites. A test suite can contain both test cases and test suites. Unlike a check or test case, a test suite is not immediately run. Instead use one of the functions described in User Interfaces or Programmatically Running Tests and Inspecting Results.

The before-thunk and after-thunk are optional thunks (functions with no argument). They are run before and after the tests are run, respectively.

Unlike a check or test case, a test suite is not immediately run. Instead use one of the functions described in User Interfaces or Programmatically Running Tests and Inspecting Results.

For example, here is a test suite that displays Before before any tests are run, and After when the tests have finished.

(test-suite

"An example suite"

#:before (lambda () (display "Before"))

#:after  (lambda () (display "After"))

(test-case

"An example test"

(check-eq? 1 1))

(test-suite "A nested test suite"

(test-case "Another test"

(check < 1 2))))

3.3.2.1 Utilities for Defining Test Suites

There are some macros that simplify the common cases of defining test suites:

For example, this code creates a binding for the name example-suite as well as creating a test suite with the name "example-suite":

(define-test-suite example-suite

(check = 1 1))

3.4 Test Control Flow

The before, after, and around macros allow you to specify code that is always run before, after, or around expressions in a test case.

Example:

The test below checks that the file test.dat contains the string "foo". The before action writes to this file. The after action deletes it.

(around

(with-output-to-file "test.dat"

(lambda ()

(write "foo")))

(with-input-from-file "test.dat"

(lambda ()

(check-equal? "foo" (read))))

(delete-file "test.dat"))

3.5 Miscellaneous Utilities

The require/expose macro allows you to access bindings that a module does not provide. It is useful for testing the private functions of modules.

Note that require/expose can be a bit fragile, especially when mixed with compiled code. Use at your own risk!

This example gets make-failure-test, which is defined in a RackUnit test:

(require/expose rackunit/private/check-test (make-failure-test))

3.6 User Interfaces

RackUnit provides a textual and a graphical user interface

3.6.1 Textual User Interface

The textual UI is in the rackunit/text-ui module. It is run via the run-tests function.

The optional verbosity is one of 'quiet, 'normal, or 'verbose. Quiet output displays only the number of successes, failures, and errors. Normal reporting suppresses some extraneous check information (such as the expression). Verbose reports all information.

run-tests returns the number of unsuccessful tests.

3.6.2 Graphical User Interface

RackUnit also provides a GUI test runner, available from the rackunit/gui module.

When wait? is true, test/gui does not return until the test runner window has been closed.

Given the following program, the RackUnit GUI will look as shown below:

#lang racket

(require rackunit rackunit/gui)

(test/gui

(test-suite

"all tests"

(test-suite

"math tests"

(test-case "addition" (check-equal? (+ 1 1) 2))

(test-case "subtraction" (check-equal? (- 0 0) 0))

(test-case "multiplication" (check-equal? (* 2 2) 5)))

(test-suite

"string tests"

(test-case "append" (check-equal? (string-append "a" "b") "ab"))

(test-case "ref" (check-equal? (string-ref "abc" 1) #\b)))))