letrecfoox0f=function|[] ->0|x::xs -> f x (foo x0 f xs)

(*$T foo  foo  0 ( + ) [1;2;3] = 6  foo  0 ( * ) [1;2;3] = 0  foo  1 ( * ) [4;5]   = 20  foo 12 ( + ) []      = 12*)

Simple Tests:T for "Test"

The most common kind of tests is the simple test, an example of which isgiven above. It is of the form

where eachstatement must be a boolean OCaml expression involving thefunction (or functions, as we will see when we study headers) referencedin theheader. The overall test is considered successful if eachstatement evaluates totrue. Note that the "close comment"*)must appear on a line of its own.

Tip: if a statement is a bit too long to fit on one line, if can bebroken using a backslash (\), immediately followed by the carriagereturn. This also applies to randomised tests.

Equality Tests:=

The vast majority of test cases tend to involve the equality of twoexpressions; using simple tests, one would write something like:

While this certainly works, the failure report for such a test does notconvey any useful information besides the simple fact that the testfailed. Wouldn’t it be nice if the report also mentioned the values ofthe left-hand side and the right-hand side ? Yes it would, andspecialised equality tests provide such functionality, at the cost of alittle bit of boilerplate code. The bare syntax is:

However, used bare, an equality test will not provide much moreinformation than a simple test: just a laconic "not equal". In orderfor the values to be printed, a "value printer" must be specified forthe test. A printer is a function of type'a → string, where'a isthe type of the expressions on both side of the equality. To pass theprinter to the test, we useparameter injection (cf. SectionHeader Parameters Injection); equality tests have an optional argumentprinter forthis purpose. In our example, we have'a = int, so the test becomes simply:

The failure report will now be more explicit, sayingexpected: 20 but got: 30.

Randomized Tests:Q for "Quickcheck"

Quickcheck is a small library useful for randomized unit tests. Using itis a bit more complex, but much more rewarding than simple tests.

Let us dive into an example straight-away:

The Quickcheck module is accessible simply asQ within inline tests;small_int is a generator, yielding a random, small integer. When thetest is run, each statement will be evaluated for a large number ofrandom values – 100 by default. Running this test for theabove definition of foo catches the mistake easily:

Note that the random value for which the test failed is provided by theerror message – here it is 2. It is also possible to generate severalrandom values simultaneously using tuples. For instance

will generate both an integer and a list of small integers randomly. Afailure will then look like

A generator such asQ.pair Q.small_int Q.printable_string is actually a value of type'a Q.arbitrary (in this particular case,(int * string) arbitrary).It combines a random generation function ('a Q.Gen.t),and optional printing, shrinking and size functions that are used todisplay counter-examples and minimize their size. It is possible, asexplained below, to define one’s own'a arbitrary values, for instancefor custom types.

Available Generators:

Simple generators

unit,bool,float,pos_float,neg_float,int,int32,int64,pos_int,small_int,neg_int,char,printable_char,numeral_char,string,printable_string,numeral_string

Structure generators

list andarray. They take one generator as their argument. Forinstance(Q.list Q.neg_int) is a generator of lists of (uniformlytaken) negative integers.

Tuple generators

pair andtriple are respectively binary and ternary. See above foran example ofpair.

Size-directed generators

string,numeral_string,printable_string,list andarray allhave*_of_size variants that take the size of the structure as theirfirst argument.

See theonline documentation of QCheckfor more details.

Tips:

Duplicate Elements in Lists

When generating lists, avoidQ.list Q.int unless you have a good reason to do so. The reason isthat, given the size of theQ.int space, you are unlikely to generateany duplicate elements. If you wish to test your function’s behaviourwith duplicates, preferQ.list Q.small_int.

Filtering Inputs

Rando, inputs can be filtered for aprecondition by stating a propertyf =⇒ g. An inputx will be tested for the propertyg only iff x holds,otherwise it is discarded and a new input is generated. The total number ofinputs generated can be capped using the~max_gen:int parameter (it shouldbe bigger than~count). The system will try to makecount tests, but stopsaftermax_gen inputs are generated to avoid looping forever if acceptableinputs are too rare.

Changing Number of Tests

If you want a specific test to executeeach of its statements a specific number of times (deviating from thedefault of 100), you can specify it explicitly throughparameter injection (cf. SectionHeader Parameters Injection) using thecount :argument.

Getting a Better Counterexample

By default, a random test stops assoon as one of its generated values yields a failure. This first failurevalue is probably not the best possible counterexample. You canforceqtest to generate and test allcount random values regardless, and todisplay the value which is smallest with respect to a certain measurewhich you define. To this end, it suffices to use parameter injection topass argumentsmall : 'a → 'b, where'a is the type of generated values and'b is any totally ordered set (wrt.<).Typically you will take'b = int or'b = float. Example:

letfuzx= xletrecflu=function|[] ->[]|x ::l ->ifList.mem x lthen flu lelse x :: flu l(*$Q fuz; flu & ~small:List.length  (Q.list Q.small_int) (fun x -> fuz x = flu x)*)

The meaning of~small:List.length is therefore simply: "choose the shortest list". For very complicated cases, you can simultaneouslyincreasecount to yield an even higher-quality counterexample.

Shrinking

A parametershrink: ('a → 'a Q.Iter.t) can be provided along with a randomgenerator.'a Q.Iter.t is an iterator on values of type'a.shrink xshould iterate on a set of values that are smaller thanx (for instance,ifx: int list,shrink x will remove each element of the list).If a generator (of type'a arbitrary) defines a shrink function, thenwhenever a counter-example is found for a property, thecounter-example will be shrunk recursively as long as it continues refutingthe property; this allows to find smaller and simpler counter-examples.However, shrinking can be slow.A parameter~max_fail:int can be given to the testby writing(*$Q & ~max_fail:5 to limit the number of counter-examplesto find, in case shrinking them is too slow.

The moduleQ.Shrink can be used to combine shrinking functions.

Example: the false property(Q.list Q.int) (fun l → not (List.mem 5 l))might be falsified by the counter-example[1;2;3;4;5;6;7;8]. By recursivelyshrinking the value (trying to remove elements one by one) the minimalcounter-example[5] will be found and displayed.

Raw Random Tests

Using($QR, similar to the raw unit test($R, it is possible towrite a random test on multiple lines without the trailing\characters.

(*$QR foo  Q.small_int    (fun i->      foo i (+) [1;2;3] = List.fold_left (+) i [1;2;3]    )*)

The(*$QR block needs to contain exactly two values:

Random Generator

of type'a Quickcheck.arbitrary

Property to test

of type'a → bool

Custom Generators

For types that are not lists of integers or strings, it can be useful to defineone’s own'a arbitrary instance for the type. The function to use isQ.make, it takes a'a Q.Gen.t random generator, and optional arguments

• ~shrink:('a → 'a Iter.t) to define how to shrink counter-examples

• ~small:('a → 'b) (where'b is ordered) to select small counter-examples

• ~print:('a → string) to print counter-examples

• ~collect:('a → string) maps inputs to astring descriptor andcounts how many values belong to each descriptor, for statistics.

  Some generators are already defined inQ.Gen. Gabriel Scherer’srandom-generator library is alsoa good basis for more advanced generators.

  Printers can be defined usingQ.Print, shrinkers usingQ.Shrink.

Raw OUnit Tests:R for "Raw"

When more specialised test pragmas are too restrictive, for instance ifthe test is too complex to reasonably fit on one line, then one can useraw OUnit tests.

Here is a small example, with two tests stringed together:

Note that if the first assertion fails, the second will not be executed;so stringing two assertions in that mode is different in that respectfrom doing so under aT pragma, for instance.

That said, raw tests should only be used as a last resort; for instanceyou don’t automatically get the source file and line number when thetest fails. IfT andQ do not satisfy your needs, then it isprobably a hint that the test is a bit complex and, maybe, belongs ina separate test suite rather than in the middle of the source code.

Exception-Throwing Tests:E for "Exception"

… not implemented yet…

The current usage is to use(*$T and the following pattern forfunctionfoo and exceptionBar:

If your project uses Batteries and no pattern-matching is needed, thenyou can also use the following, sexier pattern:

foo.ml   42    foo

Aliases

Some functions have exceedingly long names. Case in point :

The constraint that each statement must fit on one line does not playwell with very long function names. Furthermore, youknown whichfunction is being tested, it’s right there is the header; no need torepeat it a dozen times. Instead, you can define analias, and writeequivalently:

…thus saving many keystrokes, thereby contributing to thepreservation of the environment. More seriously, aliases have usesbeyond just saving a few keystrokes, as we will see in the nextsections.

Mutually Tested Functions

Most of the time, a test only pertains to one function. There are times,however, when one wishes to test two functions – or more – at the sametime. For instance

Let us say that we have the following test:

It involves botheven andodd. That question is: "what is a properheader for this test?" One could simply put "even", and thus it wouldbe referenced as being tested in the logs, butodd would not, which isunfair. Putting "odd" is symmetrically unfair. The solution is to putboth, separated by a semi-colon:

That wayboth functions are referenced in the logs:

and of course the compiler enforces that both of them are actuallyreferenced in each statement of the test. Of course, each of them can bewritten under alias, in which case the header could beeven as x; odd as y.

Testing Functions by the Dozen

Let us come back to our functionsfoo (after correction) andpretentious_drivel, as defined above.

You will not have failed to notice that they bear more than a passingresemblance to one another. If you write tests for one, odds are thatthe same test could be useful verbatim for the other. This is a verycommon case when you have closely related functions, or even severalimplementations of the same function, for instance the old, slow,naïve, trustworthy one and the new, fast, arcane, highly optimisedversion you have just written. The typical case is sorting routines, ofwhich there are many flavours.

For our example, recall that we have the following test forfoo:

The same test would apply topretentious_drivel; you could justcopy-and-paste the test and change the header, but it’s not terriblyelegant. Instead, you can just just add the other function to theheader, separating the two by a comma, and defining an alias:

This same test will be run once forx = foo, and once forx = pretentious_drivel. Actually, you need not define an alias: if theheader is of the form

then it is equivalent to

so you do not need to alter the body of the test if you subsequently addnew functions. A header which combines more than one "version" of afunction in this way is called ametaheader.

Metaheaders Unleashed

All the constructs above can be combined without constraints: thegrammar is as follows:

Header Parameters Injection

Use(*$inject foo *) to inject the piece of codefoo at thebeginning of this module’s tests. This is useful, for instance, todefine frequently used random generators, or printers, or to instantiatea functor before testing it.

Fatal error: exception Failure("Unrecognised qtest pragma: ` T foo'")

Warning: likely qtest syntax error: `(* $T foo'. Done.

Fatal error: exception Core.Bad_header_char("M", "Mod.foo")

Error: Comment not terminatedFatal error: exception Core.Unterminated_test(_, 0)

Fatal error: exception Failure("runaway test body terminator: n))*)")

$ qtest --help** qtest (qtest)USAGE: qtest [options] extract <file.mli?>...OPTIONS:--output <file.ml>    (-o) def: standard output  Open or create a file for output; the resulting file will be an OCaml  source file containing all the tests.--preamble <string>   (-p) def: empty  Add code to the tests' preamble; typically this will be an instruction  of the form 'open Module;;'--help          Displays this help page and stops

$ qtest extract foo.ml -o footest.ml$ ocamlfind ocamlopt -package qcheck -linkpkg footest.ml -o footest$ ./footest --helprun qtest suite-v-verbose  enable verbose tests-l-list     print list of tests (2 lines each). Implies -verbose-s-seed     set random seed (to repeat tests)-help     Display this list of options--help    Display this list of options

(rule  (targets run_qtest.ml)  (deps (source_tree src))  ; here is where you need to tell qtest what files to consider  (action (run qtest extract src/foo1.ml src/foo2.ml > %{targets})))(executable  (name run_qtest)  (modules run_qtest)  ; disable some warnings in qtests  (flags :standard -warn-error -a -w -33-35-27-39)  (libraries qcheck))(alias  (name    runtest)  (deps    run_qtest.exe)  (action  (run %{deps})))

src/Foosrc/sub/Bar