1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
|
Testament is an advanced automatic unittests runner for Nim tests, is used for the development of Nim itself,
offers process isolation for your tests, it can generate statistics about test cases,
supports multiple targets (C, C++, ObjectiveC, JavaScript, etc),
simulated `Dry-Runs <https://en.wikipedia.org/wiki/Dry_run_(testing)>`_,
has logging, can generate HTML reports, skip tests from a file and more,
so can be useful to run your tests, even the most complex ones.
Test files location
===================
By default Testament looks for test files on ``"./tests/*.nim"``,
you can overwrite this pattern glob using ``pattern <glob>``,
the default working directory path can be changed using ``--directory:"folder/subfolder/"``.
Testament uses the nim compiler on ``PATH`` you can change that using ``--nim:"folder/subfolder/nim"``,
running JavaScript tests with ``--targets:"js"`` requires a working NodeJS on ``PATH``.
Options
=======
* ``--print`` Also print results to the console
* ``--simulate`` See what tests would be run but don't run them (for debugging)
* ``--failing`` Only show failing/ignored tests
* ``--targets:"c c++ js objc"`` Run tests for specified targets (default: all)
* ``--nim:path`` Use a particular nim executable (default: ``$PATH/nim``)
* ``--directory:dir`` Change to directory dir before reading the tests or doing anything else.
* ``--colors:on|off`` Turn messages coloring on|off.
* ``--backendLogging:on|off`` Disable or enable backend logging. By default turned on.
* ``--skipFrom:file`` Read tests to skip from ``file`` - one test per line, # comments ignored
Running a single test
=====================
This is a minimal example to understand the basics,
not very useful for production, but easy to understand:
.. code::
$ mkdir tests
$ echo "assert 42 == 42" > tests/test0.nim
$ testament run test0.nim
PASS: tests/test0.nim C ( 0.2 sec)
$ testament r test0
PASS: tests/test0.nim C ( 0.2 sec)
$
Running all tests from a directory
==================================
.. code::
$ testament pattern "tests/*.nim"
HTML Reports
============
Generate HTML Reports ``testresults.html`` from unittests,
you have to run at least 1 test *before* generating a report:
.. code::
$ testament html
Writing Unitests
================
Example "template" **to edit** and write a Testament unittest:
.. code-block:: nim
discard """
action: "run" # What to do, one of "compile" OR "run".
exitcode: 0 # This is the Exit Code the test should return, zero typically.
output: "" # This is the Standard Output the test should print, if any.
input: "" # This is the Standard Input the test should take, if any.
errormsg: "" # Error message the test should print, if any.
batchable: true # Can be run in batch mode, or not.
joinable: true # Can be run Joined with other tests to run all togheter, or not.
valgrind: false # Can use Valgrind to check for memory leaks, or not (Linux 64Bit only).
cmd: "c -r $file" # Command the test should use to run.
maxcodesize: 666 # Maximum generated temporary intermediate code file size for the test.
timeout: 666 # Timeout microseconds to run the test.
target: "c js" # Targets to run the test into (C, C++, JavaScript, etc).
disabled: "bsd" # Disable the test by condition, here BSD is disabled just as an example.
"""
assert true
assert 42 == 42, "Assert error message"
* As you can see the "Spec" is just a ``discard """ """``.
* Spec has sane defaults, so you dont need to provide them all, any simple assert will work Ok.
* `This is not the full spec of Testament, check the Testament Spec on GitHub, see parseSpec(). <https://github.com/nim-lang/Nim/blob/devel/testament/specs.nim#L238>`_
* `Nim itself uses Testament, so theres plenty of test examples. <https://github.com/nim-lang/Nim/tree/devel/tests>`_
* Has some built-in CI compatibility, like Azure Pipelines, etc.
* `Testament supports inlined error messages on Unittests, basically comments with the expected error directly on the code. <https://github.com/nim-lang/Nim/blob/9a110047cbe2826b1d4afe63e3a1f5a08422b73f/tests/effects/teffects1.nim>`_
Unitests Examples
=================
Expected to fail:
.. code-block:: nim
discard """
errormsg: "undeclared identifier: 'not_defined'"
"""
assert not_defined == "not_defined", "not_defined is not defined"
Non-Zero exit code:
.. code-block:: nim
discard """
exitcode: 1
"""
quit "Non-Zero exit code", 1
Standard output checking:
.. code-block:: nim
discard """
output: '''
0
1
2
3
4
5
'''
"""
for i in 0..5: echo i
JavaScript tests:
.. code-block:: nim
discard """
target: "js"
"""
when defined(js):
import jsconsole
console.log("My Frontend Project")
Compile time tests:
.. code-block:: nim
discard """
action: "compile"
"""
static: assert 9 == 9, "Compile time assert"
Tests without Spec:
.. code-block:: nim
assert 1 == 1
See also:
* `Unittest <unittest.html>`_
|