1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
|
# specs
These are integration tests that execute the `deno` binary. They supersede the
`itest` macro found in the `tests/integration` folder and are the preferred way
of writing tests that use the `deno` binary.
## Structure
Tests must have the following directory structure:
```
tests/specs/<category_name>/<test_name>/__test__.json
```
## Test filtering
To run a specific test, run:
```
cargo test specs::category_name::test_name
```
Or just the following, though it might run other tests:
```
cargo test test_name
```
To run showing the output of every test use `-- --nocapture` (note: this will
cause tests to run sequentially instead of in parallel):
```
cargo test test_name -- --nocapture
```
## `__test__.json` file
This file describes the test(s) to execute and the steps to execute. A basic
example looks like:
```json
{
"args": "run main.js",
"output": "main.out"
}
```
This will run `deno run main.js` then assert that the output matches the text in
`main.out`.
Or another example that runs multiple steps:
```json
{
"tempDir": true,
"steps": [{
"args": "cache main.ts",
"output": "cache.out"
}, {
"args": "run main.ts",
"output": "error.out",
"exitCode": 1
}]
}
```
Or if you want to run several tests at the same time:
```json
{
"tests": {
"ignore_dir": {
"args": "run script.ts",
"output": "script.out"
},
"some_other_test": {
"args": "run other.ts",
"output": "other.out"
}
}
}
```
### Top level properties
- `repeat` (number) - Number of times to repeat a test.
- `tempDir` (boolean) - Copy all the non-test files to a temporary directory and
execute the command in that temporary directory.
- By default, tests are executed with a current working directory of the test,
but this may not be desirable for tests such as ones that create a
node_modules directory.
### Step properties
When writing a single step, these may be at the top level rather than nested in
a "steps" array or "tests" object.
- `args` - A string (that will be spilt on whitespace into an args array) or an
array of arguments.
- `output` - Path to use to assert the output or text (must end with an .out
extension) _or_ text to pattern match against the output.
- `flaky` - Step should be repeated until success a maximum of 3 times.
- `if` (`"windows"`, `"linux"`, `"mac"`, `"unix"`) - Whether to run this step.
- `exitCode` (number) - Expected exit code.
### Auto-complete
To get auto-complete for these files, add the following to a local
`.vscode/settings.json` file:
```json
{
"json.schemas": [{
"fileMatch": [
"__test__.jsonc"
],
"url": "./tests/specs/schema.json"
}]
}
```
## `.out` files
`.out` files are used to assert the output when running a test or test step.
Within the file, you can use the following for matching:
- `[WILDCARD]` - match any text at the wildcard
- `[WILDLINE]` - match any text on the current line
- `[WILDCHAR]` - match the next character
- `[WILDCHARS(5)]` - match any of the next 5 characters
- `[UNORDERED_START]` followed by many lines then `[UNORDERED_END]` will match
the lines in any order (useful for non-deterministic output)
- `[# example]` - line comments start with `[#` and end with `]`
|
