]>
Commit | Line | Data |
---|---|---|
a1dfa0c6 XL |
1 | # Running tests |
2 | ||
6a06907d XL |
3 | <!-- toc --> |
4 | ||
a1dfa0c6 XL |
5 | You can run the tests using `x.py`. The most basic command – which |
6 | you will almost never want to use! – is as follows: | |
7 | ||
8 | ```bash | |
60c5eb7d | 9 | ./x.py test |
a1dfa0c6 XL |
10 | ``` |
11 | ||
6a06907d | 12 | This will build the stage 1 compiler and then run the whole test |
a1dfa0c6 | 13 | suite. You probably don't want to do this very often, because it takes |
6a06907d XL |
14 | a very long time, and anyway bors / GitHub Actions will do it for you. |
15 | (Often, I will run this command in the background after opening a PR that | |
16 | I think is done, but rarely otherwise. -nmatsakis) | |
a1dfa0c6 XL |
17 | |
18 | The test results are cached and previously successful tests are | |
19 | `ignored` during testing. The stdout/stderr contents as well as a | |
20 | timestamp file for every test can be found under `build/ARCH/test/`. | |
21 | To force-rerun a test (e.g. in case the test runner fails to notice | |
22 | a change) you can simply remove the timestamp file. | |
23 | ||
532ac7d7 XL |
24 | Note that some tests require a Python-enabled gdb. You can test if |
25 | your gdb install supports Python by using the `python` command from | |
26 | within gdb. Once invoked you can type some Python code (e.g. | |
27 | `print("hi")`) followed by return and then `CTRL+D` to execute it. | |
28 | If you are building gdb from source, you will need to configure with | |
29 | `--with-python=<path-to-python-binary>`. | |
30 | ||
a1dfa0c6 XL |
31 | ## Running a subset of the test suites |
32 | ||
33 | When working on a specific PR, you will usually want to run a smaller | |
6a06907d XL |
34 | set of tests. For example, a good "smoke test" that can be used after |
35 | modifying rustc to see if things are generally working correctly would be the | |
36 | following: | |
a1dfa0c6 XL |
37 | |
38 | ```bash | |
6a06907d | 39 | ./x.py test src/test/ui |
a1dfa0c6 XL |
40 | ``` |
41 | ||
6a06907d XL |
42 | This will run the `ui` test suite. Of course, the choice |
43 | of test suites is somewhat arbitrary, and may not suit the task you are | |
44 | doing. For example, if you are hacking on debuginfo, you may be better off | |
45 | with the debuginfo test suite: | |
a1dfa0c6 XL |
46 | |
47 | ```bash | |
6a06907d | 48 | ./x.py test src/test/debuginfo |
a1dfa0c6 XL |
49 | ``` |
50 | ||
dc9dc135 XL |
51 | If you only need to test a specific subdirectory of tests for any |
52 | given test suite, you can pass that directory to `x.py test`: | |
53 | ||
54 | ```bash | |
6a06907d | 55 | ./x.py test src/test/ui/const-generics |
dc9dc135 XL |
56 | ``` |
57 | ||
58 | Likewise, you can test a single file by passing its path: | |
59 | ||
60 | ```bash | |
6a06907d | 61 | ./x.py test src/test/ui/const-generics/const-test.rs |
dc9dc135 XL |
62 | ``` |
63 | ||
a1dfa0c6 XL |
64 | ### Run only the tidy script |
65 | ||
66 | ```bash | |
74b04a01 | 67 | ./x.py test tidy |
a1dfa0c6 XL |
68 | ``` |
69 | ||
70 | ### Run tests on the standard library | |
71 | ||
72 | ```bash | |
6a06907d | 73 | ./x.py test --stage 0 library/std |
a1dfa0c6 XL |
74 | ``` |
75 | ||
74b04a01 | 76 | ### Run the tidy script and tests on the standard library |
a1dfa0c6 XL |
77 | |
78 | ```bash | |
6a06907d | 79 | ./x.py test --stage 0 tidy library/std |
a1dfa0c6 XL |
80 | ``` |
81 | ||
82 | ### Run tests on the standard library using a stage 1 compiler | |
83 | ||
84 | ```bash | |
6a06907d | 85 | ./x.py test library/std |
a1dfa0c6 XL |
86 | ``` |
87 | ||
88 | By listing which test suites you want to run you avoid having to run | |
89 | tests for components you did not change at all. | |
90 | ||
91 | **Warning:** Note that bors only runs the tests with the full stage 2 | |
92 | build; therefore, while the tests **usually** work fine with stage 1, | |
ba9703b0 | 93 | there are some limitations. |
a1dfa0c6 XL |
94 | |
95 | ## Running an individual test | |
96 | ||
97 | Another common thing that people want to do is to run an **individual | |
dc9dc135 XL |
98 | test**, often the test they are trying to fix. As mentioned earlier, |
99 | you may pass the full file path to achieve this, or alternatively one | |
100 | may invoke `x.py` with the `--test-args` option: | |
a1dfa0c6 XL |
101 | |
102 | ```bash | |
6a06907d | 103 | ./x.py test src/test/ui --test-args issue-1234 |
a1dfa0c6 XL |
104 | ``` |
105 | ||
106 | Under the hood, the test runner invokes the standard rust test runner | |
107 | (the same one you get with `#[test]`), so this command would wind up | |
dc9dc135 XL |
108 | filtering for tests that include "issue-1234" in the name. (Thus |
109 | `--test-args` is a good way to run a collection of related tests.) | |
a1dfa0c6 | 110 | |
74b04a01 XL |
111 | ## Editing and updating the reference files |
112 | ||
113 | If you have changed the compiler's output intentionally, or you are | |
114 | making a new test, you can pass `--bless` to the test subcommand. E.g. | |
115 | if some tests in `src/test/ui` are failing, you can run | |
116 | ||
117 | ```text | |
6a06907d | 118 | ./x.py test src/test/ui --bless |
74b04a01 XL |
119 | ``` |
120 | ||
121 | to automatically adjust the `.stderr`, `.stdout` or `.fixed` files of | |
122 | all tests. Of course you can also target just specific tests with the | |
123 | `--test-args your_test_name` flag, just like when running the tests. | |
124 | ||
416331ca XL |
125 | ## Passing `--pass $mode` |
126 | ||
127 | Pass UI tests now have three modes, `check-pass`, `build-pass` and | |
128 | `run-pass`. When `--pass $mode` is passed, these tests will be forced | |
129 | to run under the given `$mode` unless the directive `// ignore-pass` | |
130 | exists in the test file. For example, you can run all the tests in | |
131 | `src/test/ui` as `check-pass`: | |
132 | ||
133 | ```bash | |
6a06907d | 134 | ./x.py test src/test/ui --pass check |
416331ca XL |
135 | ``` |
136 | ||
137 | By passing `--pass $mode`, you can reduce the testing time. For each | |
138 | mode, please see [here][mode]. | |
139 | ||
60c5eb7d | 140 | [mode]: ./adding.md#tests-that-do-not-result-in-compile-errors |
416331ca | 141 | |
a1dfa0c6 XL |
142 | ## Using incremental compilation |
143 | ||
144 | You can further enable the `--incremental` flag to save additional | |
145 | time in subsequent rebuilds: | |
146 | ||
147 | ```bash | |
6a06907d | 148 | ./x.py test src/test/ui --incremental --test-args issue-1234 |
a1dfa0c6 XL |
149 | ``` |
150 | ||
151 | If you don't want to include the flag with every command, you can | |
6a06907d | 152 | enable it in the `config.toml`: |
a1dfa0c6 XL |
153 | |
154 | ```toml | |
6a06907d | 155 | [rust] |
a1dfa0c6 XL |
156 | incremental = true |
157 | ``` | |
158 | ||
159 | Note that incremental compilation will use more disk space than usual. | |
160 | If disk space is a concern for you, you might want to check the size | |
161 | of the `build` directory from time to time. | |
162 | ||
6a06907d XL |
163 | ## Running tests with different "compare modes" |
164 | ||
165 | UI tests may have different output depending on certain "modes" that | |
166 | the compiler is in. For example, when in "non-lexical lifetimes" (NLL) | |
167 | mode a test `foo.rs` will first look for expected output in | |
168 | `foo.nll.stderr`, falling back to the usual `foo.stderr` if not found. | |
169 | To run the UI test suite in NLL mode, one would use the following: | |
170 | ||
171 | ```bash | |
172 | ./x.py test src/test/ui --compare-mode=nll | |
173 | ``` | |
174 | ||
175 | Other examples of compare-modes are "noopt", "migrate", and | |
176 | [revisions](./adding.html#revisions). | |
177 | ||
a1dfa0c6 XL |
178 | ## Running tests manually |
179 | ||
180 | Sometimes it's easier and faster to just run the test by hand. Most tests are | |
181 | just `rs` files, so you can do something like | |
182 | ||
183 | ```bash | |
60c5eb7d | 184 | rustc +stage1 src/test/ui/issue-1234.rs |
a1dfa0c6 XL |
185 | ``` |
186 | ||
187 | This is much faster, but doesn't always work. For example, some tests | |
188 | include directives that specify specific compiler flags, or which rely | |
189 | on other crates, and they may not run the same without those options. |