src/doc/rustc-dev-guide/src/tests/running.md

   1 # Running tests
   2
   3 You can run the tests using `x.py`. The most basic command – which
   4 you will almost never want to use! – is as follows:
   5
   6 ```bash
   7 ./x.py test
   8 ```
   9
  10 This will build the full stage 2 compiler and then run the whole test
  11 suite. You probably don't want to do this very often, because it takes
  12 a very long time, and anyway bors / travis will do it for you. (Often,
  13 I will run this command in the background after opening a PR that I
  14 think is done, but rarely otherwise. -nmatsakis)
  15
  16 The test results are cached and previously successful tests are
  17 `ignored` during testing. The stdout/stderr contents as well as a
  18 timestamp file for every test can be found under `build/ARCH/test/`.
  19 To force-rerun a test (e.g. in case the test runner fails to notice
  20 a change) you can simply remove the timestamp file.
  21
  22 Note that some tests require a Python-enabled gdb. You can test if
  23 your gdb install supports Python by using the `python` command from
  24 within gdb. Once invoked you can type some Python code (e.g.
  25 `print("hi")`) followed by return and then `CTRL+D` to execute it.
  26 If you are building gdb from source, you will need to configure with
  27 `--with-python=<path-to-python-binary>`.
  28
  29 ## Running a subset of the test suites
  30
  31 When working on a specific PR, you will usually want to run a smaller
  32 set of tests, and with a stage 1 build. For example, a good "smoke
  33 test" that can be used after modifying rustc to see if things are
  34 generally working correctly would be the following:
  35
  36 ```bash
  37 ./x.py test --stage 1 src/test/{ui,compile-fail}
  38 ```
  39
  40 This will run the `ui` and `compile-fail` test suites,
  41 and only with the stage 1 build. Of course, the choice of test suites
  42 is somewhat arbitrary, and may not suit the task you are doing. For
  43 example, if you are hacking on debuginfo, you may be better off with
  44 the debuginfo test suite:
  45
  46 ```bash
  47 ./x.py test --stage 1 src/test/debuginfo
  48 ```
  49
  50 If you only need to test a specific subdirectory of tests for any
  51 given test suite, you can pass that directory to `x.py test`:
  52
  53 ```bash
  54 ./x.py test --stage 1 src/test/ui/const-generics
  55 ```
  56
  57 Likewise, you can test a single file by passing its path:
  58
  59 ```bash
  60 ./x.py test --stage 1 src/test/ui/const-generics/const-test.rs
  61 ```
  62
  63 ### Run only the tidy script
  64
  65 ```bash
  66 ./x.py test tidy
  67 ```
  68
  69 ### Run tests on the standard library
  70
  71 ```bash
  72 ./x.py test src/libstd
  73 ```
  74
  75 ### Run the tidy script and tests on the standard library
  76
  77 ```bash
  78 ./x.py test tidy src/libstd
  79 ```
  80
  81 ### Run tests on the standard library using a stage 1 compiler
  82
  83 ```bash
  84 >   ./x.py test src/libstd --stage 1
  85 ```
  86
  87 By listing which test suites you want to run you avoid having to run
  88 tests for components you did not change at all.
  89
  90 **Warning:** Note that bors only runs the tests with the full stage 2
  91 build; therefore, while the tests **usually** work fine with stage 1,
  92 there are some limitations.
  93
  94 ## Running an individual test
  95
  96 Another common thing that people want to do is to run an **individual
  97 test**, often the test they are trying to fix. As mentioned earlier,
  98 you may pass the full file path to achieve this, or alternatively one
  99 may invoke `x.py` with the `--test-args` option:
 100
 101 ```bash
 102 ./x.py test --stage 1 src/test/ui --test-args issue-1234
 103 ```
 104
 105 Under the hood, the test runner invokes the standard rust test runner
 106 (the same one you get with `#[test]`), so this command would wind up
 107 filtering for tests that include "issue-1234" in the name. (Thus
 108 `--test-args` is a good way to run a collection of related tests.)
 109
 110 ## Editing and updating the reference files
 111
 112 If you have changed the compiler's output intentionally, or you are
 113 making a new test, you can pass `--bless` to the test subcommand. E.g.
 114 if some tests in `src/test/ui` are failing, you can run
 115
 116 ```text
 117 ./x.py test --stage 1 src/test/ui --bless
 118 ```
 119
 120 to automatically adjust the `.stderr`, `.stdout` or `.fixed` files of
 121 all tests. Of course you can also target just specific tests with the
 122 `--test-args your_test_name` flag, just like when running the tests.
 123
 124 ## Passing `--pass $mode`
 125
 126 Pass UI tests now have three modes, `check-pass`, `build-pass` and
 127 `run-pass`. When `--pass $mode` is passed, these tests will be forced
 128 to run under the given `$mode` unless the directive `// ignore-pass`
 129 exists in the test file. For example, you can run all the tests in
 130 `src/test/ui` as `check-pass`:
 131
 132 ```bash
 133 ./x.py test --stage 1 src/test/ui --pass check
 134 ```
 135
 136 By passing `--pass $mode`, you can reduce the testing time. For each
 137 mode, please see [here][mode].
 138
 139 [mode]: ./adding.md#tests-that-do-not-result-in-compile-errors
 140
 141 ## Using incremental compilation
 142
 143 You can further enable the `--incremental` flag to save additional
 144 time in subsequent rebuilds:
 145
 146 ```bash
 147 ./x.py test --stage 1 src/test/ui --incremental --test-args issue-1234
 148 ```
 149
 150 If you don't want to include the flag with every command, you can
 151 enable it in the `config.toml`, too:
 152
 153 ```toml
 154 # Whether to always use incremental compilation when building rustc
 155 incremental = true
 156 ```
 157
 158 Note that incremental compilation will use more disk space than usual.
 159 If disk space is a concern for you, you might want to check the size
 160 of the `build` directory from time to time.
 161
 162 ## Running tests manually
 163
 164 Sometimes it's easier and faster to just run the test by hand. Most tests are
 165 just `rs` files, so you can do something like
 166
 167 ```bash
 168 rustc +stage1 src/test/ui/issue-1234.rs
 169 ```
 170
 171 This is much faster, but doesn't always work. For example, some tests
 172 include directives that specify specific compiler flags, or which rely
 173 on other crates, and they may not run the same without those options.