]>
Commit | Line | Data |
---|---|---|
7cac9316 XL |
1 | ======================================== |
2 | Compiler-rt Testing Infrastructure Guide | |
3 | ======================================== | |
4 | ||
5 | .. contents:: | |
6 | :local: | |
7 | ||
8 | Overview | |
9 | ======== | |
10 | ||
11 | This document is the reference manual for the compiler-rt modifications to the | |
12 | testing infrastructure. Documentation for the infrastructure itself can be found at | |
13 | :ref:`llvm_testing_guide`. | |
14 | ||
15 | LLVM testing infrastructure organization | |
16 | ======================================== | |
17 | ||
18 | The compiler-rt testing infrastructure contains regression tests which are run | |
19 | as part of the usual ``make check-all`` and are expected to always pass -- they | |
20 | should be run before every commit. | |
21 | ||
22 | Quick start | |
23 | =========== | |
24 | ||
25 | The regressions tests are in the "compiler-rt" module and are normally checked | |
26 | out in the directory ``llvm/projects/compiler-rt/test``. Use ``make check-all`` | |
27 | to run the regression tests after building compiler-rt. | |
28 | ||
29 | REQUIRES, XFAIL, etc. | |
30 | --------------------- | |
31 | ||
32 | Sometimes it is necessary to restrict a test to a specific target or mark it as | |
33 | an "expected fail" or XFAIL. This is normally achieved using ``REQUIRES:`` or | |
34 | ``XFAIL:`` with a substring of LLVM's default target triple. Unfortunately, the | |
35 | behaviour of this is somewhat quirky in compiler-rt. There are two main | |
36 | pitfalls to avoid. | |
37 | ||
38 | The first pitfall is that these directives perform a substring match on the | |
39 | triple and as such ``XFAIL: mips`` affects more triples than expected. For | |
40 | example, ``mips-linux-gnu``, ``mipsel-linux-gnu``, ``mips64-linux-gnu``, and | |
41 | ``mips64el-linux-gnu`` will all match a ``XFAIL: mips`` directive. Including a | |
42 | trailing ``-`` such as in ``XFAIL: mips-`` can help to mitigate this quirk but | |
43 | even that has issues as described below. | |
44 | ||
45 | The second pitfall is that the default target triple is often inappropriate for | |
46 | compiler-rt tests since compiler-rt tests may be compiled for multiple targets. | |
47 | For example, a typical build on an ``x86_64-linux-gnu`` host will often run the | |
48 | tests for both x86_64 and i386. In this situation ``XFAIL: x86_64`` will mark | |
49 | both the x86_64 and i386 tests as an expected failure while ``XFAIL: i386`` | |
50 | will have no effect at all. | |
51 | ||
52 | To remedy both pitfalls, compiler-rt tests provide a feature string which can | |
53 | be used to specify a single target. This string is of the form | |
54 | ``target-is-${arch}`` where ``${arch}}`` is one of the values from the | |
55 | following lines of the CMake output:: | |
56 | ||
57 | -- Compiler-RT supported architectures: x86_64;i386 | |
58 | -- Builtin supported architectures: i386;x86_64 | |
59 | ||
60 | So for example ``XFAIL: target-is-x86_64`` will mark a test as expected to fail | |
61 | on x86_64 without also affecting the i386 test and ``XFAIL: target-is-i386`` | |
62 | will mark a test as expected to fail on i386 even if the default target triple | |
63 | is ``x86_64-linux-gnu``. Directives that use these ``target-is-${arch}`` string | |
64 | require exact matches so ``XFAIL: target-is-mips``, | |
65 | ``XFAIL: target-is-mipsel``, ``XFAIL: target-is-mips64``, and | |
66 | ``XFAIL: target-is-mips64el`` all refer to different MIPS targets. |