]>
Commit | Line | Data |
---|---|---|
a5ebaaf9 AS |
1 | ============================================================================= |
2 | ACPI/SMBIOS avocado tests using biosbits | |
3 | ============================================================================= | |
4 | ||
5 | Biosbits is a software written by Josh Triplett that can be downloaded | |
6 | from https://biosbits.org/. The github codebase can be found | |
7 | `here <https://github.com/biosbits/bits/tree/master>`__. It is a software that executes | |
8 | the bios components such as acpi and smbios tables directly through acpica | |
9 | bios interpreter (a freely available C based library written by Intel, | |
10 | downloadable from https://acpica.org/ and is included with biosbits) without an | |
11 | operating system getting involved in between. | |
12 | There are several advantages to directly testing the bios in a real physical | |
13 | machine or VM as opposed to indirectly discovering bios issues through the | |
14 | operating system. For one thing, the OSes tend to hide bios problems from the | |
15 | end user. The other is that we have more control of what we wanted to test | |
16 | and how by directly using acpica interpreter on top of the bios on a running | |
17 | system. More details on the inspiration for developing biosbits and its real | |
18 | life uses can be found in [#a]_ and [#b]_. | |
a5ebaaf9 | 19 | For QEMU, we maintain a fork of bios bits in gitlab along with all the |
1b7a07c4 | 20 | dependent submodules here: https://gitlab.com/qemu-project/biosbits-bits |
a5ebaaf9 AS |
21 | This fork contains numerous fixes, a newer acpica and changes specific to |
22 | running this avocado QEMU tests using bits. The author of this document | |
23 | is the sole maintainer of the QEMU fork of bios bits repo. | |
24 | ||
25 | Under the directory ``tests/avocado/``, ``acpi-bits.py`` is a QEMU avocado | |
26 | test that drives all this. | |
27 | ||
28 | A brief description of the various test files follows. | |
29 | ||
30 | Under ``tests/avocado/`` as the root we have: | |
31 | ||
32 | :: | |
33 | ||
34 | ├── acpi-bits | |
35 | │ ├── bits-config | |
36 | │ │ └── bits-cfg.txt | |
37 | │ ├── bits-tests | |
1b7a07c4 AS |
38 | │ ├── smbios.py2 |
39 | │ ├── testacpi.py2 | |
40 | │ └── testcpuid.py2 | |
a5ebaaf9 AS |
41 | ├── acpi-bits.py |
42 | ||
43 | * ``tests/avocado``: | |
44 | ||
45 | ``acpi-bits.py``: | |
46 | This is the main python avocado test script that generates a | |
47 | biosbits iso. It then spawns a QEMU VM with it, collects the log and reports | |
48 | test failures. This is the script one would be interested in if they wanted | |
49 | to add or change some component of the log parsing, add a new command line | |
50 | to alter how QEMU is spawned etc. Test writers typically would not need to | |
51 | modify this script unless they wanted to enhance or change the log parsing | |
52 | for their tests. In order to enable debugging, you can set **V=1** | |
53 | environment variable. This enables verbose mode for the test and also dumps | |
54 | the entire log from bios bits and more information in case failure happens. | |
65809a60 AS |
55 | You can also set **BITS_DEBUG=1** to turn on debug mode. It will enable |
56 | verbose logs and also retain the temporary work directory the test used for | |
57 | you to inspect and run the specific commands manually. | |
a5ebaaf9 AS |
58 | |
59 | In order to run this test, please perform the following steps from the QEMU | |
60 | build directory: | |
61 | :: | |
62 | ||
63 | $ make check-venv (needed only the first time to create the venv) | |
e8e4298f | 64 | $ ./tests/venv/bin/avocado run -t acpi tests/avocado |
a5ebaaf9 AS |
65 | |
66 | The above will run all acpi avocado tests including this one. | |
67 | In order to run the individual tests, perform the following: | |
68 | :: | |
69 | ||
e8e4298f | 70 | $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py --tap - |
a5ebaaf9 AS |
71 | |
72 | The above will produce output in tap format. You can omit "--tap -" in the | |
73 | end and it will produce output like the following: | |
74 | :: | |
75 | ||
e8e4298f | 76 | $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py |
a5ebaaf9 AS |
77 | Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits |
78 | JOB ID : eab225724da7b64c012c65705dc2fa14ab1defef | |
79 | JOB LOG : /home/anisinha/avocado/job-results/job-2022-10-10T17.58-eab2257/job.log | |
80 | (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS (33.09 s) | |
81 | RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 | |
82 | JOB TIME : 39.22 s | |
83 | ||
84 | You can inspect the log file for more information about the run or in order | |
85 | to diagnoze issues. If you pass V=1 in the environment, more diagnostic logs | |
86 | would be found in the test log. | |
87 | ||
88 | * ``tests/avocado/acpi-bits/bits-config``: | |
89 | ||
90 | This location contains biosbits configuration files that determine how the | |
91 | software runs the tests. | |
92 | ||
93 | ``bits-config.txt``: | |
94 | This is the biosbits config file that determines what tests | |
95 | or actions are performed by bits. The description of the config options are | |
96 | provided in the file itself. | |
97 | ||
98 | * ``tests/avocado/acpi-bits/bits-tests``: | |
99 | ||
100 | This directory contains biosbits python based tests that are run from within | |
101 | the biosbits environment in the spawned VM. New additions of test cases can | |
102 | be made in the appropriate test file. For example, new acpi tests can go | |
103 | into testacpi.py2 and one would call testsuite.add_test() to register the new | |
104 | test so that it gets executed as a part of the ACPI tests. | |
105 | It might be occasionally necessary to disable some subtests or add a new | |
106 | test that belongs to a test suite not already present in this directory. To | |
107 | do this, please clone the bits source from | |
108 | https://gitlab.com/qemu-project/biosbits-bits/-/tree/qemu-bits. | |
109 | Note that this is the "qemu-bits" branch and not the "bits" branch of the | |
110 | repository. "qemu-bits" is the branch where we have made all the QEMU | |
111 | specific enhancements and we must use the source from this branch only. | |
112 | Copy the test suite/script that needs modification (addition of new tests | |
113 | or disabling them) from python directory into this directory. For | |
114 | example, in order to change cpuid related tests, copy the following | |
115 | file into this directory and rename it with .py2 extension: | |
116 | https://gitlab.com/qemu-project/biosbits-bits/-/blob/qemu-bits/python/testcpuid.py | |
117 | Then make your additions and changes here. Therefore, the steps are: | |
118 | ||
119 | (a) Copy unmodified test script to this directory from bits source. | |
120 | (b) Add a SPDX license header. | |
121 | (c) Perform modifications to the test. | |
122 | ||
123 | Commits (a), (b) and (c) should go under separate commits so that the original | |
124 | test script and the changes we have made are separated and clear. | |
125 | ||
126 | The test framework will then use your modified test script to run the test. | |
127 | No further changes would be needed. Please check the logs to make sure that | |
128 | appropriate changes have taken effect. | |
129 | ||
130 | The tests have an extension .py2 in order to indicate that: | |
131 | ||
132 | (a) They are python2.7 based scripts and not python 3 scripts. | |
133 | (b) They are run from within the bios bits VM and is not subjected to QEMU | |
2cb40d44 | 134 | build/test python script maintenance and dependency resolutions. |
a5ebaaf9 AS |
135 | (c) They need not be loaded by avocado framework when running tests. |
136 | ||
137 | ||
607a079b | 138 | Author: Ani Sinha <anisinha@redhat.com> |
a5ebaaf9 AS |
139 | |
140 | References: | |
141 | ----------- | |
142 | .. [#a] https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf | |
143 | .. [#b] https://www.youtube.com/watch?v=36QIepyUuhg | |
144 |