]>
Commit | Line | Data |
---|---|---|
4eb2baba SB |
1 | # Edk2 Continuous Integration\r |
2 | \r | |
3 | ## Basic Status\r | |
4 | \r | |
4403bbd7 BB |
5 | | Package | Windows VS2019 (IA32/X64)| Ubuntu GCC (IA32/X64/ARM/AARCH64) | Known Issues |\r |
6 | | :---- | :----- | :---- | :--- |\r | |
2942cb58 | 7 | | ArmPkg | | :heavy_check_mark: |\r |
a4cf1959 | 8 | | ArmPlatformPkg | | :heavy_check_mark: |\r |
4403bbd7 BB |
9 | | ArmVirtPkg | SEE PACKAGE README | SEE PACKAGE README |\r |
10 | | CryptoPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r | |
82c65f14 | 11 | | DynamicTablesPkg | | :heavy_check_mark: |\r |
4403bbd7 BB |
12 | | EmbeddedPkg |\r |
13 | | EmulatorPkg | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode\r | |
14 | | FatPkg | :heavy_check_mark: | :heavy_check_mark: |\r | |
15 | | FmpDevicePkg | :heavy_check_mark: | :heavy_check_mark: |\r | |
16 | | IntelFsp2Pkg |\r | |
17 | | IntelFsp2WrapperPkg |\r | |
18 | | MdeModulePkg | :heavy_check_mark: | :heavy_check_mark: | DxeIpl dependency on ArmPkg, Depends on StandaloneMmPkg, Spell checking in audit mode\r | |
19 | | MdePkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r | |
20 | | NetworkPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r | |
21 | | OvmfPkg | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode\r | |
22 | | PcAtChipsetPkg | :heavy_check_mark: | :heavy_check_mark: |\r | |
23 | | SecurityPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r | |
24 | | ShellPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 3 modules are not being built by DSC\r | |
25 | | SignedCapsulePkg |\r | |
26 | | SourceLevelDebugPkg |\r | |
d8d1f666 | 27 | | StandaloneMmPkg | :heavy_check_mark: | :heavy_check_mark: |\r |
4403bbd7 BB |
28 | | UefiCpuPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 2 binary modules not being built by DSC\r |
29 | | UefiPayloadPkg |\r | |
30 | | UnitTestFrameworkPkg | :heavy_check_mark: | :heavy_check_mark: |\r | |
4eb2baba SB |
31 | \r |
32 | For more detailed status look at the test results of the latest CI run on the\r | |
33 | repo readme.\r | |
34 | \r | |
35 | ## Background\r | |
36 | \r | |
37 | This Continuous integration and testing infrastructure leverages the TianoCore EDKII Tools PIP modules:\r | |
38 | [library](https://pypi.org/project/edk2-pytool-library/) and\r | |
39 | [extensions](https://pypi.org/project/edk2-pytool-extensions/) (with repos\r | |
40 | located [here](https://github.com/tianocore/edk2-pytool-library) and\r | |
41 | [here](https://github.com/tianocore/edk2-pytool-extensions)).\r | |
42 | \r | |
43 | The primary execution flows can be found in the\r | |
44 | `.azurepipelines/Windows-VS2019.yml` and `.azurepipelines/Ubuntu-GCC5.yml`\r | |
45 | files. These YAML files are consumed by the Azure Dev Ops Build Pipeline and\r | |
46 | dictate what server resources should be used, how they should be configured, and\r | |
47 | what processes should be run on them. An overview of this schema can be found\r | |
48 | [here](https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema).\r | |
49 | \r | |
50 | Inspection of these files reveals the EDKII Tools commands that make up the\r | |
51 | primary processes for the CI build: 'stuart_setup', 'stuart_update', and\r | |
52 | 'stuart_ci_build'. These commands come from the EDKII Tools PIP modules and are\r | |
53 | configured as described below. More documentation on the tools can be\r | |
54 | found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md)\r | |
55 | and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_invocables.md).\r | |
56 | \r | |
57 | ## Configuration\r | |
58 | \r | |
59 | Configuration of the CI process consists of (in order of precedence):\r | |
60 | \r | |
61 | * command-line arguments passed in via the Pipeline YAML\r | |
62 | * a per-package configuration file (e.g. `<package-name>.ci.yaml`) that is\r | |
63 | detected by the CI system in EDKII Tools.\r | |
64 | * a global configuration Python module (e.g. `CISetting.py`) passed in via the\r | |
65 | command-line\r | |
66 | \r | |
67 | The global configuration file is described in\r | |
68 | [this readme](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_settings_manager.md)\r | |
69 | from the EDKII Tools documentation. This configuration is written as a Python\r | |
70 | module so that decisions can be made dynamically based on command line\r | |
71 | parameters and codebase state.\r | |
72 | \r | |
73 | The per-package configuration file can override most settings in the global\r | |
74 | configuration file, but is not dynamic. This file can be used to skip or\r | |
75 | customize tests that may be incompatible with a specific package. Each test generally requires\r | |
76 | per package configuration which comes from this file.\r | |
77 | \r | |
78 | ## Running CI locally\r | |
79 | \r | |
80 | The EDKII Tools environment (and by extension the ci) is designed to support\r | |
0358c0bf | 81 | easily and consistently running locally and in a cloud ci environment. To do\r |
4eb2baba SB |
82 | that a few steps should be followed. Details of EDKII Tools can be found in the\r |
83 | [docs folder here](https://github.com/tianocore/edk2-pytool-extensions/tree/master/docs)\r | |
84 | \r | |
85 | ### Prerequisets\r | |
86 | \r | |
87 | 1. A supported toolchain (others might work but this is what is tested and validated)\r | |
88 | * Windows 10:\r | |
89 | * VS 2017 or VS 2019\r | |
90 | * Windows SDK (for rc)\r | |
91 | * Windows WDK (for capsules)\r | |
4403bbd7 | 92 | * Ubuntu 18.04 or Fedora\r |
4eb2baba SB |
93 | * GCC5\r |
94 | * Easy to add more but this is the current state\r | |
95 | 2. Python 3.7.x or newer on path\r | |
96 | 3. git on path\r | |
97 | 4. Recommended to setup and activate a python virtual environment\r | |
98 | 5. Install the requirements `pip install --upgrade pip-requirements.txt`\r | |
99 | \r | |
100 | ### Running CI\r | |
101 | \r | |
102 | 1. clone your edk2 repo\r | |
103 | 2. Activate your python virtual environment in cmd window\r | |
104 | 3. Get code dependencies (done only when submodules change)\r | |
105 | * `stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`\r | |
106 | 4. Update other dependencies (done more often)\r | |
107 | * `stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`\r | |
108 | 5. Run CI build (--help will give you options)\r | |
109 | * `stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`\r | |
110 | * -p <pkg1,pkg2,pkg3> : To build only certain packages use a CSV list\r | |
111 | * -a <arch1,arch2,arch3>: To run only certain architectures use a CSV list\r | |
112 | * -t <target1,target2>: To run only tests related to certain targets use a\r | |
113 | CSV list\r | |
114 | * By default all tests are opted in. Then given a package.ci.yaml file those\r | |
115 | tests can be configured for a package. Finally setting the check to the\r | |
116 | value `skip` will skip that plugin. Examples:\r | |
117 | * `CompilerPlugin=skip` skip the build test\r | |
118 | * `GuidCheck=skip` skip the Guid check\r | |
119 | * `SpellCheck=skip` skip the spell checker\r | |
120 | * etc\r | |
121 | 6. Detailed reports and logs per package are captured in the `Build` directory\r | |
122 | \r | |
123 | ## Current PyTool Test Capabilities\r | |
124 | \r | |
125 | All CI tests are instances of EDKII Tools plugins. Documentation on the plugin\r | |
126 | system can be found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_plugin_manager.md)\r | |
127 | and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_plugin_manager.md).\r | |
128 | Upon invocation, each plugin will be passed the path to the current package\r | |
129 | under test and a dictionary containing its targeted configuration, as assembled\r | |
130 | from the command line, per-package configuration, and global configuration.\r | |
131 | \r | |
132 | Note: CI plugins are considered unique from build plugins and helper plugins,\r | |
133 | even though some CI plugins may execute steps of a build.\r | |
134 | \r | |
135 | In the example, these plugins live alongside the code under test (in the\r | |
136 | `.pytool/Plugin` directory), but may be moved to the 'edk2-test' repo if that\r | |
137 | location makes more sense for the community.\r | |
138 | \r | |
139 | ### Module Inclusion Test - DscCompleteCheck\r | |
140 | \r | |
4403bbd7 BB |
141 | This scans all INF files from a package and confirms they are\r |
142 | listed in the package level DSC file. The test considers it an error if any INF\r | |
143 | does not appear in the `Components` section of the package-level DSC (indicating\r | |
144 | that it would not be built if the package were built). This is critical because\r | |
145 | much of the CI infrastructure assumes that all modules will be listed in the DSC\r | |
146 | and compiled.\r | |
147 | \r | |
148 | This test will ignore INFs in the following cases:\r | |
149 | \r | |
150 | 1. When `MODULE_TYPE` = `HOST_APPLICATION`\r | |
151 | 2. When a Library instance **only** supports the `HOST_APPLICATION` environment\r | |
152 | \r | |
153 | ### Host Module Inclusion Test - HostUnitTestDscCompleteCheck\r | |
154 | \r | |
155 | This test scans all INF files from a package for those related to host\r | |
156 | based unit tests and confirms they are listed in the unit test DSC file for the package.\r | |
157 | The test considers it an error if any INF meeting the requirements does not appear\r | |
158 | in the `Components` section of the unit test DSC. This is critical because\r | |
159 | much of the CI infrastructure assumes that modules will be listed in the DSC\r | |
160 | and compiled.\r | |
161 | \r | |
162 | This test will only require INFs in the following cases:\r | |
163 | \r | |
164 | 1. When `MODULE_TYPE` = `HOST_APPLICATION`\r | |
165 | 2. When a Library instance explicitly supports the `HOST_APPLICATION` environment\r | |
4eb2baba SB |
166 | \r |
167 | ### Code Compilation Test - CompilerPlugin\r | |
168 | \r | |
169 | Once the Module Inclusion Test has verified that all modules would be built if\r | |
170 | all package-level DSCs were built, the Code Compilation Test simply runs through\r | |
171 | and builds every package-level DSC on every toolchain and for every architecture\r | |
172 | that is supported. Any module that fails to build is considered an error.\r | |
173 | \r | |
4403bbd7 BB |
174 | ### Host Unit Test Compilation and Run Test - HostUnitTestCompilerPlugin\r |
175 | \r | |
176 | A test that compiles the dsc for host based unit test apps.\r | |
177 | On Windows this will also enable a build plugin to execute that will run the unit tests and verify the results.\r | |
178 | \r | |
179 | These tools will be invoked on any CI\r | |
180 | pass that includes the NOOPT target. In order for these tools to do their job,\r | |
181 | the package and tests must be configured in a particular way...\r | |
182 | \r | |
183 | #### Including Host-Based Tests in the Package YAML\r | |
184 | \r | |
185 | For example, looking at the `MdeModulePkg.ci.yaml` config file, there are two\r | |
186 | config options that control HostBased test behavior:\r | |
187 | \r | |
188 | ```json\r | |
189 | ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin\r | |
190 | "HostUnitTestCompilerPlugin": {\r | |
191 | "DscPath": "Test/MdeModulePkgHostTest.dsc"\r | |
192 | },\r | |
193 | ```\r | |
194 | \r | |
195 | This option tell the test builder to run. The test builder needs to know which\r | |
196 | modules in this package are host-based tests, so that DSC path is provided.\r | |
197 | \r | |
198 | #### Configuring the HostBased DSC\r | |
199 | \r | |
200 | The HostBased DSC for `MdeModulePkg` is located at\r | |
201 | `MdeModulePkg/Test/MdeModulePkgHostTest.dsc`.\r | |
202 | \r | |
203 | To add automated host-based unit test building to a new package, create a\r | |
204 | similar DSC. The new DSC should make sure to have the `NOOPT` BUILD_TARGET\r | |
205 | and should include the line:\r | |
206 | \r | |
207 | ```\r | |
208 | !include UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc\r | |
209 | ```\r | |
210 | \r | |
211 | All of the modules that are included in the `Components` section of this\r | |
212 | DSC should be of type HOST_APPLICATION.\r | |
213 | \r | |
4eb2baba SB |
214 | ### GUID Uniqueness Test - GuidCheck\r |
215 | \r | |
216 | This test works on the collection of all packages rather than an individual\r | |
217 | package. It looks at all FILE_GUIDs and GUIDs declared in DEC files and ensures\r | |
218 | that they are unique for the codebase. This prevents, for example, accidental\r | |
219 | duplication of GUIDs when using an existing INF as a template for a new module.\r | |
220 | \r | |
221 | ### Cross-Package Dependency Test - DependencyCheck\r | |
222 | \r | |
223 | This test compares the list of all packages used in INFs files for a given\r | |
224 | package against a list of "allowed dependencies" in plugin configuration for\r | |
225 | that package. Any module that depends on a disallowed package will cause a test\r | |
226 | failure.\r | |
227 | \r | |
228 | ### Library Declaration Test - LibraryClassCheck\r | |
229 | \r | |
230 | This test scans at all library header files found in the `Library` folders in\r | |
231 | all of the package's declared include directories and ensures that all files\r | |
232 | have a matching LibraryClass declaration in the DEC file for the package. Any\r | |
233 | missing declarations will cause a failure.\r | |
234 | \r | |
235 | ### Invalid Character Test - CharEncodingCheck\r | |
236 | \r | |
237 | This test scans all files in a package to make sure that there are no invalid\r | |
238 | Unicode characters that may cause build errors in some character\r | |
239 | sets/localizations.\r | |
240 | \r | |
241 | ### Spell Checking - cspell\r | |
242 | \r | |
243 | This test runs a spell checker on all files within the package. This is done\r | |
244 | using the NodeJs cspell tool. For details check `.pytool/Plugin/SpellCheck`.\r | |
245 | For this plugin to run during ci you must install nodejs and cspell and have\r | |
246 | both available to the command line when running your CI.\r | |
247 | \r | |
248 | Install\r | |
249 | \r | |
250 | * Install nodejs from https://nodejs.org/en/\r | |
251 | * Install cspell\r | |
252 | 1. Open cmd prompt with access to node and npm\r | |
253 | 2. Run `npm install -g cspell`\r | |
254 | \r | |
255 | More cspell info: https://github.com/streetsidesoftware/cspell\r | |
256 | \r | |
ec9683ec PG |
257 | ### License Checking - LicenseCheck\r |
258 | \r | |
259 | Scans all new added files in a package to make sure code is contributed under\r | |
260 | BSD-2-Clause-Patent.\r | |
261 | \r | |
262 | ### Ecc tool - EccCheck\r | |
263 | \r | |
264 | Run the Ecc tool on the package. The Ecc tool is available in the BaseTools\r | |
265 | package. It checks that the code complies to the EDKII coding standard.\r | |
266 | \r | |
dc453b51 MK |
267 | ### Coding Standard Compliance - UncrustifyCheck\r |
268 | \r | |
269 | Runs the Uncrustify application to check for coding standard compliance issues.\r | |
270 | \r | |
4eb2baba SB |
271 | ## PyTool Scopes\r |
272 | \r | |
273 | Scopes are how the PyTool ext_dep, path_env, and plugins are activated. Meaning\r | |
274 | that if an invocable process has a scope active then those ext_dep and path_env\r | |
275 | will be active. To allow easy integration of PyTools capabilities there are a\r | |
276 | few standard scopes.\r | |
277 | \r | |
278 | | Scope | Invocable | Description |\r | |
279 | | :---- | :----- | :---- |\r | |
280 | | global | edk2_invocable++ - should be base_abstract_invocable | Running an invocables |\r | |
281 | | global-win | edk2_invocable++ | Running on Microsoft Windows |\r | |
282 | | global-nix | edk2_invocable++ | Running on Linux based OS |\r | |
283 | | edk2-build | | This indicates that an invocable is building EDK2 based UEFI code |\r | |
284 | | cibuild | set in .pytool/CISettings.py | Suggested target for edk2 continuous integration builds. Tools used for CiBuilds can use this scope. Example: asl compiler |\r | |
4403bbd7 BB |
285 | | host-based-test | set in .pytool/CISettings.py | Turns on the host based tests and plugin |\r |
286 | | host-test-win | set in .pytool/CISettings.py | Enables the host based test runner for Windows |\r | |
4eb2baba SB |
287 | \r |
288 | ## Future investments\r | |
289 | \r | |
290 | * PatchCheck tests as plugins\r | |
291 | * MacOS/xcode support\r | |
292 | * Clang/LLVM support\r | |
293 | * Visual Studio AARCH64 and ARM support\r | |
294 | * BaseTools C tools CI/PR and binary release process\r | |
295 | * BaseTools Python tools CI/PR process\r | |
4eb2baba | 296 | * Extensible private/closed source platform reporting\r |
4eb2baba SB |
297 | * UEFI SCTs\r |
298 | * Other automation\r |