1 # Edk2 Continuous Integration
5 | Package | Windows VS2019 (IA32/X64)| Ubuntu GCC (IA32/X64/ARM/AARCH64) | Known Issues |
6 | :---- | :----- | :---- | :--- |
10 | CryptoPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
14 | FatPkg | :heavy_check_mark: | :heavy_check_mark: |
15 | FmpDevicePkg | :heavy_check_mark: | :heavy_check_mark: |
17 | IntelFsp2WrapperPkg |
18 | MdeModulePkg | :heavy_check_mark: | :heavy_check_mark: | DxeIpl dependency on ArmPkg, Depends on StandaloneMmPkg, Spell checking in audit mode
19 | MdePkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
20 | NetworkPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
22 | PcAtChipsetPkg | :heavy_check_mark: | :heavy_check_mark: |
23 | SecurityPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
24 | ShellPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 3 modules are not being built by DSC
26 | SourceLevelDebugPkg |
28 | UefiCpuPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 2 binary modules not being built by DSC
31 For more detailed status look at the test results of the latest CI run on the
36 This Continuous integration and testing infrastructure leverages the TianoCore EDKII Tools PIP modules:
37 [library](https://pypi.org/project/edk2-pytool-library/) and
38 [extensions](https://pypi.org/project/edk2-pytool-extensions/) (with repos
39 located [here](https://github.com/tianocore/edk2-pytool-library) and
40 [here](https://github.com/tianocore/edk2-pytool-extensions)).
42 The primary execution flows can be found in the
43 `.azurepipelines/Windows-VS2019.yml` and `.azurepipelines/Ubuntu-GCC5.yml`
44 files. These YAML files are consumed by the Azure Dev Ops Build Pipeline and
45 dictate what server resources should be used, how they should be configured, and
46 what processes should be run on them. An overview of this schema can be found
47 [here](https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema).
49 Inspection of these files reveals the EDKII Tools commands that make up the
50 primary processes for the CI build: 'stuart_setup', 'stuart_update', and
51 'stuart_ci_build'. These commands come from the EDKII Tools PIP modules and are
52 configured as described below. More documentation on the tools can be
53 found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md)
54 and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_invocables.md).
58 Configuration of the CI process consists of (in order of precedence):
60 * command-line arguments passed in via the Pipeline YAML
61 * a per-package configuration file (e.g. `<package-name>.ci.yaml`) that is
62 detected by the CI system in EDKII Tools.
63 * a global configuration Python module (e.g. `CISetting.py`) passed in via the
66 The global configuration file is described in
67 [this readme](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_settings_manager.md)
68 from the EDKII Tools documentation. This configuration is written as a Python
69 module so that decisions can be made dynamically based on command line
70 parameters and codebase state.
72 The per-package configuration file can override most settings in the global
73 configuration file, but is not dynamic. This file can be used to skip or
74 customize tests that may be incompatible with a specific package. Each test generally requires
75 per package configuration which comes from this file.
79 The EDKII Tools environment (and by extension the ci) is designed to support
80 easily and consistantly running locally and in a cloud ci environment. To do
81 that a few steps should be followed. Details of EDKII Tools can be found in the
82 [docs folder here](https://github.com/tianocore/edk2-pytool-extensions/tree/master/docs)
86 1. A supported toolchain (others might work but this is what is tested and validated)
89 * Windows SDK (for rc)
90 * Windows WDK (for capsules)
93 * Easy to add more but this is the current state
94 2. Python 3.7.x or newer on path
96 4. Recommended to setup and activate a python virtual environment
97 5. Install the requirements `pip install --upgrade pip-requirements.txt`
101 1. clone your edk2 repo
102 2. Activate your python virtual environment in cmd window
103 3. Get code dependencies (done only when submodules change)
104 * `stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`
105 4. Update other dependencies (done more often)
106 * `stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`
107 5. Run CI build (--help will give you options)
108 * `stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`
109 * -p <pkg1,pkg2,pkg3> : To build only certain packages use a CSV list
110 * -a <arch1,arch2,arch3>: To run only certain architectures use a CSV list
111 * -t <target1,target2>: To run only tests related to certain targets use a
113 * By default all tests are opted in. Then given a package.ci.yaml file those
114 tests can be configured for a package. Finally setting the check to the
115 value `skip` will skip that plugin. Examples:
116 * `CompilerPlugin=skip` skip the build test
117 * `GuidCheck=skip` skip the Guid check
118 * `SpellCheck=skip` skip the spell checker
120 6. Detailed reports and logs per package are captured in the `Build` directory
122 ## Current PyTool Test Capabilities
124 All CI tests are instances of EDKII Tools plugins. Documentation on the plugin
125 system can be found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_plugin_manager.md)
126 and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_plugin_manager.md).
127 Upon invocation, each plugin will be passed the path to the current package
128 under test and a dictionary containing its targeted configuration, as assembled
129 from the command line, per-package configuration, and global configuration.
131 Note: CI plugins are considered unique from build plugins and helper plugins,
132 even though some CI plugins may execute steps of a build.
134 In the example, these plugins live alongside the code under test (in the
135 `.pytool/Plugin` directory), but may be moved to the 'edk2-test' repo if that
136 location makes more sense for the community.
138 ### Module Inclusion Test - DscCompleteCheck
140 This test scans all available modules (via INF files) and compares them to the
141 package-level DSC file for the package each module is contained within. The test
142 considers it an error if any module does not appear in the `Components` section
143 of at least one package-level DSC (indicating that it would not be built if the
146 ### Code Compilation Test - CompilerPlugin
148 Once the Module Inclusion Test has verified that all modules would be built if
149 all package-level DSCs were built, the Code Compilation Test simply runs through
150 and builds every package-level DSC on every toolchain and for every architecture
151 that is supported. Any module that fails to build is considered an error.
153 ### GUID Uniqueness Test - GuidCheck
155 This test works on the collection of all packages rather than an individual
156 package. It looks at all FILE_GUIDs and GUIDs declared in DEC files and ensures
157 that they are unique for the codebase. This prevents, for example, accidental
158 duplication of GUIDs when using an existing INF as a template for a new module.
160 ### Cross-Package Dependency Test - DependencyCheck
162 This test compares the list of all packages used in INFs files for a given
163 package against a list of "allowed dependencies" in plugin configuration for
164 that package. Any module that depends on a disallowed package will cause a test
167 ### Library Declaration Test - LibraryClassCheck
169 This test scans at all library header files found in the `Library` folders in
170 all of the package's declared include directories and ensures that all files
171 have a matching LibraryClass declaration in the DEC file for the package. Any
172 missing declarations will cause a failure.
174 ### Invalid Character Test - CharEncodingCheck
176 This test scans all files in a package to make sure that there are no invalid
177 Unicode characters that may cause build errors in some character
180 ### Spell Checking - cspell
182 This test runs a spell checker on all files within the package. This is done
183 using the NodeJs cspell tool. For details check `.pytool/Plugin/SpellCheck`.
184 For this plugin to run during ci you must install nodejs and cspell and have
185 both available to the command line when running your CI.
189 * Install nodejs from https://nodejs.org/en/
191 1. Open cmd prompt with access to node and npm
192 2. Run `npm install -g cspell`
194 More cspell info: https://github.com/streetsidesoftware/cspell
198 Scopes are how the PyTool ext_dep, path_env, and plugins are activated. Meaning
199 that if an invocable process has a scope active then those ext_dep and path_env
200 will be active. To allow easy integration of PyTools capabilities there are a
203 | Scope | Invocable | Description |
204 | :---- | :----- | :---- |
205 | global | edk2_invocable++ - should be base_abstract_invocable | Running an invocables |
206 | global-win | edk2_invocable++ | Running on Microsoft Windows |
207 | global-nix | edk2_invocable++ | Running on Linux based OS |
208 | edk2-build | | This indicates that an invocable is building EDK2 based UEFI code |
209 | cibuild | set in .pytool/CISettings.py | Suggested target for edk2 continuous integration builds. Tools used for CiBuilds can use this scope. Example: asl compiler |
211 ## Future investments
213 * PatchCheck tests as plugins
214 * MacOS/xcode support
216 * Visual Studio AARCH64 and ARM support
217 * BaseTools C tools CI/PR and binary release process
218 * BaseTools Python tools CI/PR process
219 * Host based unit testing
220 * Extensible private/closed source platform reporting
221 * Platform builds, validation