.pytool/Readme.md

   1 # Edk2 Continuous Integration
   2
   3 ## Basic Status
   4
   5 | Package             | Windows VS2019 (IA32/X64)| Ubuntu GCC (IA32/X64/ARM/AARCH64) | Known Issues |
   6 | :----               | :-----                   | :----                             | :---         |
   7 | ArmPkg              |
   8 | ArmPlatformPkg      |
   9 | ArmVirtPkg          | SEE PACKAGE README | SEE PACKAGE README |
  10 | CryptoPkg           | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
  11 | DynamicTablesPkg    |
  12 | EmbeddedPkg         |
  13 | EmulatorPkg         | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode
  14 | FatPkg              | :heavy_check_mark: | :heavy_check_mark: |
  15 | FmpDevicePkg        | :heavy_check_mark: | :heavy_check_mark: |
  16 | IntelFsp2Pkg        |
  17 | IntelFsp2WrapperPkg |
  18 | MdeModulePkg        | :heavy_check_mark: | :heavy_check_mark: | DxeIpl dependency on ArmPkg, Depends on StandaloneMmPkg, Spell checking in audit mode
  19 | MdePkg              | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
  20 | NetworkPkg          | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
  21 | OvmfPkg             | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode
  22 | PcAtChipsetPkg      | :heavy_check_mark: | :heavy_check_mark: |
  23 | SecurityPkg         | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
  24 | ShellPkg            | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 3 modules are not being built by DSC
  25 | SignedCapsulePkg    |
  26 | SourceLevelDebugPkg |
  27 | StandaloneMmPkg     |
  28 | UefiCpuPkg          | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 2 binary modules not being built by DSC
  29 | UefiPayloadPkg      |
  30
  31 For more detailed status look at the test results of the latest CI run on the
  32 repo readme.
  33
  34 ## Background
  35
  36 This Continuous integration and testing infrastructure leverages the TianoCore EDKII Tools PIP modules:
  37 [library](https://pypi.org/project/edk2-pytool-library/) and
  38 [extensions](https://pypi.org/project/edk2-pytool-extensions/) (with repos
  39 located [here](https://github.com/tianocore/edk2-pytool-library) and
  40 [here](https://github.com/tianocore/edk2-pytool-extensions)).
  41
  42 The primary execution flows can be found in the
  43 `.azurepipelines/Windows-VS2019.yml` and `.azurepipelines/Ubuntu-GCC5.yml`
  44 files. These YAML files are consumed by the Azure Dev Ops Build Pipeline and
  45 dictate what server resources should be used, how they should be configured, and
  46 what processes should be run on them. An overview of this schema can be found
  47 [here](https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema).
  48
  49 Inspection of these files reveals the EDKII Tools commands that make up the
  50 primary processes for the CI build: 'stuart_setup', 'stuart_update', and
  51 'stuart_ci_build'. These commands come from the EDKII Tools PIP modules and are
  52 configured as described below. More documentation on the tools can be
  53 found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md)
  54 and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_invocables.md).
  55
  56 ## Configuration
  57
  58 Configuration of the CI process consists of (in order of precedence):
  59
  60 * command-line arguments passed in via the Pipeline YAML
  61 * a per-package configuration file (e.g. `<package-name>.ci.yaml`) that is
  62   detected by the CI system in EDKII Tools.
  63 * a global configuration Python module (e.g. `CISetting.py`) passed in via the
  64   command-line
  65
  66 The global configuration file is described in
  67 [this readme](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_settings_manager.md)
  68 from the EDKII Tools documentation. This configuration is written as a Python
  69 module so that decisions can be made dynamically based on command line
  70 parameters and codebase state.
  71
  72 The per-package configuration file can override most settings in the global
  73 configuration file, but is not dynamic. This file can be used to skip or
  74 customize tests that may be incompatible with a specific package.  Each test generally requires
  75 per package configuration which comes from this file.
  76
  77 ## Running CI locally
  78
  79 The EDKII Tools environment (and by extension the ci) is designed to support
  80 easily and consistently running locally and in a cloud ci environment.  To do
  81 that a few steps should be followed.  Details of EDKII Tools can be found in the
  82 [docs folder here](https://github.com/tianocore/edk2-pytool-extensions/tree/master/docs)
  83
  84 ### Prerequisets
  85
  86 1. A supported toolchain (others might work but this is what is tested and validated)
  87    * Windows 10:
  88      * VS 2017 or VS 2019
  89      * Windows SDK (for rc)
  90      * Windows WDK (for capsules)
  91    * Ubuntu 16.04
  92      * GCC5
  93    * Easy to add more but this is the current state
  94 2. Python 3.7.x or newer on path
  95 3. git on path
  96 4. Recommended to setup and activate a python virtual environment
  97 5. Install the requirements `pip install --upgrade pip-requirements.txt`
  98
  99 ### Running CI
 100
 101 1. clone your edk2 repo
 102 2. Activate your python virtual environment in cmd window
 103 3. Get code dependencies (done only when submodules change)
 104    * `stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`
 105 4. Update other dependencies (done more often)
 106    * `stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`
 107 5. Run CI build (--help will give you options)
 108    * `stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`
 109    * -p <pkg1,pkg2,pkg3> : To build only certain packages use a CSV list
 110    * -a <arch1,arch2,arch3>: To run only certain architectures use a CSV list
 111    * -t <target1,target2>: To run only tests related to certain targets use a
 112      CSV list
 113    * By default all tests are opted in.  Then given a package.ci.yaml file those
 114      tests can be configured for a package. Finally setting the check to the
 115      value `skip` will skip that plugin.  Examples:
 116      * `CompilerPlugin=skip` skip the build test
 117      * `GuidCheck=skip` skip the Guid check
 118      * `SpellCheck=skip` skip the spell checker
 119      * etc
 120 6. Detailed reports and logs per package are captured in the `Build` directory
 121
 122 ## Current PyTool Test Capabilities
 123
 124 All CI tests are instances of EDKII Tools plugins. Documentation on the plugin
 125 system can be found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_plugin_manager.md)
 126 and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_plugin_manager.md).
 127 Upon invocation, each plugin will be passed the path to the current package
 128 under test and a dictionary containing its targeted configuration, as assembled
 129 from the command line, per-package configuration, and global configuration.
 130
 131 Note: CI plugins are considered unique from build plugins and helper plugins,
 132 even though some CI plugins may execute steps of a build.
 133
 134 In the example, these plugins live alongside the code under test (in the
 135 `.pytool/Plugin` directory), but may be moved to the 'edk2-test' repo if that
 136 location makes more sense for the community.
 137
 138 ### Module Inclusion Test - DscCompleteCheck
 139
 140 This test scans all available modules (via INF files) and compares them to the
 141 package-level DSC file for the package each module is contained within. The test
 142 considers it an error if any module does not appear in the `Components` section
 143 of at least one package-level DSC (indicating that it would not be built if the
 144 package were built).
 145
 146 ### Code Compilation Test - CompilerPlugin
 147
 148 Once the Module Inclusion Test has verified that all modules would be built if
 149 all package-level DSCs were built, the Code Compilation Test simply runs through
 150 and builds every package-level DSC on every toolchain and for every architecture
 151 that is supported. Any module that fails to build is considered an error.
 152
 153 ### GUID Uniqueness Test - GuidCheck
 154
 155 This test works on the collection of all packages rather than an individual
 156 package. It looks at all FILE_GUIDs and GUIDs declared in DEC files and ensures
 157 that they are unique for the codebase. This prevents, for example, accidental
 158 duplication of GUIDs when using an existing INF as a template for a new module.
 159
 160 ### Cross-Package Dependency Test - DependencyCheck
 161
 162 This test compares the list of all packages used in INFs files for a given
 163 package against a list of "allowed dependencies" in plugin configuration for
 164 that package. Any module that depends on a disallowed package will cause a test
 165 failure.
 166
 167 ### Library Declaration Test - LibraryClassCheck
 168
 169 This test scans at all library header files found in the `Library` folders in
 170 all of the package's declared include directories and ensures that all files
 171 have a matching LibraryClass declaration in the DEC file for the package. Any
 172 missing declarations will cause a failure.
 173
 174 ### Invalid Character Test - CharEncodingCheck
 175
 176 This test scans all files in a package to make sure that there are no invalid
 177 Unicode characters that may cause build errors in some character
 178 sets/localizations.
 179
 180 ### Spell Checking - cspell
 181
 182 This test runs a spell checker on all files within the package.  This is done
 183 using the NodeJs cspell tool.  For details check `.pytool/Plugin/SpellCheck`.
 184 For this plugin to run during ci you must install nodejs and cspell and have
 185 both available to the command line when running your CI.
 186
 187 Install
 188
 189 * Install nodejs from https://nodejs.org/en/
 190 * Install cspell
 191   1. Open cmd prompt with access to node and npm
 192   2. Run `npm install -g cspell`
 193
 194   More cspell info: https://github.com/streetsidesoftware/cspell
 195
 196 ## PyTool Scopes
 197
 198 Scopes are how the PyTool ext_dep, path_env, and plugins are activated.  Meaning
 199 that if an invocable process has a scope active then those ext_dep and path_env
 200 will be active. To allow easy integration of PyTools capabilities there are a
 201 few standard scopes.
 202
 203 | Scope      | Invocable                                            | Description    |
 204 | :----      | :-----                                               | :----          |
 205 | global     | edk2_invocable++ - should be base_abstract_invocable | Running an invocables |
 206 | global-win | edk2_invocable++                                     | Running on Microsoft Windows |
 207 | global-nix | edk2_invocable++                                     | Running on Linux based OS    |
 208 | edk2-build |                                                      | This indicates that an invocable is building EDK2 based UEFI code |
 209 | cibuild    | set in .pytool/CISettings.py                         | Suggested target for edk2 continuous integration builds.  Tools used for CiBuilds can use this scope.  Example: asl compiler |
 210
 211 ## Future investments
 212
 213 * PatchCheck tests as plugins
 214 * MacOS/xcode support
 215 * Clang/LLVM support
 216 * Visual Studio AARCH64 and ARM support
 217 * BaseTools C tools CI/PR and binary release process
 218 * BaseTools Python tools CI/PR process
 219 * Extensible private/closed source platform reporting
 220 * UEFI SCTs
 221 * Other automation