# Edk2 Continuous Integration\r
\r
+This file focuses on information for those working with the `.pytools` directory\r
+directly or interested in lower-level details about how CI works.\r
+\r
+If you just want to get started building code, visit\r
+[Build Instructions](https://github.com/tianocore/tianocore.github.io/wiki/Build-Instruction)\r
+on the TianoCore wiki.\r
+\r
## Basic Status\r
\r
-| Package | Windows VS2019 (IA32/X64)| Ubuntu GCC (IA32/X64/ARM/AARCH64) | Known Issues |\r
-| :---- | :----- | :---- | :--- |\r
-| ArmPkg |\r
-| ArmPlatformPkg |\r
-| ArmVirtPkg | SEE PACKAGE README | SEE PACKAGE README |\r
-| CryptoPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
-| DynamicTablesPkg |\r
-| EmbeddedPkg |\r
-| EmulatorPkg | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode\r
-| FatPkg | :heavy_check_mark: | :heavy_check_mark: |\r
-| FmpDevicePkg | :heavy_check_mark: | :heavy_check_mark: |\r
-| IntelFsp2Pkg |\r
-| IntelFsp2WrapperPkg |\r
-| MdeModulePkg | :heavy_check_mark: | :heavy_check_mark: | DxeIpl dependency on ArmPkg, Depends on StandaloneMmPkg, Spell checking in audit mode\r
-| MdePkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
-| NetworkPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
-| OvmfPkg | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode\r
-| PcAtChipsetPkg | :heavy_check_mark: | :heavy_check_mark: |\r
-| SecurityPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
-| ShellPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 3 modules are not being built by DSC\r
-| SignedCapsulePkg |\r
-| SourceLevelDebugPkg |\r
-| StandaloneMmPkg |\r
-| UefiCpuPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 2 binary modules not being built by DSC\r
-| UefiPayloadPkg |\r
+| Package | Windows VS2019 (IA32/X64)| Ubuntu GCC (IA32/X64/ARM/AARCH64) | Known Issues |\r
+| :---- | :----- | :---- | :--- |\r
+| ArmPkg | | :heavy_check_mark: |\r
+| ArmPlatformPkg | | :heavy_check_mark: |\r
+| ArmVirtPkg | SEE PACKAGE README | SEE PACKAGE README |\r
+| CryptoPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
+| DynamicTablesPkg | | :heavy_check_mark: |\r
+| EmbeddedPkg |\r
+| EmulatorPkg | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode\r
+| FatPkg | :heavy_check_mark: | :heavy_check_mark: |\r
+| FmpDevicePkg | :heavy_check_mark: | :heavy_check_mark: |\r
+| IntelFsp2Pkg |\r
+| IntelFsp2WrapperPkg |\r
+| MdeModulePkg | :heavy_check_mark: | :heavy_check_mark: | DxeIpl dependency on ArmPkg, Depends on StandaloneMmPkg, Spell checking in audit mode\r
+| MdePkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
+| NetworkPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
+| OvmfPkg | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode\r
+| PcAtChipsetPkg | :heavy_check_mark: | :heavy_check_mark: |\r
+| SecurityPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
+| ShellPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 3 modules are not being built by DSC\r
+| SignedCapsulePkg |\r
+| SourceLevelDebugPkg |\r
+| StandaloneMmPkg | :heavy_check_mark: | :heavy_check_mark: |\r
+| UefiCpuPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 2 binary modules not being built by DSC\r
+| UefiPayloadPkg |\r
+| UnitTestFrameworkPkg | :heavy_check_mark: | :heavy_check_mark: |\r
\r
For more detailed status look at the test results of the latest CI run on the\r
repo readme.\r
that a few steps should be followed. Details of EDKII Tools can be found in the\r
[docs folder here](https://github.com/tianocore/edk2-pytool-extensions/tree/master/docs)\r
\r
-### Prerequisets\r
-\r
-1. A supported toolchain (others might work but this is what is tested and validated)\r
- * Windows 10:\r
- * VS 2017 or VS 2019\r
- * Windows SDK (for rc)\r
- * Windows WDK (for capsules)\r
- * Ubuntu 16.04\r
- * GCC5\r
- * Easy to add more but this is the current state\r
-2. Python 3.7.x or newer on path\r
-3. git on path\r
-4. Recommended to setup and activate a python virtual environment\r
-5. Install the requirements `pip install --upgrade pip-requirements.txt`\r
-\r
### Running CI\r
\r
-1. clone your edk2 repo\r
-2. Activate your python virtual environment in cmd window\r
-3. Get code dependencies (done only when submodules change)\r
- * `stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`\r
-4. Update other dependencies (done more often)\r
- * `stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`\r
-5. Run CI build (--help will give you options)\r
- * `stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`\r
- * -p <pkg1,pkg2,pkg3> : To build only certain packages use a CSV list\r
- * -a <arch1,arch2,arch3>: To run only certain architectures use a CSV list\r
- * -t <target1,target2>: To run only tests related to certain targets use a\r
- CSV list\r
- * By default all tests are opted in. Then given a package.ci.yaml file those\r
- tests can be configured for a package. Finally setting the check to the\r
- value `skip` will skip that plugin. Examples:\r
- * `CompilerPlugin=skip` skip the build test\r
- * `GuidCheck=skip` skip the Guid check\r
- * `SpellCheck=skip` skip the spell checker\r
- * etc\r
-6. Detailed reports and logs per package are captured in the `Build` directory\r
+Quick notes:\r
+\r
+* By default all CI plugins are opted in.\r
+ * Setting the plugin to `skip` as an argument will skip running the plugin.\r
+ Examples:\r
+ * `CompilerPlugin=skip` skip the build test\r
+ * `GuidCheck=skip` skip the Guid check\r
+ * `SpellCheck=skip` skip the spell checker\r
+ * etc.\r
+* Detailed reports and logs per package are captured in the `Build` directory.\r
\r
## Current PyTool Test Capabilities\r
\r
\r
### Module Inclusion Test - DscCompleteCheck\r
\r
-This test scans all available modules (via INF files) and compares them to the\r
-package-level DSC file for the package each module is contained within. The test\r
-considers it an error if any module does not appear in the `Components` section\r
-of at least one package-level DSC (indicating that it would not be built if the\r
-package were built).\r
+This scans all INF files from a package and confirms they are\r
+listed in the package level DSC file. The test considers it an error if any INF\r
+does not appear in the `Components` section of the package-level DSC (indicating\r
+that it would not be built if the package were built). This is critical because\r
+much of the CI infrastructure assumes that all modules will be listed in the DSC\r
+and compiled.\r
+\r
+This test will ignore INFs in the following cases:\r
+\r
+1. When `MODULE_TYPE` = `HOST_APPLICATION`\r
+2. When a Library instance **only** supports the `HOST_APPLICATION` environment\r
+\r
+### Host Module Inclusion Test - HostUnitTestDscCompleteCheck\r
+\r
+This test scans all INF files from a package for those related to host\r
+based unit tests and confirms they are listed in the unit test DSC file for the package.\r
+The test considers it an error if any INF meeting the requirements does not appear\r
+in the `Components` section of the unit test DSC. This is critical because\r
+much of the CI infrastructure assumes that modules will be listed in the DSC\r
+and compiled.\r
+\r
+This test will only require INFs in the following cases:\r
+\r
+1. When `MODULE_TYPE` = `HOST_APPLICATION`\r
+2. When a Library instance explicitly supports the `HOST_APPLICATION` environment\r
\r
### Code Compilation Test - CompilerPlugin\r
\r
and builds every package-level DSC on every toolchain and for every architecture\r
that is supported. Any module that fails to build is considered an error.\r
\r
+### Host Unit Test Compilation and Run Test - HostUnitTestCompilerPlugin\r
+\r
+A test that compiles the dsc for host based unit test apps.\r
+On Windows this will also enable a build plugin to execute that will run the unit tests and verify the results.\r
+\r
+These tools will be invoked on any CI\r
+pass that includes the NOOPT target. In order for these tools to do their job,\r
+the package and tests must be configured in a particular way...\r
+\r
+#### Including Host-Based Tests in the Package YAML\r
+\r
+For example, looking at the `MdeModulePkg.ci.yaml` config file, there are two\r
+config options that control HostBased test behavior:\r
+\r
+```json\r
+ ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin\r
+ "HostUnitTestCompilerPlugin": {\r
+ "DscPath": "Test/MdeModulePkgHostTest.dsc"\r
+ },\r
+```\r
+\r
+This option tell the test builder to run. The test builder needs to know which\r
+modules in this package are host-based tests, so that DSC path is provided.\r
+\r
+#### Configuring the HostBased DSC\r
+\r
+The HostBased DSC for `MdeModulePkg` is located at\r
+`MdeModulePkg/Test/MdeModulePkgHostTest.dsc`.\r
+\r
+To add automated host-based unit test building to a new package, create a\r
+similar DSC. The new DSC should make sure to have the `NOOPT` BUILD_TARGET\r
+and should include the line:\r
+\r
+```\r
+!include UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc\r
+```\r
+\r
+All of the modules that are included in the `Components` section of this\r
+DSC should be of type HOST_APPLICATION.\r
+\r
### GUID Uniqueness Test - GuidCheck\r
\r
This test works on the collection of all packages rather than an individual\r
\r
More cspell info: https://github.com/streetsidesoftware/cspell\r
\r
+### License Checking - LicenseCheck\r
+\r
+Scans all new added files in a package to make sure code is contributed under\r
+BSD-2-Clause-Patent.\r
+\r
+### Ecc tool - EccCheck\r
+\r
+Run the Ecc tool on the package. The Ecc tool is available in the BaseTools\r
+package. It checks that the code complies to the EDKII coding standard.\r
+\r
+### Coding Standard Compliance - UncrustifyCheck\r
+\r
+Runs the Uncrustify application to check for coding standard compliance issues.\r
+\r
## PyTool Scopes\r
\r
Scopes are how the PyTool ext_dep, path_env, and plugins are activated. Meaning\r
| global-nix | edk2_invocable++ | Running on Linux based OS |\r
| edk2-build | | This indicates that an invocable is building EDK2 based UEFI code |\r
| cibuild | set in .pytool/CISettings.py | Suggested target for edk2 continuous integration builds. Tools used for CiBuilds can use this scope. Example: asl compiler |\r
+| host-based-test | set in .pytool/CISettings.py | Turns on the host based tests and plugin |\r
+| host-test-win | set in .pytool/CISettings.py | Enables the host based test runner for Windows |\r
\r
## Future investments\r
\r
projects / mirror_edk2.git / blobdiff