UefiCpuPkg: Move AsmRelocateApLoopStart from Mpfuncs.nasm to AmdSev.nasm

[mirror_edk2.git] / .pytool / Readme.md

diff --git a/.pytool/Readme.md b/.pytool/Readme.md
index c7dce3b64ca031c5adeac8482258b57a1908725d..b2688d168753ad641d9e3fa4dbf5ce066a5e8b3e 100644 (file)
--- a/.pytool/Readme.md
+++ b/.pytool/Readme.md
@@ -1,32 +1,40 @@
 # Edk2 Continuous Integration\r
 \r
 # Edk2 Continuous Integration\r
 \r

+This file focuses on information for those working with the `.pytools` directory\r
+directly or interested in lower-level details about how CI works.\r
+\r
+If you just want to get started building code, visit\r
+[Build Instructions](https://github.com/tianocore/tianocore.github.io/wiki/Build-Instruction)\r
+on the TianoCore wiki.\r
+\r

 ## Basic Status\r
 \r
 ## Basic Status\r
 \r

-| Package             | Windows VS2019 (IA32/X64)| Ubuntu GCC (IA32/X64/ARM/AARCH64) | Known Issues |\r
-| :----               | :-----                   | :----                             | :---         |\r
-| ArmPkg              |\r
-| ArmPlatformPkg      |\r
-| ArmVirtPkg          | SEE PACKAGE README | SEE PACKAGE README |\r
-| CryptoPkg           | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
-| DynamicTablesPkg    |\r
-| EmbeddedPkg         |\r
-| EmulatorPkg         | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode\r
-| FatPkg              | :heavy_check_mark: | :heavy_check_mark: |\r
-| FmpDevicePkg        | :heavy_check_mark: | :heavy_check_mark: |\r
-| IntelFsp2Pkg        |\r
-| IntelFsp2WrapperPkg |\r
-| MdeModulePkg        | :heavy_check_mark: | :heavy_check_mark: | DxeIpl dependency on ArmPkg, Depends on StandaloneMmPkg, Spell checking in audit mode\r
-| MdePkg              | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
-| NetworkPkg          | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
-| OvmfPkg             | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode\r
-| PcAtChipsetPkg      | :heavy_check_mark: | :heavy_check_mark: |\r
-| SecurityPkg         | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
-| ShellPkg            | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 3 modules are not being built by DSC\r
-| SignedCapsulePkg    |\r
-| SourceLevelDebugPkg |\r
-| StandaloneMmPkg     |\r
-| UefiCpuPkg          | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 2 binary modules not being built by DSC\r
-| UefiPayloadPkg      |\r
+| Package              | Windows VS2019 (IA32/X64)| Ubuntu GCC (IA32/X64/ARM/AARCH64) | Known Issues |\r
+| :----                | :-----                   | :----                             | :---         |\r
+| ArmPkg               |                    | :heavy_check_mark: |\r
+| ArmPlatformPkg       |                    | :heavy_check_mark: |\r
+| ArmVirtPkg           | SEE PACKAGE README | SEE PACKAGE README |\r
+| CryptoPkg            | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
+| DynamicTablesPkg     |                    | :heavy_check_mark: |\r
+| EmbeddedPkg          |\r
+| EmulatorPkg          | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode\r
+| FatPkg               | :heavy_check_mark: | :heavy_check_mark: |\r
+| FmpDevicePkg         | :heavy_check_mark: | :heavy_check_mark: |\r
+| IntelFsp2Pkg         |\r
+| IntelFsp2WrapperPkg  |\r
+| MdeModulePkg         | :heavy_check_mark: | :heavy_check_mark: | DxeIpl dependency on ArmPkg, Depends on StandaloneMmPkg, Spell checking in audit mode\r
+| MdePkg               | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
+| NetworkPkg           | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
+| OvmfPkg              | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode\r
+| PcAtChipsetPkg       | :heavy_check_mark: | :heavy_check_mark: |\r
+| SecurityPkg          | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode\r
+| ShellPkg             | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 3 modules are not being built by DSC\r
+| SignedCapsulePkg     |\r
+| SourceLevelDebugPkg  |\r
+| StandaloneMmPkg      | :heavy_check_mark: | :heavy_check_mark: |\r
+| UefiCpuPkg           | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 2 binary modules not being built by DSC\r
+| UefiPayloadPkg       |\r
+| UnitTestFrameworkPkg | :heavy_check_mark: | :heavy_check_mark: |\r

 \r
 For more detailed status look at the test results of the latest CI run on the\r
 repo readme.\r
 \r
 For more detailed status look at the test results of the latest CI run on the\r
 repo readme.\r


@@ -81,43 +89,18 @@ easily and consistently running locally and in a cloud ci environment.  To do
 that a few steps should be followed.  Details of EDKII Tools can be found in the\r
 [docs folder here](https://github.com/tianocore/edk2-pytool-extensions/tree/master/docs)\r
 \r
 that a few steps should be followed.  Details of EDKII Tools can be found in the\r
 [docs folder here](https://github.com/tianocore/edk2-pytool-extensions/tree/master/docs)\r
 \r

-### Prerequisets\r
-\r
-1. A supported toolchain (others might work but this is what is tested and validated)\r
-   * Windows 10:\r
-     * VS 2017 or VS 2019\r
-     * Windows SDK (for rc)\r
-     * Windows WDK (for capsules)\r
-   * Ubuntu 16.04\r
-     * GCC5\r
-   * Easy to add more but this is the current state\r
-2. Python 3.7.x or newer on path\r
-3. git on path\r
-4. Recommended to setup and activate a python virtual environment\r
-5. Install the requirements `pip install --upgrade pip-requirements.txt`\r
-\r

 ### Running CI\r
 \r
 ### Running CI\r
 \r

-1. clone your edk2 repo\r
-2. Activate your python virtual environment in cmd window\r
-3. Get code dependencies (done only when submodules change)\r
-   * `stuart_setup -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`\r
-4. Update other dependencies (done more often)\r
-   * `stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`\r
-5. Run CI build (--help will give you options)\r
-   * `stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=<your tag here>`\r
-   * -p <pkg1,pkg2,pkg3> : To build only certain packages use a CSV list\r
-   * -a <arch1,arch2,arch3>: To run only certain architectures use a CSV list\r
-   * -t <target1,target2>: To run only tests related to certain targets use a\r
-     CSV list\r
-   * By default all tests are opted in.  Then given a package.ci.yaml file those\r
-     tests can be configured for a package. Finally setting the check to the\r
-     value `skip` will skip that plugin.  Examples:\r
-     * `CompilerPlugin=skip` skip the build test\r
-     * `GuidCheck=skip` skip the Guid check\r
-     * `SpellCheck=skip` skip the spell checker\r
-     * etc\r
-6. Detailed reports and logs per package are captured in the `Build` directory\r
+Quick notes:\r
+\r
+* By default all CI plugins are opted in.\r
+  * Setting the plugin to `skip` as an argument will skip running the plugin.\r
+    Examples:\r
+    * `CompilerPlugin=skip` skip the build test\r
+    * `GuidCheck=skip` skip the Guid check\r
+    * `SpellCheck=skip` skip the spell checker\r
+    * etc.\r
+* Detailed reports and logs per package are captured in the `Build` directory.\r

 \r
 ## Current PyTool Test Capabilities\r
 \r
 \r
 ## Current PyTool Test Capabilities\r
 \r


@@ -137,11 +120,31 @@ location makes more sense for the community.
 \r
 ### Module Inclusion Test - DscCompleteCheck\r
 \r
 \r
 ### Module Inclusion Test - DscCompleteCheck\r
 \r

-This test scans all available modules (via INF files) and compares them to the\r
-package-level DSC file for the package each module is contained within. The test\r
-considers it an error if any module does not appear in the `Components` section\r
-of at least one package-level DSC (indicating that it would not be built if the\r
-package were built).\r
+This scans all INF files from a package and confirms they are\r
+listed in the package level DSC file. The test considers it an error if any INF\r
+does not appear in the `Components` section of the package-level DSC (indicating\r
+that it would not be built if the package were built). This is critical because\r
+much of the CI infrastructure assumes that all modules will be listed in the DSC\r
+and compiled.\r
+\r
+This test will ignore INFs in the following cases:\r
+\r
+1. When `MODULE_TYPE` = `HOST_APPLICATION`\r
+2. When a Library instance **only** supports the `HOST_APPLICATION` environment\r
+\r
+### Host Module Inclusion Test - HostUnitTestDscCompleteCheck\r
+\r
+This test scans all INF files from a package for those related to host\r
+based unit tests and confirms they are listed in the unit test DSC file for the package.\r
+The test considers it an error if any INF meeting the requirements does not appear\r
+in the `Components` section of the unit test DSC. This is critical because\r
+much of the CI infrastructure assumes that  modules will be listed in the DSC\r
+and compiled.\r
+\r
+This test will only require INFs in the following cases:\r
+\r
+1. When `MODULE_TYPE` = `HOST_APPLICATION`\r
+2. When a Library instance explicitly supports the `HOST_APPLICATION` environment\r

 \r
 ### Code Compilation Test - CompilerPlugin\r
 \r
 \r
 ### Code Compilation Test - CompilerPlugin\r
 \r


@@ -150,6 +153,46 @@ all package-level DSCs were built, the Code Compilation Test simply runs through
 and builds every package-level DSC on every toolchain and for every architecture\r
 that is supported. Any module that fails to build is considered an error.\r
 \r
 and builds every package-level DSC on every toolchain and for every architecture\r
 that is supported. Any module that fails to build is considered an error.\r
 \r

+### Host Unit Test Compilation and Run Test - HostUnitTestCompilerPlugin\r
+\r
+A test that compiles the dsc for host based unit test apps.\r
+On Windows this will also enable a build plugin to execute that will run the unit tests and verify the results.\r
+\r
+These tools will be invoked on any CI\r
+pass that includes the NOOPT target. In order for these tools to do their job,\r
+the package and tests must be configured in a particular way...\r
+\r
+#### Including Host-Based Tests in the Package YAML\r
+\r
+For example, looking at the `MdeModulePkg.ci.yaml` config file, there are two\r
+config options that control HostBased test behavior:\r
+\r
+```json\r
+    ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin\r
+    "HostUnitTestCompilerPlugin": {\r
+        "DscPath": "Test/MdeModulePkgHostTest.dsc"\r
+    },\r
+```\r
+\r
+This option tell the test builder to run. The test builder needs to know which\r
+modules in this package are host-based tests, so that DSC path is provided.\r
+\r
+#### Configuring the HostBased DSC\r
+\r
+The HostBased DSC for `MdeModulePkg` is located at\r
+`MdeModulePkg/Test/MdeModulePkgHostTest.dsc`.\r
+\r
+To add automated host-based unit test building to a new package, create a\r
+similar DSC. The new DSC should make sure to have the `NOOPT` BUILD_TARGET\r
+and should include the line:\r
+\r
+```\r
+!include UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc\r
+```\r
+\r
+All of the modules that are included in the `Components` section of this\r
+DSC should be of type HOST_APPLICATION.\r
+\r

 ### GUID Uniqueness Test - GuidCheck\r
 \r
 This test works on the collection of all packages rather than an individual\r
 ### GUID Uniqueness Test - GuidCheck\r
 \r
 This test works on the collection of all packages rather than an individual\r


@@ -193,6 +236,20 @@ Install
 \r
   More cspell info: https://github.com/streetsidesoftware/cspell\r
 \r
 \r
   More cspell info: https://github.com/streetsidesoftware/cspell\r
 \r

+### License Checking - LicenseCheck\r
+\r
+Scans all new added files in a package to make sure code is contributed under\r
+BSD-2-Clause-Patent.\r
+\r
+### Ecc tool - EccCheck\r
+\r
+Run the Ecc tool on the package. The Ecc tool is available in the BaseTools\r
+package. It checks that the code complies to the EDKII coding standard.\r
+\r
+### Coding Standard Compliance - UncrustifyCheck\r
+\r
+Runs the Uncrustify application to check for coding standard compliance issues.\r
+\r

 ## PyTool Scopes\r
 \r
 Scopes are how the PyTool ext_dep, path_env, and plugins are activated.  Meaning\r
 ## PyTool Scopes\r
 \r
 Scopes are how the PyTool ext_dep, path_env, and plugins are activated.  Meaning\r


@@ -207,6 +264,8 @@ few standard scopes.
 | global-nix | edk2_invocable++                                     | Running on Linux based OS    |\r
 | edk2-build |                                                      | This indicates that an invocable is building EDK2 based UEFI code |\r
 | cibuild    | set in .pytool/CISettings.py                         | Suggested target for edk2 continuous integration builds.  Tools used for CiBuilds can use this scope.  Example: asl compiler |\r
 | global-nix | edk2_invocable++                                     | Running on Linux based OS    |\r
 | edk2-build |                                                      | This indicates that an invocable is building EDK2 based UEFI code |\r
 | cibuild    | set in .pytool/CISettings.py                         | Suggested target for edk2 continuous integration builds.  Tools used for CiBuilds can use this scope.  Example: asl compiler |\r

+| host-based-test | set in .pytool/CISettings.py                    | Turns on the host based tests and plugin |\r
+| host-test-win | set in .pytool/CISettings.py                      | Enables the host based test runner for Windows |\r

 \r
 ## Future investments\r
 \r
 \r
 ## Future investments\r
 \r