UefiCpuPkg/MpInitLib: Decrease NumApsExecuting only for ApInitConfig

The patch fixes the bug that the memory under 1MB is modified by
firmware in S3 boot.

Root cause is a racing condition in MpInitLib:
1. BSP: WakeUpByInitSipiSipi is set by NotifyOnS3SmmInitDonePpi()
2. BSP: WakeUpAP() wakes all APs to run certain procedure.
  2.1. AllocateResetVector() uses <1MB memory for wake up vector.
  2.1. FillExchangeInfoData() resets NumApsExecuting to 0.
  2.2. WaitApWakeup() waits AP to clear WAKEUP_AP_SIGNAL.
3. AP: ApWakeupFunction() clears WAKEUP_AP_SIGNAL to inform BSP.
5. BSP: FreeResetVector() restores the <1MB memory
4. AP: ApWakeupFunction() calls the certain procedure.
  4.1. NumApsExecuting is decreased.

#4.1 happens after the 1MB memory is restored so the result is
memory below 1MB is changed by #4.1
It happens only when the AP executes procedure a bit longer.
AP returns back to ApWakeupFunction() from procedure after
BSP restores the <1MB memory.

Since NumApsExecuting is only used when InitFlag == ApInitConfig
for counting the processor count.
The patch moves the NumApsExecuting decrease to the path when
InitFlag == ApInitConfig.

Signed-off-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Eric Dong <eric.dong@intel.com>
Cc: Nandagopal Sathyanarayanan <nandagopal.sathyanarayanan@intel.com>

diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c b/UefiCpuPkg/Library/MpInitLib/MpLib.c
index 33374888922a85d80fbe4f4735f1f24446b03926..6f51bc4ebfb90b12231318052fb83b9a63bb5211 100644 (file)
--- a/UefiCpuPkg/Library/MpInitLib/MpLib.c
+++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c
@@ -1,7 +1,7 @@
 /** @file\r
   CPU MP Initialize Library common functions.\r
 \r
 /** @file\r
   CPU MP Initialize Library common functions.\r
 \r

-  Copyright (c) 2016 - 2018, Intel Corporation. All rights reserved.<BR>\r
+  Copyright (c) 2016 - 2019, Intel Corporation. All rights reserved.<BR>\r

   SPDX-License-Identifier: BSD-2-Clause-Patent\r
 \r
 **/\r
   SPDX-License-Identifier: BSD-2-Clause-Patent\r
 \r
 **/\r


@@ -619,6 +619,8 @@ ApWakeupFunction (
       RestoreVolatileRegisters (&CpuMpData->CpuData[0].VolatileRegisters, FALSE);\r
       InitializeApData (CpuMpData, ProcessorNumber, BistData, ApTopOfStack);\r
       ApStartupSignalBuffer = CpuMpData->CpuData[ProcessorNumber].StartupApSignal;\r
       RestoreVolatileRegisters (&CpuMpData->CpuData[0].VolatileRegisters, FALSE);\r
       InitializeApData (CpuMpData, ProcessorNumber, BistData, ApTopOfStack);\r
       ApStartupSignalBuffer = CpuMpData->CpuData[ProcessorNumber].StartupApSignal;\r

+\r
+      InterlockedDecrement ((UINT32 *) &CpuMpData->MpCpuExchangeInfo->NumApsExecuting);\r

     } else {\r
       //\r
       // Execute AP function if AP is ready\r
     } else {\r
       //\r
       // Execute AP function if AP is ready\r


@@ -698,7 +700,6 @@ ApWakeupFunction (
     // AP finished executing C code\r
     //\r
     InterlockedIncrement ((UINT32 *) &CpuMpData->FinishedCount);\r
     // AP finished executing C code\r
     //\r
     InterlockedIncrement ((UINT32 *) &CpuMpData->FinishedCount);\r

-    InterlockedDecrement ((UINT32 *) &CpuMpData->MpCpuExchangeInfo->NumApsExecuting);\r

 \r
     //\r
     // Place AP is specified loop mode\r
 \r
     //\r
     // Place AP is specified loop mode\r