Star Zeng [Thu, 2 Jul 2015 00:50:42 +0000 (00:50 +0000)]
MdeModulePkg Variable: Minor enhancement to VarCheck
1. When Property->Attributes == 0, don't check input Attributes.
2. Enhance comments to be indent aligned and debug message
to be aligned with other messages.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Star Zeng <star.zeng@intel.com> Reviewed-by: Jiewen Yao <jiewen.yao@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17794 6f19259b-4bc3-4df7-8a09-765794883524
Star Zeng [Wed, 1 Jul 2015 03:13:02 +0000 (03:13 +0000)]
SecurityPkg: Delete Auth Variable driver
1. Delete TpmMeasurementLib LibraryClass from SecurityPkg after it moved to MdeModulePkg.
2. Update DxeTpmMeasurementLib.inf to include MdeModulePkg.dec.
3. Delete authenticated variable definition from AuthenticatedVariableFormat.h after
them moved to VariableFormat.h.
4. Replace VARIABLE_HEADER with AUTHENTICATED_VARIABLE_HEADER in EsalVariableDxeSal.
5. Delete VariableInfo from SecurityPkg after it merged to VariableInfo in MdeModulePkg.
6. Delete VariablePei from SecurityPkg after it merged to VariablePei in MdeModulePkg.
7. Delete Auth Variable driver from SecurityPkg after it merged to Variable driver in
MdeModulePkg.
8. Also update PACKAGE_GUID and PACKAGE_VERSION in SecurityPkg.dec after the deletion
of authenticated variable definition, VariableInfo, VariablePei and Auth Variable
driver from SecurityPkg; update PLATFORM_VERSION in SecurityPkg.dsc.
Star Zeng [Wed, 1 Jul 2015 03:10:27 +0000 (03:10 +0000)]
ArmVirtPkg: Cleanup redundant library class resolution of Variable modules
SecurityPkg/VariableAuthenticated/Pei/VariablePei.inf doesn't consume
BaseCryptLib directly or indirectly, cleanup it.
SecurityPkg/VariableAuthenticated/Pei/VariablePei.inf {
<LibraryClasses>
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/PeiCryptLib.inf
}
BaseCryptLib has been declared at [LibraryClasses.common.DXE_RUNTIME_DRIVER]
section in ArmVirt.dsc.inc.
OpensslLib has been declared at [LibraryClasses.common]
section in ArmVirt.dsc.inc.
So cleanup them.
SecurityPkg/VariableAuthenticated/RuntimeDxe/VariableRuntimeDxe.inf {
<LibraryClasses>
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/RuntimeCryptLib.inf
OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
}
Star Zeng [Wed, 1 Jul 2015 03:10:01 +0000 (03:10 +0000)]
ArmPlatformPkg: Use the merged Variable driver
Remove gVariableAuthenticatedRuntimeDxeFileGuid definition in
ArmPlatformPkg.dec and use gVariableRuntimeDxeFileGuid in
NorFlashAuthenticatedDxe.inf as auth Variable driver in SecurityPkg
and Variable driver in MdeModulePkg have been merged.
Star Zeng [Wed, 1 Jul 2015 03:08:29 +0000 (03:08 +0000)]
MdeModulePkg Variable: Merge from Auth Variable driver in SecurityPkg
What to do:
1. Merge from Auth Variable driver in SecurityPkg to Variable drive in
MdeModulePkg. Then the merged Variable driver in MdeModulePkg will
link to AuthVariableLib and TpmMeasurementLib.
AuthVariableLibNull and TpmMeasurementLibNull in MdeModulePkg
could be used for non auth variable support.
AuthVariableLib and DxeTpmMeasurementLib in SecurityPkg
may be used for auth variable support.
Why to do:
1. Remove code duplication and reduce maintenance effort.
After auth variable service separated from Auth Variable driver in SecurityPkg
to AuthVariableLib. The remaining code logic of Auth Variable driver in SecurityPkg
will be almost same with Variable driver in MdeModulePkg. Now it is to
merge them.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Star Zeng <star.zeng@intel.com> Reviewed-by: Jiewen Yao <jiewen.yao@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17765 6f19259b-4bc3-4df7-8a09-765794883524
Star Zeng [Wed, 1 Jul 2015 03:07:27 +0000 (03:07 +0000)]
Vlv2TbltDevicePkg: Link AuthVariableLib for following merged variable driver deploy
AuthVariableLib and TpmMeasurementLib library classes are now linked with
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRuntimeDxe.inf
to optionally support secure variables.
For Vlv2TbltDevicePkg,
link AuthVariableLib and DxeTpmMeasurementLib in SecurityPkg
when SECURE_BOOT_ENABLE = TRUE,
and link AuthVariableLibNull and TpmMeasurementLibNull in MdeModulePkg
when SECURE_BOOT_ENABLE = FALSE.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Star Zeng <star.zeng@intel.com> Reviewed-by: David Wei <david.wei@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17764 6f19259b-4bc3-4df7-8a09-765794883524
Star Zeng [Wed, 1 Jul 2015 03:07:02 +0000 (03:07 +0000)]
ArmVirtPkg: Link AuthVariableLib for following merged variable driver deploy
AuthVariableLib and TpmMeasurementLib library classes are now linked with
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRuntimeDxe.inf
to optionally support secure variables.
For ArmVirtPkg,
link AuthVariableLib and DxeTpmMeasurementLib in SecurityPkg
when SECURE_BOOT_ENABLE = TRUE,
and link AuthVariableLibNull and TpmMeasurementLibNull in MdeModulePkg
when SECURE_BOOT_ENABLE = FALSE.
Star Zeng [Wed, 1 Jul 2015 03:05:47 +0000 (03:05 +0000)]
OvmfPkg: Link AuthVariableLib for following merged variable driver deploy
AuthVariableLib and TpmMeasurementLib library classes are now linked with
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRuntimeDxe.inf
to optionally support secure variables.
For OvmfPkg,
link AuthVariableLib and DxeTpmMeasurementLib in SecurityPkg
when SECURE_BOOT_ENABLE = TRUE,
and link AuthVariableLibNull and TpmMeasurementLibNull in MdeModulePkg
when SECURE_BOOT_ENABLE = FALSE.
Star Zeng [Wed, 1 Jul 2015 03:05:22 +0000 (03:05 +0000)]
Nt32Pkg: Link AuthVariableLib for following merged variable driver deploy
AuthVariableLib and TpmMeasurementLib library classes are now linked with
MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRuntimeDxe.inf
to optionally support secure variables.
For Nt32Pkg,
link AuthVariableLib and DxeTpmMeasurementLib in SecurityPkg
when SECURE_BOOT_ENABLE = TRUE,
and link AuthVariableLibNull and TpmMeasurementLibNull in MdeModulePkg
when SECURE_BOOT_ENABLE = FALSE.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Star Zeng <star.zeng@intel.com> Reviewed-by: Ruiyu Ni <ruiyu.ni@intel.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17759 6f19259b-4bc3-4df7-8a09-765794883524
What to do:
1. Implement AuthVariableLib library instance.
2. Temporarily add VARIABLE_ENTRY_CONSISTENCY and
variable attribute combinations definitions to
AuthenticatedVariableFormat.h for git bisect.
Why to do:
1. Share code.
Separate auth variable service from Auth Variable driver in
SecurityPkg to AuthVariableLib. Then the AuthVariableLib could benefit
and be used by different implementation of Auth Variable drivers.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Star Zeng <star.zeng@intel.com> Reviewed-by: Jiewen Yao <jiewen.yao@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17758 6f19259b-4bc3-4df7-8a09-765794883524
Star Zeng [Wed, 1 Jul 2015 03:04:18 +0000 (03:04 +0000)]
MdeModulePkg: Add AuthVariableLib LibraryClass
What to do:
1. Add AuthVariableLib LibraryClass definitions.
2. Implement a NULL AuthVariableLib library instance.
3. Add VARIABLE_ENTRY_PROPERTY definition to VarCheck.h.
4. Add VARIABLE_ENTRY_CONSISTENCY and variable attribute
combinations definitions to VariableFormat.h.
Why to do:
1. Share code.
1.1. Separate auth variable service from Auth Variable driver in
SecurityPkg to AuthVariableLib. Then the AuthVariableLib could benefit
and be used by different implementation of Auth Variable drivers.
1.2 The VARIABLE_ENTRY_PROPERTY definition to VarCheck.h,
VARIABLE_ENTRY_CONSISTENCY and variable attribute
combinations definitions to VariableFormat.h will be shared
by merged Variable driver and AuthVariableLib.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Star Zeng <star.zeng@intel.com> Reviewed-by: Jiewen Yao <jiewen.yao@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17757 6f19259b-4bc3-4df7-8a09-765794883524
Star Zeng [Wed, 1 Jul 2015 03:03:26 +0000 (03:03 +0000)]
MdeModulePkg VariablePei: Merge from VariablePei in SecurityPkg
What to do:
1. Merge from VariablePei in SecurityPkg to VariablePei in MdeModulePkg.
Why to do:
1. Remove code duplication and reduce maintenance effort.
The code logic of VariablePei in SecurityPkg is same with VariablePei
in MdeModulePkg.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Star Zeng <star.zeng@intel.com> Reviewed-by: Jiewen Yao <jiewen.yao@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17756 6f19259b-4bc3-4df7-8a09-765794883524
Star Zeng [Wed, 1 Jul 2015 03:02:45 +0000 (03:02 +0000)]
MdeModulePkg VariableInfo: Merge from VariableInfo in SecurityPkg
What to do:
1. Merge VariableInfo in SecurityPkg to VariableInfo in MdeModulePkg.
Why to do:
1. Remove code duplication and reduce maintenance effort.
The functionality of VariableInfo in SecurityPkg has covered VariableInfo
in MdeModulePkg.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Star Zeng <star.zeng@intel.com> Reviewed-by: Jiewen Yao <jiewen.yao@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17755 6f19259b-4bc3-4df7-8a09-765794883524
Star Zeng [Wed, 1 Jul 2015 03:02:08 +0000 (03:02 +0000)]
MdeModulePkg: Move authenticated variable definition from SecurityPkg
What to do:
1. Move authenticated variable definition from AuthenticatedVariableFormat.h
to VariableFormat.h with gEfiAuthenticatedVariableGuid and
AUTHENTICATED_VARIABLE_HEADER.
Why to do:
1. Share code.
We are moving forward to separate auth variable service from Auth Variable
driver in SecurityPkg to AuthVariableLib. Then the AuthVariableLib could benefit
and be used by different implementation of Auth Variable drivers.
2. Remove code duplication and reduce maintenance effort.
After auth variable service separated from Auth Variable driver in SecurityPkg
to AuthVariableLib. The remaining code logic of Auth Variable driver in SecurityPkg
will be almost same with Variable driver in MdeModulePkg. We are going to
merge them.
3. gEfiAuthenticatedVariableGuid will be used by both merged Variable driver
and AuthVariableLib, AUTHENTICATED_VARIABLE_HEADER will be used by
merged Variable driver.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Star Zeng <star.zeng@intel.com> Reviewed-by: Jiewen Yao <jiewen.yao@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17754 6f19259b-4bc3-4df7-8a09-765794883524
Star Zeng [Wed, 1 Jul 2015 03:01:34 +0000 (03:01 +0000)]
MdeModulePkg: Move TpmMeasurementLib LibraryClass from SecurityPkg
What to do:
1. Move TpmMeasurementLib LibraryClass from SecurityPkg to MdeModulePkg.
2. Implement a NULL TpmMeasurementLib library instance.
Why to do:
1. Share code.
We are moving forward to separate auth variable service from Auth Variable
driver in SecurityPkg to AuthVariableLib. Then the AuthVariableLib could benefit
and be used by different implementation of Auth Variable drivers.
2. Remove code duplication and reduce maintenance effort.
After auth variable service separated from Auth Variable driver in SecurityPkg
to AuthVariableLib. The remaining code logic of Auth Variable driver in SecurityPkg
will be almost same with Variable driver in MdeModulePkg. We are going to
merge them.
3. TpmMeasurementLib is consumed by Auth Variable driver in SecurityPkg now,
as Auth Variable driver in SecurityPkg will be merged to Variable driver in
MdeModulePkg, so the library class also needs to be moved to MdeModulePkg.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Star Zeng <star.zeng@intel.com> Reviewed-by: Jiewen Yao <jiewen.yao@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17753 6f19259b-4bc3-4df7-8a09-765794883524
NetworkPkg: Add IPv6 DUID configuration support using platform PCD.
The code today always uses DUID-UUID unless there is an error in reading the UUID,
which results in falling through to DUID-LLT. This change allows the platform to
control the DUID method (DUID-UUID or DUID-LLT) using a platform dynamic PCD.
CHAR16 array mGaugeString[DP_GAUGE_STRING_LENGTH + 1] is pass into
function GetShortPdbFileName(). However, in this function it treats the
size of the input buffer as DXE_PERFORMANCE_STRING_SIZE.
Though DXE_PERFORMANCE_STRING_SIZE is smaller than DP_GAUGE_STRING_LENGTH
now, but this manner might introduce a potential risk of buffer overflow.
Yingke Liu [Mon, 29 Jun 2015 03:10:25 +0000 (03:10 +0000)]
BaseTools: Update IASL download link and remove specific version info.
The version of IASL compiler in the tools_def.template file no longer exists on the acpica.org site.
Update download link and remove the specific version info from the tools_def.template file.
Laszlo Ersek [Fri, 26 Jun 2015 16:09:52 +0000 (16:09 +0000)]
OvmfPkg: PlatformPei: invert MTRR setup in QemuInitializeRam()
At the moment we work with a UC default MTRR type, and set three memory
ranges to WB:
- [0, 640 KB),
- [1 MB, LowerMemorySize),
- [4 GB, 4 GB + UpperMemorySize).
Unfortunately, coverage for the third range can fail with a high
likelihood. If the alignment of the base (ie. 4 GB) and the alignment of
the size (UpperMemorySize) differ, then MtrrLib creates a series of
variable MTRR entries, with power-of-two sized MTRR masks. And, it's
really easy to run out of variable MTRR entries, dependent on the
alignment difference.
This is a problem because a Linux guest will loudly reject any high memory
that is not covered my MTRR.
So, let's follow the inverse pattern (loosely inspired by SeaBIOS):
- flip the MTRR default type to WB,
- set [0, 640 KB) to WB -- fixed MTRRs have precedence over the default
type and variable MTRRs, so we can't avoid this,
- set [640 KB, 1 MB) to UC -- implemented with fixed MTRRs,
- set [LowerMemorySize, 4 GB) to UC -- should succeed with variable MTRRs
more likely than the other scheme (due to less chaotic alignment
differences).
Effects of this patch can be observed by setting DEBUG_CACHE (0x00200000)
in PcdDebugPrintErrorLevel.
Laszlo Ersek [Fri, 26 Jun 2015 16:09:43 +0000 (16:09 +0000)]
OvmfPkg: PlatformPei: create the CPU HOB with dynamic memory space width
Maoming reported that guest memory sizes equal to or larger than 64GB
were not correctly handled by OVMF.
Enabling the DEBUG_GCD (0x00100000) bit in PcdDebugPrintErrorLevel, and
starting QEMU with 64GB guest RAM size, I found the following error in the
OVMF debug log:
This message is emitted when the DXE core is initializing the memory space
map, processing the "above 4GB" memory resource descriptor HOB that was
created by OVMF's QemuInitializeRam() function (see "UpperMemorySize").
The DXE core's call chain fails in:
CoreInternalAddMemorySpace() [MdeModulePkg/Core/Dxe/Gcd/Gcd.c]
CoreConvertSpace()
//
// Search for the list of descriptors that cover the range BaseAddress
// to BaseAddress+Length
//
CoreSearchGcdMapEntry()
CoreSearchGcdMapEntry() fails because the one entry (with type
"nonexistent") in the initial GCD memory space map is too small, and
cannot be split to cover the memory space range being added:
We'll soon increase the maximum guest-physical RAM size supported by OVMF.
For more RAM, the DXE IPL is going to build more page tables, and for that
it's going to need a bigger chunk from the permanent PEI RAM.
Otherwise CreateIdentityMappingPageTables() would fail with:
> DXE IPL Entry
> Loading PEIM at 0x000BFF61000 EntryPoint=0x000BFF61260 DxeCore.efi
> Loading DXE CORE at 0x000BFF61000 EntryPoint=0x000BFF61260
> AllocatePages failed: No 0x40201 Pages is available.
> There is only left 0x3F1F pages memory resource to be allocated.
> ASSERT .../MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c(123):
> BigPageAddress != 0
(The above example belongs to the artificially high, maximal address width
of 52, clamped by the DXE core to 48. The address width of 48 bits
corresponds to 256 TB or RAM, and requires a bit more than 1GB for paging
structures.)
Cc: Maoming <maoming.maoming@huawei.com> Cc: Huangpeng (Peter) <peter.huangpeng@huawei.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Brian J. Johnson <bjohnson@sgi.com>
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Laszlo Ersek <lersek@redhat.com> Reviewed-by: Brian J. Johnson <bjohnson@sgi.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17719 6f19259b-4bc3-4df7-8a09-765794883524
Tian Feng [Fri, 26 Jun 2015 08:42:46 +0000 (08:42 +0000)]
SecurityPkg/TcgMor: move TPer Reset operation to this module
The TPer Reset operation is a common logic. So it's added into
SecurityPkg/Tcg/MemoryOverwriteControl/TcgMor.inf module and
would be triggered at EndOfDxe.
By this way, all encrypted drives which produce EFI_STORAGE_SECURITY_
RPOTOCOL interface would be force reset when MOR is set.
Jeff Fan [Fri, 26 Jun 2015 03:06:50 +0000 (03:06 +0000)]
SourceLevelDebugPkg/DebugAgent: Add typecast to fix sign extension
OffsetHigh is 16bit value and its type is UINT32 and defined in structure.
It will be 32bit int type after do 16-bit left-shift operation. Then it will
sign extension if cast it to UINT64 if its high bit is 1.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Jeff Fan <jeff.fan@intel.com> Reviewed-by: Ruiyu Ni <ruiyu.ni@intel.com> Reviewed-by: Scott Duplichan <scott@notabs.org>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17716 6f19259b-4bc3-4df7-8a09-765794883524
Ard Biesheuvel [Thu, 25 Jun 2015 14:49:06 +0000 (14:49 +0000)]
ArmVirtPkg: signal EndOxDxe event in PlatformBsdInit
Currently, the ArmVirtPkg platforms built with the Intel BDS fail
to signal the end-of-DXE event 'gEfiEndOfDxeEventGroupGuid' when
entering the BDS phase, which results in some loss of functionality,
i.e., variable reclaim in the VariableDxe drivers, and the splitting
of the memory regions that is part of the recently added UEFI 2.5
properties table feature.
it is up to the platform BDS to signal that event, since there may be
platform specific ordering constraints with respect to the signalling
of the event that are difficult to honor at the generic level.
So add the SignalEvent () call to PlatformBdsInit () of ArmVirtPkg's
PlatformBdsLib implementation for the Intel BDS.
Liming Gao [Thu, 25 Jun 2015 03:29:38 +0000 (03:29 +0000)]
MdeModulePkg: Update PeiCore to handle specific child FV
Child FV may be placed in parent FV image without process required. Then,
Child FV and parent FV will be in the same continuous space. For FileHandle,
FileHandleToVolume() function needs to find the best matched FV handle.
Liming Gao [Thu, 25 Jun 2015 03:29:11 +0000 (03:29 +0000)]
MdeModulePkg: Add Memory Capabilities for MMIO and Reserved Range
UEFI2.5 spec, GetMemoryMap(), says:
Attribute: Attributes of the memory region that describe the bit mask
of capabilities for that memory region, and not necessarily the current
settings for that memory region.
But, GetMemoryMap() implementation doesn't append memory capabilities
for MMIO and Reserved memory range. This will break UEFI2.5 Properties
Table feature, because Properties Table need return EFI_MEMORY_RO or
EFI_MEMORY_XP capabilities for OS.
This patch appends memory capabilities for those memory range.
Qiu Shumin [Wed, 24 Jun 2015 08:09:05 +0000 (08:09 +0000)]
MdePkg\Library\UefiFileHandleLib: Make FileHandleWriteLine support both ASCII and UNICODE file.
When the file is a UNICODE file (with UNICODE file tag) write UNICODE text.
When the file is an ASCII file write ASCII text.
If the file size is zero (without the file tag at the beginning) write ASCII text as default.
Jordan Justen [Tue, 23 Jun 2015 23:34:33 +0000 (23:34 +0000)]
BaseTools/Tests: Verify 32-bit UTF-8 chars are rejected
Since UTF-8 .uni unicode files might contain strings with unicode code
points larger than 16-bits, and UEFI only supports UCS-2 characters,
we need to make sure that BaseTools rejects these characters in UTF-8
.uni source files.
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com> Reviewed-by: Yingke Liu <yingke.d.liu@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17697 6f19259b-4bc3-4df7-8a09-765794883524
Jordan Justen [Tue, 23 Jun 2015 23:34:28 +0000 (23:34 +0000)]
BaseTools/UniClassObject: Support UTF-8 string data in .uni files
This allows .uni input files to be encoded with UTF-8. Today, we only
support UTF-16 encoding.
The strings are still converted to UCS-2 data for use in EDK II
modules. (This is the only unicode character format supported by UEFI
and EDK II.)
Although UTF-8 would allow any UCS-4 character to be present in the
source file, we restrict the entire file to the UCS-2 range.
(Including comments.) This allows the files to be converted to UTF-16
if needed.
v2:
* Drop .utf8 extension. Use .uni file for UTF-8 data (mdkinney)
* Merge in 'BaseTools/UniClassObject: Verify string data is 16-bit'
commit
v3:
* Restrict the entire file's characters (including comments) to the
UCS-2 range in addition to string data. (mdkinney)
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com> Reviewed-by: Yingke Liu <yingke.d.liu@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17696 6f19259b-4bc3-4df7-8a09-765794883524
Jordan Justen [Tue, 23 Jun 2015 23:34:19 +0000 (23:34 +0000)]
BaseTools/UniClassObject: Verify valid UCS-2 chars in UTF-16 .uni files
Supplementary Plane characters can exist in UTF-16 files,
but they are not valid UCS-2 characters.
For example, refer to this python interpreter code:
>>> import codecs
>>> codecs.encode(u'\U00010300', 'utf-16')
'\xff\xfe\x00\xd8\x00\xdf'
Therefore the UCS-4 0x00010300 character is encoded as two
16-bit numbers (0xd800 0xdf00) in a little endian UTF-16
file.
For more information, see:
http://en.wikipedia.org/wiki/UTF-16#U.2B10000_to_U.2B10FFFF
This means that our current BaseTools code could be allowing
unsupported UTF-16 characters be used. To fix this, we decode the file
using python's utf-16 decode support. Then we verify that each
character's code point is 0xffff or less.
v3:
* Based on Mike Kinney's feedback, we now read the whole file and
verify up-front that it contains valid UCS-2 characters. Thanks
also to Laszlo Ersek for pointing out the Supplementary Plane
characters.
v4:
* Reject code points in 0xd800-0xdfff range since they are reserved
for UTF-16 surrogate pairs. (lersek)
Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Laszlo Ersek <lersek@redhat.com> Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com> Reviewed-by: Yingke Liu <yingke.d.liu@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17694 6f19259b-4bc3-4df7-8a09-765794883524
projects / mirror_edk2.git / log