]> git.proxmox.com Git - mirror_edk2.git/commit - BaseTools/Tests/CheckUnicodeSourceFiles.py
BaseTools/Tests: Verify unsupported UTF-16 are rejected
authorJordan Justen <jordan.l.justen@intel.com>
Tue, 23 Jun 2015 23:34:22 +0000 (23:34 +0000)
committerjljusten <jljusten@Edk2>
Tue, 23 Jun 2015 23:34:22 +0000 (23:34 +0000)
commitdadfab5b23c3f02ab8396a5487122de797188845
treef1c11df3a8df1e8da144a1071e054ffd44da8ff9
parentd80e451b187c9d33cbd771253fbd5119670f75c6
BaseTools/Tests: Verify unsupported UTF-16 are rejected

Supplementary Plane characters can exist in UTF-16 files,
but they are not valid UCS-2 characters.

For example, this python interpreter code:
>>> import codecs
>>> codecs.encode(u'\U00010300', 'utf-16')
'\xff\xfe\x00\xd8\x00\xdf'

Therefore the UCS-4 0x00010300 character is encoded as two
16-bit numbers (0xd800 0xdf00) in a little endian UTF-16
file.

For more information, see:
http://en.wikipedia.org/wiki/UTF-16#U.2B10000_to_U.2B10FFFF

This test checks to make sure that BaseTools will reject these
characters in UTF-16 files.

The range of 0xd800 - 0xdfff should also be rejected as unicode code
points because they are reserved for the surrogate pair usage in
UTF-16 files.

This test was fixed by the previous commit:
"BaseTools/UniClassObject: Verify valid UCS-2 chars in UTF-16 .uni files"

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yingke Liu <yingke.d.liu@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17695 6f19259b-4bc3-4df7-8a09-765794883524
BaseTools/Tests/CheckUnicodeSourceFiles.py