]> git.proxmox.com Git - mirror_edk2.git/commit
BaseTools/Tests: Verify unsupported UTF-8 data is rejected
authorJordan Justen <jordan.l.justen@intel.com>
Tue, 23 Jun 2015 23:34:38 +0000 (23:34 +0000)
committerjljusten <jljusten@Edk2>
Tue, 23 Jun 2015 23:34:38 +0000 (23:34 +0000)
commit8fb5a0cae32261d8650c9d6bb645c10872b8426f
tree463c605a876a131a3d93c8a926c797014894dddd
parent15c3a04cd0fafd598c4c413d7d32d9827ee7b794
BaseTools/Tests: Verify unsupported UTF-8 data is rejected

Surrogate pair characters can be encoded in UTF-8 files, but they are
not valid UCS-2 characters.

For example, this python interpreter code:
>>> import codecs
>>> codecs.encode(u'\ud801', 'utf-8')
'\xed\xa0\x81'

But, the range of 0xd800 - 0xdfff should be rejected as unicode code
points because they are reserved for the surrogate pair usage in
UTF-16 files.

We test that this case is rejected for UTF-8 with and without the
UTF-8 BOM.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yingke Liu <yingke.d.liu@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17698 6f19259b-4bc3-4df7-8a09-765794883524
BaseTools/Tests/CheckUnicodeSourceFiles.py