qapi: Support (subset of) \u escapes in strings

The handling of \ inside QAPI strings was less than ideal, and
really only worked JSON's \/, \\, \", and our extension of \'
(an obvious extension, when you realize we use '' instead of ""
for strings).  For other things, like '\n', it resulted in a
literal 'n' instead of a newline.

Of course, at the moment, we really have no use for escaped
characters, as QAPI has to map to C identifiers, and we currently
support ASCII only for that.  But down the road, we may add
support for default values for string parameters to a command
or struct; if that happens, it would be nice to correctly support
all JSON escape sequences, such as \n or \uXXXX.  This gets us
closer, by supporting Unicode escapes in the ASCII range.

Since JSON does not require \OCTAL or \xXX escapes, and our QMP
implementation does not understand them either, I intentionally
reject it here, but it would be an easy addition if we desired it.
Likewise, intentionally refusing the NUL byte means we don't have
to worry about C strings being shorter than the qapi input.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 44898b082af214399cb189788e82b2ba7064697b..6a9aa24a6c0bff354c3b1f9c582ce3543c23e4d5 100644 (file)
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -173,7 +173,41 @@ class QAPISchema:
                         raise QAPISchemaError(self,
                                               'Missing terminating "\'"')
                     if esc:
-                        string += ch
+                        if ch == 'b':
+                            string += '\b'
+                        elif ch == 'f':
+                            string += '\f'
+                        elif ch == 'n':
+                            string += '\n'
+                        elif ch == 'r':
+                            string += '\r'
+                        elif ch == 't':
+                            string += '\t'
+                        elif ch == 'u':
+                            value = 0
+                            for x in range(0, 4):
+                                ch = self.src[self.cursor]
+                                self.cursor += 1
+                                if ch not in "0123456789abcdefABCDEF":
+                                    raise QAPISchemaError(self,
+                                                          '\\u escape needs 4 '
+                                                          'hex digits')
+                                value = (value << 4) + int(ch, 16)
+                            # If Python 2 and 3 didn't disagree so much on
+                            # how to handle Unicode, then we could allow
+                            # Unicode string defaults.  But most of QAPI is
+                            # ASCII-only, so we aren't losing much for now.
+                            if not value or value > 0x7f:
+                                raise QAPISchemaError(self,
+                                                      'For now, \\u escape '
+                                                      'only supports non-zero '
+                                                      'values up to \\u007f')
+                            string += chr(value)
+                        elif ch in "\\/'\"":
+                            string += ch
+                        else:
+                            raise QAPISchemaError(self,
+                                                  "Unknown escape \\%s" %ch)
                         esc = False
                     elif ch == "\\":
                         esc = True

diff --git a/tests/Makefile b/tests/Makefile
index e2a3bd349e2688ce972b12ba2f7952ab4d16419d..547a2499bedf91e80196260897ccb32a9afd84ea 100644 (file)
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -212,6 +212,8 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
        enum-clash-member.json enum-max-member.json enum-union-clash.json \
        enum-bad-name.json funny-char.json indented-expr.json \
        missing-type.json bad-ident.json ident-with-escape.json \
+       escape-outside-string.json unknown-escape.json \
+       escape-too-short.json escape-too-big.json unicode-str.json \
        double-type.json bad-base.json bad-type-bool.json bad-type-int.json \
        bad-type-dict.json double-data.json unknown-expr-key.json \
        redefined-type.json redefined-command.json redefined-builtin.json \

diff --git a/tests/qapi-schema/escape-outside-string.err b/tests/qapi-schema/escape-outside-string.err
new file mode 100644 (file)
index 0000000..b9b8837
--- /dev/null
+++ b/tests/qapi-schema/escape-outside-string.err
@@ -0,0 +1 @@
+tests/qapi-schema/escape-outside-string.json:3:27: Stray "\"

diff --git a/tests/qapi-schema/escape-outside-string.exit b/tests/qapi-schema/escape-outside-string.exit
new file mode 100644 (file)
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/escape-outside-string.exit
@@ -0,0 +1 @@
+1

diff --git a/tests/qapi-schema/escape-outside-string.json b/tests/qapi-schema/escape-outside-string.json
new file mode 100644 (file)
index 0000000..482f795
--- /dev/null
+++ b/tests/qapi-schema/escape-outside-string.json
@@ -0,0 +1,3 @@
+# escape sequences are permitted only inside strings
+# { 'command': 'foo', 'data': {} }
+{ 'command': 'foo', 'data'\u003a{} }

diff --git a/tests/qapi-schema/escape-outside-string.out b/tests/qapi-schema/escape-outside-string.out
new file mode 100644 (file)
index 0000000..e69de29

diff --git a/tests/qapi-schema/escape-too-big.err b/tests/qapi-schema/escape-too-big.err
new file mode 100644 (file)
index 0000000..d9aeb5d
--- /dev/null
+++ b/tests/qapi-schema/escape-too-big.err
@@ -0,0 +1 @@
+tests/qapi-schema/escape-too-big.json:3:14: For now, \u escape only supports non-zero values up to \u007f

diff --git a/tests/qapi-schema/escape-too-big.exit b/tests/qapi-schema/escape-too-big.exit
new file mode 100644 (file)
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/escape-too-big.exit
@@ -0,0 +1 @@
+1

diff --git a/tests/qapi-schema/escape-too-big.json b/tests/qapi-schema/escape-too-big.json
new file mode 100644 (file)
index 0000000..62bcecd
--- /dev/null
+++ b/tests/qapi-schema/escape-too-big.json
@@ -0,0 +1,3 @@
+# we don't support full Unicode strings, yet
+# { 'command': 'é' }
+{ 'command': '\u00e9' }

diff --git a/tests/qapi-schema/escape-too-big.out b/tests/qapi-schema/escape-too-big.out
new file mode 100644 (file)
index 0000000..e69de29

diff --git a/tests/qapi-schema/escape-too-short.err b/tests/qapi-schema/escape-too-short.err
new file mode 100644 (file)
index 0000000..934de59
--- /dev/null
+++ b/tests/qapi-schema/escape-too-short.err
@@ -0,0 +1 @@
+tests/qapi-schema/escape-too-short.json:3:14: \u escape needs 4 hex digits

diff --git a/tests/qapi-schema/escape-too-short.exit b/tests/qapi-schema/escape-too-short.exit
new file mode 100644 (file)
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/escape-too-short.exit
@@ -0,0 +1 @@
+1

diff --git a/tests/qapi-schema/escape-too-short.json b/tests/qapi-schema/escape-too-short.json
new file mode 100644 (file)
index 0000000..6cb1dec
--- /dev/null
+++ b/tests/qapi-schema/escape-too-short.json
@@ -0,0 +1,3 @@
+# the \u escape requires 4 hex digits
+# { 'command': 'a' }
+{ 'command': '\u61' }

diff --git a/tests/qapi-schema/escape-too-short.out b/tests/qapi-schema/escape-too-short.out
new file mode 100644 (file)
index 0000000..e69de29

diff --git a/tests/qapi-schema/ident-with-escape.err b/tests/qapi-schema/ident-with-escape.err
index f7d1c5532739626b51ee50b9cd4c0be03c7962b2..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 100644 (file)
--- a/tests/qapi-schema/ident-with-escape.err
+++ b/tests/qapi-schema/ident-with-escape.err
@@ -1 +0,0 @@
-tests/qapi-schema/ident-with-escape.json:3: Expression is missing metatype

diff --git a/tests/qapi-schema/ident-with-escape.exit b/tests/qapi-schema/ident-with-escape.exit
index d00491fd7e5bb6fa28c517a0bb32b8b506539d4d..573541ac9702dd3969c9bc859d2b91ec1f7e6e56 100644 (file)
--- a/tests/qapi-schema/ident-with-escape.exit
+++ b/tests/qapi-schema/ident-with-escape.exit
@@ -1 +1 @@
-1
+0

diff --git a/tests/qapi-schema/ident-with-escape.json b/tests/qapi-schema/ident-with-escape.json
index cfb205052a44c0a9c9a1be1d8646c77854b60422..56617501e7b0d79bd005feed6008c4a7905711bc 100644 (file)
--- a/tests/qapi-schema/ident-with-escape.json
+++ b/tests/qapi-schema/ident-with-escape.json
@@ -1,4 +1,4 @@
-# FIXME: we should allow escape sequences in strings, if they map back to ASCII
+# we allow escape sequences in strings, if they map back to ASCII
 # { 'command': 'fooA', 'data': { 'bar1': 'str' } }
 { 'c\u006fmmand': '\u0066\u006f\u006FA',
   'd\u0061ta': { '\u0062\u0061\u00721': '\u0073\u0074\u0072' } }

diff --git a/tests/qapi-schema/ident-with-escape.out b/tests/qapi-schema/ident-with-escape.out
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..402843081b30f45ce0966c8b88e6011a45ebb0bd 100644 (file)
--- a/tests/qapi-schema/ident-with-escape.out
+++ b/tests/qapi-schema/ident-with-escape.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'fooA'), ('data', OrderedDict([('bar1', 'str')]))])]
+[]
+[]

diff --git a/tests/qapi-schema/unicode-str.err b/tests/qapi-schema/unicode-str.err
new file mode 100644 (file)
index 0000000..f621cd6
--- /dev/null
+++ b/tests/qapi-schema/unicode-str.err
@@ -0,0 +1 @@
+tests/qapi-schema/unicode-str.json:2: 'command' uses invalid name 'é'

diff --git a/tests/qapi-schema/unicode-str.exit b/tests/qapi-schema/unicode-str.exit
new file mode 100644 (file)
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/unicode-str.exit
@@ -0,0 +1 @@
+1

diff --git a/tests/qapi-schema/unicode-str.json b/tests/qapi-schema/unicode-str.json
new file mode 100644 (file)
index 0000000..5253a1b
--- /dev/null
+++ b/tests/qapi-schema/unicode-str.json
@@ -0,0 +1,2 @@
+# we don't support full Unicode strings, yet
+{ 'command': 'é' }

diff --git a/tests/qapi-schema/unicode-str.out b/tests/qapi-schema/unicode-str.out
new file mode 100644 (file)
index 0000000..e69de29

diff --git a/tests/qapi-schema/unknown-escape.err b/tests/qapi-schema/unknown-escape.err
new file mode 100644 (file)
index 0000000..000e30d
--- /dev/null
+++ b/tests/qapi-schema/unknown-escape.err
@@ -0,0 +1 @@
+tests/qapi-schema/unknown-escape.json:3:21: Unknown escape \x

diff --git a/tests/qapi-schema/unknown-escape.exit b/tests/qapi-schema/unknown-escape.exit
new file mode 100644 (file)
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/unknown-escape.exit
@@ -0,0 +1 @@
+1

diff --git a/tests/qapi-schema/unknown-escape.json b/tests/qapi-schema/unknown-escape.json
new file mode 100644 (file)
index 0000000..8e6891e
--- /dev/null
+++ b/tests/qapi-schema/unknown-escape.json
@@ -0,0 +1,3 @@
+# we only recognize JSON escape sequences, plus our \' extension (no \x)
+# { 'command': 'foo', 'data': {} }
+{ 'command': 'foo', 'dat\x61':{} }

diff --git a/tests/qapi-schema/unknown-escape.out b/tests/qapi-schema/unknown-escape.out
new file mode 100644 (file)
index 0000000..e69de29