We have multiple clients of qemu_strtosz (qemu-io, the opts visitor,
the keyval visitor), and it gets annoying that edge-case testing is
impacted by implicit rounding to 53 bits of precision due to parsing
with strtod(). As an example posted by Rich Jones:
$ nbdkit memory $(( 2**63 - 2**30 )) --run \
'build/qemu-io -f raw "$uri" -c "w -P 3 $(( 2**63 - 2**30 - 512 )) 512" '
write failed: Input/output error
because
9223372035781033472 got rounded to 0x7fffffffc0000000 which is
out of bounds.
It is also worth noting that our existing parser, by virtue of using
strtod(), accepts decimal AND hex numbers, even though test-cutils
previously lacked any coverage of the latter until the previous patch.
We do have existing clients that expect a hex parse to work (for
example, iotest 33 using qemu-io -c "write -P 0xa 0x200 0x400"), but
strtod() parses "08" as 8 rather than as an invalid octal number, so
we know there are no clients that depend on octal. Our use of
strtod() also means that "0x1.8k" would actually parse as 1536 (the
fraction is 8/16), rather than 1843 (if the fraction were 8/10); but
as this was not covered in the testsuite, I have no qualms forbidding
hex fractions as invalid, so this patch declares that the use of
fractions is only supported with decimal input, and enhances the
testsuite to document that.
Our previous use of strtod() meant that -1 parsed as a negative; now
that we parse with strtoull(), negative values can wrap around modulo
2^64, so we have to explicitly check whether the user passed in a '-';
and make it consistent to also reject '-0'. This has the minor effect
of treating negative values as EINVAL (with no change to endptr)
rather than ERANGE (with endptr advanced to what was parsed), visible
in the updated iotest output.
We also had no testsuite coverage of "1.1e0k", which happened to parse
under strtod() but is unlikely to occur in practice; as long as we are
making things more robust, it is easy enough to reject the use of
exponents in a strtod parse.
The fix is done by breaking the parse into an integer prefix (no loss
in precision), rejecting negative values (since we can no longer rely
on strtod() to do that), determining if a decimal or hexadecimal parse
was intended (with the new restriction that a fractional hex parse is
not allowed), and where appropriate, using a floating point fractional
parse (where we also scan to reject use of exponents in the fraction).
The bulk of the patch is then updates to the testsuite to match our
new precision, as well as adding new cases we reject (whether they
were rejected or inadvertently accepted before).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <
20210211204438.
1184395-3-eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
== 3. Invalid sizes ==
qemu-img create -f qcow2 TEST_DIR/t.qcow2 -- -1024
-qemu-img: Invalid image size specified. Must be between 0 and 9223372036854775807.
+qemu-img: Invalid image size specified. You may use k, M, G, T, P or E suffixes for
+qemu-img: kilobytes, megabytes, gigabytes, terabytes, petabytes and exabytes.
qemu-img create -f qcow2 -o size=-1024 TEST_DIR/t.qcow2
-qemu-img: TEST_DIR/t.qcow2: Value '-1024' is out of range for parameter 'size'
+qemu-img: TEST_DIR/t.qcow2: Parameter 'size' expects a non-negative number below 2^64
+Optional suffix k, M, G, T, P or E means kilo-, mega-, giga-, tera-, peta-
+and exabytes, respectively.
qemu-img create -f qcow2 TEST_DIR/t.qcow2 -- -1k
-qemu-img: Invalid image size specified. Must be between 0 and 9223372036854775807.
+qemu-img: Invalid image size specified. You may use k, M, G, T, P or E suffixes for
+qemu-img: kilobytes, megabytes, gigabytes, terabytes, petabytes and exabytes.
qemu-img create -f qcow2 -o size=-1k TEST_DIR/t.qcow2
-qemu-img: TEST_DIR/t.qcow2: Value '-1k' is out of range for parameter 'size'
+qemu-img: TEST_DIR/t.qcow2: Parameter 'size' expects a non-negative number below 2^64
+Optional suffix k, M, G, T, P or E means kilo-, mega-, giga-, tera-, peta-
+and exabytes, respectively.
qemu-img create -f qcow2 TEST_DIR/t.qcow2 -- 1kilobyte
qemu-img: Invalid image size specified. You may use k, M, G, T, P or E suffixes for
qemu-img: Invalid parameter 'snapshot.foo'
qemu-img: Failed in parsing snapshot param 'snapshot.foo=bar'
qemu-img: --output must be used with human or json as argument.
-qemu-img: Invalid image size specified. Must be between 0 and 9223372036854775807.
+qemu-img: Invalid image size specified. You may use k, M, G, T, P or E suffixes for
+qemu-img: kilobytes, megabytes, gigabytes, terabytes, petabytes and exabytes.
qemu-img: Unknown file format 'foo'
== Size calculation for a new file (human) ==
qemu-img: Invalid parameter 'snapshot.foo'
qemu-img: Failed in parsing snapshot param 'snapshot.foo=bar'
qemu-img: --output must be used with human or json as argument.
-qemu-img: Invalid image size specified. Must be between 0 and 9223372036854775807.
+qemu-img: Invalid image size specified. You may use k, M, G, T, P or E suffixes for
+qemu-img: kilobytes, megabytes, gigabytes, terabytes, petabytes and exabytes.
qemu-img: Unknown file format 'foo'
== Size calculation for a new file (human) ==
g_assert_cmpint(err, ==, 0);
g_assert_cmpint(res, ==, 12345);
- /* Note: precision is 53 bits since we're parsing with strtod() */
-
str = "9007199254740991"; /* 2^53-1 */
err = qemu_strtosz(str, &endptr, &res);
g_assert_cmpint(err, ==, 0);
g_assert_cmpint(res, ==, 0x20000000000000);
g_assert(endptr == str + 16);
- str = "9007199254740993"; /* 2^53+1 FIXME loss of precision is a bug */
+ str = "9007199254740993"; /* 2^53+1 */
err = qemu_strtosz(str, &endptr, &res);
g_assert_cmpint(err, ==, 0);
- g_assert_cmpint(res, ==, 0x20000000000000); /* rounded to 53 bits */
+ g_assert_cmpint(res, ==, 0x20000000000001);
g_assert(endptr == str + 16);
str = "18446744073709549568"; /* 0xfffffffffffff800 (53 msbs set) */
g_assert_cmpint(res, ==, 0xfffffffffffff800);
g_assert(endptr == str + 20);
- str = "18446744073709550591"; /* 0xfffffffffffffbff FIXME */
+ str = "18446744073709550591"; /* 0xfffffffffffffbff */
err = qemu_strtosz(str, &endptr, &res);
g_assert_cmpint(err, ==, 0);
- g_assert_cmpint(res, ==, 0xfffffffffffff800); /* rounded to 53 bits */
+ g_assert_cmpint(res, ==, 0xfffffffffffffbff);
g_assert(endptr == str + 20);
- /*
- * FIXME 0xfffffffffffffe00..0xffffffffffffffff get rounded to
- * 2^64, thus -ERANGE; see test_qemu_strtosz_erange()
- */
+ str = "18446744073709551615"; /* 0xffffffffffffffff */
+ err = qemu_strtosz(str, &endptr, &res);
+ g_assert_cmpint(err, ==, 0);
+ g_assert_cmpint(res, ==, 0xffffffffffffffff);
+ g_assert(endptr == str + 20);
}
static void test_qemu_strtosz_hex(void)
g_assert(endptr == str);
/* Fractional values require scale larger than bytes */
- /* FIXME endptr shouldn't move on -EINVAL */
str = "1.1B";
err = qemu_strtosz(str, &endptr, &res);
g_assert_cmpint(err, ==, -EINVAL);
- g_assert(endptr == str + 4);
+ g_assert(endptr == str);
- /* FIXME endptr shouldn't move on -EINVAL */
str = "1.1";
err = qemu_strtosz(str, &endptr, &res);
g_assert_cmpint(err, ==, -EINVAL);
- g_assert(endptr == str + 3);
+ g_assert(endptr == str);
- /* FIXME we should reject floating point exponents */
+ /* No floating point exponents */
str = "1.5e1k";
err = qemu_strtosz(str, &endptr, &res);
- g_assert_cmpint(err, ==, 0);
- g_assert_cmpint(res, ==, 15360);
- g_assert(endptr == str + 6);
+ g_assert_cmpint(err, ==, -EINVAL);
+ g_assert(endptr == str);
- /* FIXME we should reject floating point exponents */
str = "1.5E+0k";
err = qemu_strtosz(str, &endptr, &res);
- g_assert_cmpint(err, ==, 0);
- g_assert_cmpint(res, ==, 1536);
- g_assert(endptr == str + 7);
+ g_assert_cmpint(err, ==, -EINVAL);
+ g_assert(endptr == str);
- /* FIXME we should reject hex fractions */
+ /* No hex fractions */
str = "0x1.8k";
err = qemu_strtosz(str, &endptr, &res);
- g_assert_cmpint(err, ==, 0);
- g_assert_cmpint(res, ==, 1536);
- g_assert(endptr == str + 6);
+ g_assert_cmpint(err, ==, -EINVAL);
+ g_assert(endptr == str);
- /* FIXME we reject all other attempts at negative, why not -0 */
+ /* No negative values */
str = "-0";
err = qemu_strtosz(str, &endptr, &res);
- g_assert_cmpint(err, ==, 0);
- g_assert_cmpint(res, ==, 0);
- g_assert(endptr == str + 2);
+ g_assert_cmpint(err, ==, -EINVAL);
+ g_assert(endptr == str);
+
+ str = "-1";
+ err = qemu_strtosz(str, &endptr, &res);
+ g_assert_cmpint(err, ==, -EINVAL);
+ g_assert(endptr == str);
}
static void test_qemu_strtosz_trailing(void)
int err;
uint64_t res = 0xbaadf00d;
- str = "-1";
- err = qemu_strtosz(str, &endptr, &res);
- g_assert_cmpint(err, ==, -ERANGE);
- g_assert(endptr == str + 2);
-
- str = "18446744073709550592"; /* 0xfffffffffffffc00 FIXME accept this */
- err = qemu_strtosz(str, &endptr, &res);
- g_assert_cmpint(err, ==, -ERANGE);
- g_assert(endptr == str + 20);
-
- str = "18446744073709551615"; /* 2^64-1 FIXME accept this */
- err = qemu_strtosz(str, &endptr, &res);
- g_assert_cmpint(err, ==, -ERANGE);
- g_assert(endptr == str + 20);
-
- str = "18446744073709551616"; /* 2^64 */
+ str = "18446744073709551616"; /* 2^64; see strtosz_simple for 2^64-1 */
err = qemu_strtosz(str, &endptr, &res);
g_assert_cmpint(err, ==, -ERANGE);
g_assert(endptr == str + 20);
visit_end_struct(v, NULL);
visit_free(v);
- /* Note: precision is 53 bits since we're parsing with strtod() */
+ /* Note: full 64 bits of precision */
- /* Around limit of precision: 2^53-1, 2^53, 2^53+1 */
+ /* Around double limit of precision: 2^53-1, 2^53, 2^53+1 */
qdict = keyval_parse("sz1=9007199254740991,"
"sz2=9007199254740992,"
"sz3=9007199254740993",
visit_type_size(v, "sz2", &sz, &error_abort);
g_assert_cmphex(sz, ==, 0x20000000000000);
visit_type_size(v, "sz3", &sz, &error_abort);
- g_assert_cmphex(sz, ==, 0x20000000000000);
+ g_assert_cmphex(sz, ==, 0x20000000000001);
visit_check_struct(v, &error_abort);
visit_end_struct(v, NULL);
visit_free(v);
- /* Close to signed upper limit 0x7ffffffffffffc00 (53 msbs set) */
- qdict = keyval_parse("sz1=9223372036854774784," /* 7ffffffffffffc00 */
- "sz2=9223372036854775295", /* 7ffffffffffffdff */
+ /* Close to signed integer limit 2^63 */
+ qdict = keyval_parse("sz1=9223372036854775807," /* 7fffffffffffffff */
+ "sz2=9223372036854775808," /* 8000000000000000 */
+ "sz3=9223372036854775809", /* 8000000000000001 */
NULL, NULL, &error_abort);
v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
qobject_unref(qdict);
visit_start_struct(v, NULL, NULL, 0, &error_abort);
visit_type_size(v, "sz1", &sz, &error_abort);
- g_assert_cmphex(sz, ==, 0x7ffffffffffffc00);
+ g_assert_cmphex(sz, ==, 0x7fffffffffffffff);
visit_type_size(v, "sz2", &sz, &error_abort);
- g_assert_cmphex(sz, ==, 0x7ffffffffffffc00);
+ g_assert_cmphex(sz, ==, 0x8000000000000000);
+ visit_type_size(v, "sz3", &sz, &error_abort);
+ g_assert_cmphex(sz, ==, 0x8000000000000001);
visit_check_struct(v, &error_abort);
visit_end_struct(v, NULL);
visit_free(v);
visit_type_size(v, "sz1", &sz, &error_abort);
g_assert_cmphex(sz, ==, 0xfffffffffffff800);
visit_type_size(v, "sz2", &sz, &error_abort);
- g_assert_cmphex(sz, ==, 0xfffffffffffff800);
+ g_assert_cmphex(sz, ==, 0xfffffffffffffbff);
+ visit_check_struct(v, &error_abort);
+ visit_end_struct(v, NULL);
+ visit_free(v);
+
+ /* Actual limit 2^64-1*/
+ qdict = keyval_parse("sz1=18446744073709551615", /* ffffffffffffffff */
+ NULL, NULL, &error_abort);
+ v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
+ qobject_unref(qdict);
+ visit_start_struct(v, NULL, NULL, 0, &error_abort);
+ visit_type_size(v, "sz1", &sz, &error_abort);
+ g_assert_cmphex(sz, ==, 0xffffffffffffffff);
visit_check_struct(v, &error_abort);
visit_end_struct(v, NULL);
visit_free(v);
/* Beyond limits */
qdict = keyval_parse("sz1=-1,"
- "sz2=18446744073709550592", /* fffffffffffffc00 */
+ "sz2=18446744073709551616", /* 2^64 */
NULL, NULL, &error_abort);
v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
qobject_unref(qdict);
g_assert_cmpuint(opts_count(opts), ==, 1);
g_assert_cmpuint(qemu_opt_get_size(opts, "size1", 1), ==, 0);
- /* Note: precision is 53 bits since we're parsing with strtod() */
+ /* Note: full 64 bits of precision */
- /* Around limit of precision: 2^53-1, 2^53, 2^54 */
+ /* Around double limit of precision: 2^53-1, 2^53, 2^53+1 */
opts = qemu_opts_parse(&opts_list_02,
"size1=9007199254740991,"
"size2=9007199254740992,"
g_assert_cmphex(qemu_opt_get_size(opts, "size2", 1),
==, 0x20000000000000);
g_assert_cmphex(qemu_opt_get_size(opts, "size3", 1),
- ==, 0x20000000000000);
+ ==, 0x20000000000001);
- /* Close to signed upper limit 0x7ffffffffffffc00 (53 msbs set) */
+ /* Close to signed int limit: 2^63-1, 2^63, 2^63+1 */
opts = qemu_opts_parse(&opts_list_02,
- "size1=9223372036854774784," /* 7ffffffffffffc00 */
- "size2=9223372036854775295", /* 7ffffffffffffdff */
+ "size1=9223372036854775807," /* 7fffffffffffffff */
+ "size2=9223372036854775808," /* 8000000000000000 */
+ "size3=9223372036854775809", /* 8000000000000001 */
false, &error_abort);
- g_assert_cmpuint(opts_count(opts), ==, 2);
+ g_assert_cmpuint(opts_count(opts), ==, 3);
g_assert_cmphex(qemu_opt_get_size(opts, "size1", 1),
- ==, 0x7ffffffffffffc00);
+ ==, 0x7fffffffffffffff);
g_assert_cmphex(qemu_opt_get_size(opts, "size2", 1),
- ==, 0x7ffffffffffffc00);
+ ==, 0x8000000000000000);
+ g_assert_cmphex(qemu_opt_get_size(opts, "size3", 1),
+ ==, 0x8000000000000001);
/* Close to actual upper limit 0xfffffffffffff800 (53 msbs set) */
opts = qemu_opts_parse(&opts_list_02,
g_assert_cmphex(qemu_opt_get_size(opts, "size1", 1),
==, 0xfffffffffffff800);
g_assert_cmphex(qemu_opt_get_size(opts, "size2", 1),
- ==, 0xfffffffffffff800);
+ ==, 0xfffffffffffffbff);
+
+ /* Actual limit, 2^64-1 */
+ opts = qemu_opts_parse(&opts_list_02,
+ "size1=18446744073709551615", /* ffffffffffffffff */
+ false, &error_abort);
+ g_assert_cmpuint(opts_count(opts), ==, 1);
+ g_assert_cmphex(qemu_opt_get_size(opts, "size1", 1),
+ ==, 0xffffffffffffffff);
/* Beyond limits */
opts = qemu_opts_parse(&opts_list_02, "size1=-1", false, &err);
error_free_or_abort(&err);
g_assert(!opts);
opts = qemu_opts_parse(&opts_list_02,
- "size1=18446744073709550592", /* fffffffffffffc00 */
+ "size1=18446744073709551616", /* 2^64 */
false, &err);
error_free_or_abort(&err);
g_assert(!opts);
}
/*
- * Convert string to bytes, allowing either B/b for bytes, K/k for KB,
- * M/m for MB, G/g for GB or T/t for TB. End pointer will be returned
- * in *end, if not NULL. Return -ERANGE on overflow, and -EINVAL on
- * other error.
+ * Convert size string to bytes.
+ *
+ * The size parsing supports the following syntaxes
+ * - 12345 - decimal, scale determined by @default_suffix and @unit
+ * - 12345{bBkKmMgGtTpPeE} - decimal, scale determined by suffix and @unit
+ * - 12345.678{kKmMgGtTpPeE} - decimal, scale determined by suffix, and
+ * fractional portion is truncated to byte
+ * - 0x7fEE - hexadecimal, unit determined by @default_suffix
+ *
+ * The following are intentionally not supported
+ * - octal, such as 08
+ * - fractional hex, such as 0x1.8
+ * - floating point exponents, such as 1e3
+ *
+ * The end pointer will be returned in *end, if not NULL. If there is
+ * no fraction, the input can be decimal or hexadecimal; if there is a
+ * fraction, then the input must be decimal and there must be a suffix
+ * (possibly by @default_suffix) larger than Byte, and the fractional
+ * portion may suffer from precision loss or rounding. The input must
+ * be positive.
+ *
+ * Return -ERANGE on overflow (with *@end advanced), and -EINVAL on
+ * other error (with *@end left unchanged).
*/
static int do_strtosz(const char *nptr, const char **end,
const char default_suffix, int64_t unit,
uint64_t *result)
{
int retval;
- const char *endptr;
+ const char *endptr, *f;
unsigned char c;
- int mul_required = 0;
- double val, mul, integral, fraction;
+ bool mul_required = false;
+ uint64_t val;
+ int64_t mul;
+ double fraction = 0.0;
- retval = qemu_strtod_finite(nptr, &endptr, &val);
+ /* Parse integral portion as decimal. */
+ retval = qemu_strtou64(nptr, &endptr, 10, &val);
if (retval) {
goto out;
}
- fraction = modf(val, &integral);
- if (fraction != 0) {
- mul_required = 1;
+ if (memchr(nptr, '-', endptr - nptr) != NULL) {
+ endptr = nptr;
+ retval = -EINVAL;
+ goto out;
+ }
+ if (val == 0 && (*endptr == 'x' || *endptr == 'X')) {
+ /* Input looks like hex, reparse, and insist on no fraction. */
+ retval = qemu_strtou64(nptr, &endptr, 16, &val);
+ if (retval) {
+ goto out;
+ }
+ if (*endptr == '.') {
+ endptr = nptr;
+ retval = -EINVAL;
+ goto out;
+ }
+ } else if (*endptr == '.') {
+ /*
+ * Input looks like a fraction. Make sure even 1.k works
+ * without fractional digits. If we see an exponent, treat
+ * the entire input as invalid instead.
+ */
+ f = endptr;
+ retval = qemu_strtod_finite(f, &endptr, &fraction);
+ if (retval) {
+ fraction = 0.0;
+ endptr++;
+ } else if (memchr(f, 'e', endptr - f) || memchr(f, 'E', endptr - f)) {
+ endptr = nptr;
+ retval = -EINVAL;
+ goto out;
+ } else if (fraction != 0) {
+ mul_required = true;
+ }
}
c = *endptr;
mul = suffix_mul(c, unit);
- if (mul >= 0) {
+ if (mul > 0) {
endptr++;
} else {
mul = suffix_mul(default_suffix, unit);
- assert(mul >= 0);
+ assert(mul > 0);
}
if (mul == 1 && mul_required) {
+ endptr = nptr;
retval = -EINVAL;
goto out;
}
- /*
- * Values near UINT64_MAX overflow to 2**64 when converting to double
- * precision. Compare against the maximum representable double precision
- * value below 2**64, computed as "the next value after 2**64 (0x1p64) in
- * the direction of 0".
- */
- if ((val * mul > nextafter(0x1p64, 0)) || val < 0) {
+ if (val > (UINT64_MAX - ((uint64_t) (fraction * mul))) / mul) {
retval = -ERANGE;
goto out;
}
- *result = val * mul;
+ *result = val * mul + (uint64_t) (fraction * mul);
retval = 0;
out:
projects / mirror_qemu.git / commitdiff