]> git.proxmox.com Git - ceph.git/blob - ceph/src/boost/libs/test/doc/testing_tools/testing_tools_reference.qbk
add subtree-ish sources for 12.0.3
[ceph.git] / ceph / src / boost / libs / test / doc / testing_tools / testing_tools_reference.qbk
1 [/
2 / Copyright (c) 2003 Boost.Test contributors
3 /
4 / Distributed under the Boost Software License, Version 1.0. (See accompanying
5 / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
6 /]
7
8
9 [section:testing_tool_ref Reference API for writing tests]
10
11
12
13 [/ ###############################################################################################]
14 [section:assertion_boost_test_universal_macro `BOOST_TEST`]
15
16
17 ``
18 BOOST_TEST(statement);
19 BOOST_TEST_<level>(statement);
20
21 // replacement failure message, requires variadic macros
22 BOOST_TEST(statement, "failure message");
23
24 // Floating point comparison, requires variadic macros, auto and decltype
25 BOOST_TEST(statement, floating_point_comparison_manipulation);
26
27 // bitwise comparison, requires variadic macros, auto and decltype
28 BOOST_TEST(statement, boost::test_tools::bitwise() );
29
30 // element-wise comparison, for containers
31 BOOST_TEST(statement, boost::test_tools::per_element() );
32
33 // lexicographic comparison, for containers
34 BOOST_TEST(statement, boost::test_tools::lexicographic() );
35 ``
36
37 The full documentation of this macro is located [link boost_test.testing_tools.boost_test_universal_macro here].
38
39 The macro is available in three variants, corresponding to different [link boost_test.testing_tools.tools_assertion_severity_level assertion severity levels]:
40
41 ``
42 BOOST_TEST // or BOOST_TEST_CHECK
43 BOOST_TEST_REQUIRE
44 BOOST_TEST_WARN
45 ``
46
47
48 * `"failure message"` is a C-string printed in case of failure in place of the default message.
49 See [link boost_test.testing_tools.reports this section] for
50 more details.
51 * `floating_point_comparison_manipulation` is one of the floating point comparison manipulators.
52 See [link boost_test.testing_tools.boost_test_universal_macro this section]
53 for more details.
54 * [classref boost::test_tools::bitwise] is a manipulator indicating that the comparison should be performed bitwise. See
55 [link boost_test.testing_tools.extended_comparison.bitwise this section] for more details
56 * [classref boost::test_tools::per_element] is a manipulator indicating that the comparison should be performed on each element, in sequence, rather
57 than on containers. See
58 [link boost_test_coll_perelement this section] for more details
59 * [classref boost::test_tools::lexicographic] is a manipulator indicating that the comparison should be performed with the lexicographic order. See
60 [link boost_test_coll_default_lex this section] for more details
61
62 [h3 Limitations and workaround]
63 There are some restrictions on the statements that are supported by this tool. Those are explained in details in
64 [link boost_test_statement_limitations this] section.
65
66 [endsect]
67
68
69
70 [/ DECORATORS ###############################################################################################]
71 [/-----------------------------------------------------------------]
72
73 [section:decorator_expected_failures expected_failures (decorator)]
74
75 ``
76 expected_failures(counter_t number);
77 ``
78
79 Indicates the expected failures for a test unit.
80 See [link boost_test.testing_tools.expected_failures here] for more details.
81
82 [endsect] [/ section expected_failures]
83
84
85 [/-----------------------------------------------------------------]
86 [section:decorator_timeout timeout (decorator)]
87
88 ``
89 timeout(unsigned seconds);
90 ``
91
92 Specifies a time-out for a *test-case*, above which the test-case is forced to stop and reported as failing.
93 See [link boost_test.testing_tools.timeout here] for more details.
94
95 [endsect] [/ section timeout]
96
97
98 [/-----------------------------------------------------------------]
99 [section:decorator_tolerance tolerance (decorator)]
100
101 ``
102 template <typename FPT>
103 tolerance(FPT eps);
104
105 template <typename FPT>
106 tolerance(test_tools::fpc::percent_tolerance_t<FPT> eps)
107 ``
108
109 Decorator `tolerance` specifies the default comparison tolerance for floating point type `FTP` in the decorated test
110 unit. The default tolerance only applies to a particular type, so it makes sense to provide more than one `tolerance`
111 decorator if we are comparing different floating point types. For more details see __floating_points_testing_impl__.
112 The variant with `percent_tolerance` uses value `eps / 100` as tolerance.
113
114 [bt_example decorator_13..decorator tolerance..run-fail]
115
116 In the above example, in `test1`, checks on `double`s fail because they differ by more what tolerance for `double`s
117 specifies. In `test2` the tolerance for `double`s is greater and therefore the checks succeed. In `test3`, we specify
118 only tolerance for type `float`, and since the checks use type `double` the specified tolerance does not apply. Tolerance
119 in `test4` is equivalent to that in `test1`, therefore its checks also fail. Tolerance in `test5` is equivalent to
120 that in `test2`, therefore its checks also succeed.
121 [endsect] [/ section decorator_tolerance]
122
123
124
125
126
127 [/ DEPRECATED API]
128 [/ ###############################################################################################]
129 [#ref_BOOST_level][section:assertion_boost_level `BOOST_<level>`]
130
131
132 ``
133 BOOST_WARN(predicate);
134 BOOST_CHECK(predicate);
135 BOOST_REQUIRE(predicate);
136 ``
137
138 These tools are used to validate the predicate value. The only parameter for these tools is a boolean predicate
139 value that gets validated. It could be any expression that could be evaluated and converted to boolean value. The
140 expression gets evaluated only once, so it's safe to pass complex expression for validation.
141
142 [bt_example example34..BOOST_<level> usage..run-fail]
143
144 See also:
145
146 * __BOOST_LEVEL_MESSAGE__
147
148 [endsect]
149
150
151 [/ ###############################################################################################]
152 [section:assertion_boost_level_bitwise_eq `BOOST_<level>_BITWISE_EQUAL`]
153
154
155 ``
156 BOOST_WARN_BITWISE_EQUAL(left, right);
157 BOOST_CHECK_BITWISE_EQUAL(left, right);
158 BOOST_REQUIRE_BITWISE_EQUAL(left, right);
159 ``
160
161 These tools are used to perform bitwise comparison of two values. The check shows all positions where left and
162 right value's bits mismatch.
163
164 The first parameter is the left compared value. The second parameter is the right compared value. Parameters are
165 not required to be of the same type, but warning is issued if their type's size does not coincide.
166
167 [bt_example example33..BOOST_<level>_BITWISE_EQUAL usage..run-fail]
168
169 See also:
170
171 * __BOOST_LEVEL_EQUAL__
172
173 [endsect]
174
175 [/ ###############################################################################################]
176 [section:assertion_boost_level_eq `BOOST_<level>_EQUAL`]
177
178 ``
179 BOOST_WARN_EQUAL(left, right);
180 BOOST_CHECK_EQUAL(left, right);
181 BOOST_REQUIRE_EQUAL(left, right);
182 ``
183
184 Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__(left == right)`.
185 The difference is that the mismatched values are reported as well.
186
187 [note It is bad idea to use these tools to compare floating point values. Use __BOOST_LEVEL_CLOSE__ or
188 __BOOST_LEVEL_CLOSE_FRACTION__ tools instead.
189 ]
190
191 [bt_example example35..BOOST_<level>_EQUAL usage..run-fail]
192
193 See also:
194
195 * __BOOST_LEVEL__
196 * __BOOST_LEVEL_CLOSE__
197 * __BOOST_LEVEL_NE__
198 * __BOOST_LEVEL_EQUAL_COLLECTIONS__
199
200 [endsect]
201
202 [/ ###############################################################################################]
203 [section:assertion_boost_level_eq_collections `BOOST_<level>_EQUAL_COLLECTIONS`]
204
205 ``
206 BOOST_WARN_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
207 BOOST_CHECK_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
208 BOOST_REQUIRE_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
209 ``
210
211 These tools are used to perform an element by element comparison of two collections. They print all mismatched
212 positions, collection elements at these positions and check that the collections have the same size. The first two
213 parameters designate begin and end of the first collection. The two last parameters designate begin and end of the
214 second collection.
215
216 [bt_example example36..BOOST_<level>_EQUAL_COLLECTIONS usage..run-fail]
217
218 See also:
219
220 * __BOOST_LEVEL_EQUAL__
221
222 [endsect]
223
224 [/ ###############################################################################################]
225 [section:assertion_boost_level_close `BOOST_<level>_CLOSE`]
226
227 ``
228 BOOST_WARN_CLOSE(left, right, tolerance);
229 BOOST_CHECK_CLOSE(left, right, tolerance);
230 BOOST_REQUIRE_CLOSE(left, right, tolerance);
231 ``
232
233 These tools are used to check on closeness using strong relationship defined by the predicate
234 ``check_is_close( left, right, tolerance )``
235
236 To check for the weak relationship use
237 __BOOST_LEVEL_PREDICATE__ family of tools with explicit `check_is_close` invocation.
238
239
240 The first parameter is the ['left] compared value. The second parameter is the
241 ['right] compared value. Last third parameter defines the tolerance for the comparison in
242 [link boost_test.testing_tools.extended_comparison.floating_point [*percentage units]].
243
244 [note It is required for left and right parameters to be of the same floating point type. You will need to explicitly
245 resolve any type mismatch to select which type to use for comparison.
246 ]
247
248 [note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`.
249 ]
250
251 [bt_example example42..BOOST_<level>_CLOSE usage with small values..run-fail]
252 [bt_example example43..BOOST_<level>_CLOSE usage with big values..run]
253
254 See also:
255
256 * __BOOST_LEVEL_CLOSE_FRACTION__
257 * __BOOST_LEVEL_SMALL__
258 * __BOOST_LEVEL_EQUAL__
259 * __floating_points_testing_tools__
260
261 [endsect]
262
263 [/ ###############################################################################################]
264 [section:assertion_boost_level_close_fraction `BOOST_<level>_CLOSE_FRACTION`]
265
266 ``
267 BOOST_WARN_CLOSE_FRACTION(left, right, tolerance);
268 BOOST_CHECK_CLOSE_FRACTION(left, right, tolerance);
269 BOOST_REQUIRE_CLOSE_FRACTION(left, right, tolerance);
270 ``
271
272 These tools are used to check on closeness using strong relationship defined by the predicate
273 ``check_is_close(left, right, tolerance)``
274
275 To check for the weak relationship use __BOOST_LEVEL_PREDICATE__ family of tools with explicit `check_is_close` invocation.
276
277 The first parameter is the ['left] compared value. The second parameter is the
278 ['right] compared value. Last third parameter defines the tolerance for the comparison as
279 [link boost_test.testing_tools.extended_comparison.floating_point [*fraction of absolute values being compared]].
280
281 [note It is required for left and right parameters to be of the same floating point type. You will need to explicitly
282 resolve any type mismatch to select which type to use for comparison.]
283
284 [note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`.]
285
286 [bt_example example44..BOOST_<level>_CLOSE_FRACTION usage..run-fail]
287
288 See also:
289
290 * __BOOST_LEVEL_CLOSE__
291 * __BOOST_LEVEL_SMALL__
292 * __BOOST_LEVEL_EQUAL__
293 * __floating_points_testing_tools__
294
295 [endsect]
296
297 [/ ###############################################################################################]
298 [section:assertion_boost_level_ge `BOOST_<level>_GE`]
299
300 ``
301 BOOST_WARN_GE(left, right);
302 BOOST_CHECK_GE(left, right);
303 BOOST_REQUIRE_GE(left, right);
304 ``
305
306 Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left >= right )`.
307 The difference is that the argument values are reported as well.
308
309 [bt_example example57..BOOST_<level>_GE usage..run-fail]
310
311 See also:
312
313 * __BOOST_LEVEL_LE__
314 * __BOOST_LEVEL_LT__
315 * __BOOST_LEVEL_GT__
316
317 [endsect]
318
319
320 [/ ###############################################################################################]
321 [section:assertion_boost_level_gt `BOOST_<level>_GT`]
322
323
324 ``
325 BOOST_WARN_GT(left, right);
326 BOOST_CHECK_GT(left, right);
327 BOOST_REQUIRE_GT(left, right);
328 ``
329
330 Check performed by these tools is the same as the one performed by __BOOST_LEVEL__`( left > right )`.
331 The difference is that the argument values are reported as well.
332
333 [bt_example example58..BOOST_<level>_GT usage..run-fail]
334
335 See also:
336
337 * __BOOST_LEVEL_LE__
338 * __BOOST_LEVEL_LT__
339 * __BOOST_LEVEL_GE__
340
341 [endsect]
342
343 [/ ###############################################################################################]
344 [section:assertion_boost_level_le `BOOST_<level>_LE`]
345
346 ``
347 BOOST_WARN_LE(left, right);
348 BOOST_CHECK_LE(left, right);
349 BOOST_REQUIRE_LE(left, right);
350 ``
351
352 Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left <= right )`.
353 The difference is that the argument values are reported as well.
354
355 [bt_example example55..BOOST_<level>_LE usage..run-fail]
356
357 See also:
358
359 * __BOOST_LEVEL_LE__
360 * __BOOST_LEVEL_GE__
361 * __BOOST_LEVEL_GT__
362
363 [endsect]
364
365 [/ ###############################################################################################]
366 [section:assertion_boost_level_lt `BOOST_<level>_LT`]
367
368 ``
369 BOOST_WARN_LT(left, right);
370 BOOST_CHECK_LT(left, right);
371 BOOST_REQUIRE_LT(left, right);
372 ``
373
374 Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left < right )`.
375 The difference is that the argument values are reported as well.
376
377 [bt_example example56..BOOST_<level>_LT usage..run-fail]
378
379 See also:
380
381 * __BOOST_LEVEL_LE__
382 * __BOOST_LEVEL_GE__
383 * __BOOST_LEVEL_GT__
384
385 [endsect]
386
387 [/ ###############################################################################################]
388 [section:assertion_boost_level_message `BOOST_<level>_MESSAGE`]
389
390 ``
391 BOOST_WARN_MESSAGE(predicate, message);
392 BOOST_CHECK_MESSAGE(predicate, message);
393 BOOST_REQUIRE_MESSAGE(predicate, message);
394 ``
395
396 These tools perform exactly the same check as __BOOST_LEVEL__ tools. The only difference is that
397 instead of generating an error/confirm message these use the supplied one.
398
399 The first parameter is the boolean expression. The second parameter is the message reported in case of check
400 failure. The message argument can be constructed of components of any type supporting the
401 `std::ostream& operator<<(std::ostream&)`.
402
403 [bt_example example38..BOOST_<level>_MESSAGE usage..run]
404
405 See also:
406
407 * __BOOST_LEVEL__
408
409 [endsect]
410
411
412 [/ ###############################################################################################]
413 [section:assertion_boost_level_ne `BOOST_<level>_NE`]
414
415
416 ``
417 BOOST_WARN_NE(left, right);
418 BOOST_CHECK_NE(left, right);
419 BOOST_REQUIRE_NE(left, right);
420 ``
421
422 Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left != right )`.
423 The difference is that the matched values are reported as well.
424
425 [bt_example example54..BOOST_<level>_NE usage..run-fail]
426
427 See also:
428
429 * __BOOST_LEVEL_EQUAL__
430
431 [endsect]
432
433 [/ ###############################################################################################]
434 [section:assertion_boost_level_predicate `BOOST_<level>_PREDICATE`]
435
436
437 ``
438 BOOST_WARN_PREDICATE(predicate, arguments_list);
439 BOOST_CHECK_PREDICATE(predicate, arguments_list);
440 BOOST_REQUIRE_PREDICATE(predicate, arguments_list);
441 ``
442
443 These are generic tools used to validate an arbitrary supplied predicate functor (there is a compile time limit on
444 predicate arity defined by the configurable macro `BOOST_TEST_MAX_PREDICATE_ARITY`). To
445 validate zero arity predicate use __BOOST_LEVEL__ tools. In other cases prefer theses tools. The
446 advantage of these tools is that they show arguments values in case of predicate failure.
447
448 The first parameter is the predicate itself. The second parameter is the list of predicate arguments each wrapped
449 in round brackets (`BOOST_PP` sequence format).
450
451 [bt_example example40..BOOST_<level>_PREDICATE usage..run]
452
453 [note Note difference in error log from __BOOST_LEVEL__]
454
455 See also:
456
457 * __BOOST_LEVEL__
458
459 [endsect]
460
461 [/ ###############################################################################################]
462 [section:assertion_boost_level_no_throw `BOOST_<level>_NO_THROW`]
463
464 ``
465 BOOST_WARN_NO_THROW(expression);
466 BOOST_CHECK_NO_THROW(expression);
467 BOOST_REQUIRE_NO_THROW(expression);
468 ``
469
470 These assertions validate that the execution of `expression` does not throw any exception.
471 To that extent, all possible exception are caught by assertion itself and no exception is propagated to
472 the test body.
473
474 [tip
475 It is possible to test for complex expressions with the use of constructs such as `do { /* ... */} while(0)` block.
476 ]
477
478 [bt_example exception_nothrow..BOOST_<level>_NO_THROW usage..run-fail]
479
480 See also:
481
482 * __BOOST_LEVEL_THROW__
483 * [link boost_test.testing_tools.exception_correctness Exception correctness] section
484
485 [endsect]
486
487 [/ ###############################################################################################]
488 [section:assertion_boost_level_throw `BOOST_<level>_THROW`]
489
490 ``
491 BOOST_WARN_THROW(expression, exception_type);
492 BOOST_CHECK_THROW(expression, exception_type);
493 BOOST_REQUIRE_THROW(expression, exception_type);
494 ``
495
496 These assertions validate that the execution of `expression` raises an /expected/ exception, which means an exception of
497 the supplied `exception_type` type or of any child type.
498
499 * If `expression` raises an unexpected exception, this exception is not caught by `BOOST_<level>_THROW` assertion and
500 might propagate to the test body. If not caught at all, the framework will catch it and terminate the test case
501 with the status /failed/.
502 * If `expression` does not raise any exception, the the assertion fails.
503
504 [warning the assertion catches only the expected exceptions.]
505
506 [tip
507 It is possible to test for complex expressions with the use of constructs such as `do { /* ... */} while(0)` block.
508 ]
509
510 [bt_example exception_check..BOOST_<level>_THROW usage..run-fail]
511
512 See also:
513
514 * __BOOST_LEVEL_NO_THROW__
515 * [link boost_test.testing_tools.exception_correctness Exception correctness] section
516
517
518 [endsect]
519
520 [/ ###############################################################################################]
521 [section:assertion_boost_level_exception `BOOST_<level>_EXCEPTION`]
522
523 ``
524 BOOST_WARN_EXCEPTION(expression, exception_type, predicate);
525 BOOST_CHECK_EXCEPTION(expression, exception_type, predicate);
526 BOOST_REQUIRE_EXCEPTION(expression, exception_type, predicate);
527 ``
528
529 As for __BOOST_LEVEL_THROW__, these assertions validate that `expression` raises an exception of the
530 type specified by `exception_type` or any of its child type, with additional checks on the exception instance.
531
532 * If an expected exception is raised by `expression`, the instance of the exception is passed to `predicate`
533 for further validation.
534 * It behaves like __BOOST_LEVEL_THROW__ if `expression` does not raise any exception, or an unrelated exception is raised.
535
536 `predicate` should be a unary function accepting an instance of `exception_type` or any of its child, and that should return
537 a boolean indicating a success (`true`) or a failure (`false`).
538
539 [warning the assertion catches only the expected exceptions.]
540
541 [tip
542 It is possible to test for complex expressions with the use of constructs such as `do { /* ... */} while(0)` block.
543 ]
544
545
546 The example below checks that the exception carries the proper error code.
547
548 [bt_example exception_check_predicate..BOOST_<level>_EXCEPTION usage..run-fail]
549
550 See also:
551
552 * __BOOST_LEVEL_THROW__
553 * [link boost_test.testing_tools.exception_correctness Exception correctness] section
554
555 [endsect]
556
557 [/ ###############################################################################################]
558 [section:assertion_boost_level_small `BOOST_<level>_SMALL`]
559
560 ``
561 BOOST_WARN_SMALL(value, tolerance);
562 BOOST_CHECK_SMALL(value, tolerance);
563 BOOST_REQUIRE_SMALL(value, tolerance);
564 ``
565
566 These tools are used to check that supplied value is small enough. The "smallness" is defined by absolute value
567 of the tolerance supplied as a second argument. Use these tools with caution. To compare to values on closeness
568 it's preferable to use __BOOST_LEVEL_CLOSE__ tools instead.
569
570 The first parameter is the value to check. The second parameter is the tolerance.
571
572 [note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`.]
573
574 [bt_example example41..BOOST_<level>_SMALL usage..run-fail]
575
576 See also:
577
578 * __BOOST_LEVEL_CLOSE__
579 * __BOOST_LEVEL_CLOSE_FRACTION__
580 * __floating_points_testing_tools__
581
582 [endsect]
583
584
585
586 [/ ###############################################################################################]
587 [section:test_org_boost_test_case_expected_failure `BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES`]
588 Indicates the number of failures for a test case.
589
590 See [link boost_test.testing_tools.expected_failures here] for more details.
591 [endsect] [/ expected failures]
592
593 [/ ###############################################################################################]
594 [section:assertion_boost_error `BOOST_ERROR`]
595
596 ``
597 BOOST_ERROR(message);
598 ``
599
600 __BOOST_ERROR__ tool behave the same way as `__BOOST_TEST__(false, message)`. This tool is used for
601 an unconditional error counter increasing and message logging.
602
603 The tool's only parameter is an error message to log.
604
605 [bt_example example46..BOOST_ERROR usage..run-fail]
606
607 See also:
608
609 * __BOOST_TEST__
610
611 [endsect]
612
613
614 [/ ###############################################################################################]
615 [section:assertion_boost_fail `BOOST_FAIL`]
616
617 ``
618 BOOST_FAIL(message);
619 ``
620
621 `__BOOST_FAIL__(message)` behave the same way as `__BOOST_TEST_REQUIRE__(false, message)`. This tool is used for an
622 unconditional error counter increasing, message logging and the current test case aborting.
623
624 The tool's only parameter is an error message to log.
625
626 [bt_example example47..BOOST_FAIL usage..run-fail]
627
628 See also:
629
630 * __BOOST_TEST__
631 *
632
633 [endsect]
634
635
636 [/ ###############################################################################################]
637 [section:assertion_boost_is_defined `BOOST_IS_DEFINED`]
638
639 ``
640 BOOST_IS_DEFINED(symbol);
641 ``
642
643 Unlike the rest of the tools in the toolbox this tool does not perform the logging itself. Its only purpose
644 is to check at runtime whether or not the supplied preprocessor symbol is defined. Use it in combination with
645 __BOOST_LEVEL__ to perform and log validation. Macros of any arity could be checked. To check the
646 macro definition with non-zero arity specify dummy arguments for it. See below for example.
647
648 The only tool's parameter is a preprocessor symbol that gets validated.
649
650 [bt_example example48..BOOST_IS_DEFINED usage..run-fail]
651
652 See also:
653
654 * __BOOST_LEVEL__
655
656 [endsect]
657
658 [/ ###############################################################################################]
659 [section:assertion_control_under_debugger `BOOST_TEST_TOOLS_UNDER_DEBUGGER`]
660 When defined, assertions evaluate their expression eagerly, as described [link boost_test.testing_tools.debugging here].
661 [endsect] [/ assertion_control_under_debugger]
662
663 [/ ###############################################################################################]
664 [section:assertion_control_under_debuggable `BOOST_TEST_TOOLS_DEBUGGABLE`]
665 When defined, test assertions are compiled in two modes (debugger-friendly and full-featured) and the version is selected at run-time, as described [link boost_test.testing_tools.debugging here].
666 [endsect] [/ assertion_control_under_debuggable]
667
668 [endsect] [/ testing_tool_ref]