2 / Copyright (c) 2003 Boost.Test contributors
4 / Distributed under the Boost Software License, Version 1.0. (See accompanying
5 / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
9 [section:testing_tool_ref Reference API for writing tests]
13 [/ ###############################################################################################]
14 [section:assertion_boost_test_universal_macro `BOOST_TEST`]
18 BOOST_TEST(statement);
19 BOOST_TEST_<level>(statement);
21 // replacement failure message, requires variadic macros
22 BOOST_TEST(statement, "failure message");
24 // Floating point comparison, requires variadic macros, auto and decltype
25 BOOST_TEST(statement, floating_point_comparison_manipulation);
27 // bitwise comparison, requires variadic macros, auto and decltype
28 BOOST_TEST(statement, boost::test_tools::bitwise() );
30 // element-wise comparison, for containers
31 BOOST_TEST(statement, boost::test_tools::per_element() );
33 // lexicographic comparison, for containers
34 BOOST_TEST(statement, boost::test_tools::lexicographic() );
37 The full documentation of this macro is located [link boost_test.testing_tools.boost_test_universal_macro here].
39 The macro is available in three variants, corresponding to different [link boost_test.testing_tools.tools_assertion_severity_level assertion severity levels]:
42 BOOST_TEST // or BOOST_TEST_CHECK
48 * `"failure message"` is a C-string printed in case of failure in place of the default message.
49 See [link boost_test.testing_tools.reports this section] for
51 * `floating_point_comparison_manipulation` is one of the floating point comparison manipulators.
52 See [link boost_test.testing_tools.boost_test_universal_macro this section]
54 * [classref boost::test_tools::bitwise] is a manipulator indicating that the comparison should be performed bitwise. See
55 [link boost_test.testing_tools.extended_comparison.bitwise this section] for more details
56 * [classref boost::test_tools::per_element] is a manipulator indicating that the comparison should be performed on each element, in sequence, rather
57 than on containers. See
58 [link boost_test_coll_perelement this section] for more details
59 * [classref boost::test_tools::lexicographic] is a manipulator indicating that the comparison should be performed with the lexicographic order. See
60 [link boost_test_coll_default_lex this section] for more details
62 [h3 Limitations and workaround]
63 There are some restrictions on the statements that are supported by this tool. Those are explained in details in
64 [link boost_test_statement_limitations this] section.
70 [/ DECORATORS ###############################################################################################]
71 [/-----------------------------------------------------------------]
73 [section:decorator_expected_failures expected_failures (decorator)]
76 expected_failures(counter_t number);
79 Indicates the expected failures for a test unit.
80 See [link boost_test.testing_tools.expected_failures here] for more details.
82 [endsect] [/ section expected_failures]
85 [/-----------------------------------------------------------------]
86 [section:decorator_timeout timeout (decorator)]
89 timeout(unsigned seconds);
92 Specifies a time-out for a *test-case*, above which the test-case is forced to stop and reported as failing.
93 See [link boost_test.testing_tools.timeout here] for more details.
95 [endsect] [/ section timeout]
98 [/-----------------------------------------------------------------]
99 [section:decorator_tolerance tolerance (decorator)]
102 template <typename FPT>
105 template <typename FPT>
106 tolerance(test_tools::fpc::percent_tolerance_t<FPT> eps)
109 Decorator `tolerance` specifies the default comparison tolerance for floating point type `FTP` in the decorated test
110 unit. The default tolerance only applies to a particular type, so it makes sense to provide more than one `tolerance`
111 decorator if we are comparing different floating point types. For more details see __floating_points_testing_impl__.
112 The variant with `percent_tolerance` uses value `eps / 100` as tolerance.
114 [bt_example decorator_13..decorator tolerance..run-fail]
116 In the above example, in `test1`, checks on `double`s fail because they differ by more what tolerance for `double`s
117 specifies. In `test2` the tolerance for `double`s is greater and therefore the checks succeed. In `test3`, we specify
118 only tolerance for type `float`, and since the checks use type `double` the specified tolerance does not apply. Tolerance
119 in `test4` is equivalent to that in `test1`, therefore its checks also fail. Tolerance in `test5` is equivalent to
120 that in `test2`, therefore its checks also succeed.
121 [endsect] [/ section decorator_tolerance]
128 [/ ###############################################################################################]
129 [#ref_BOOST_level][section:assertion_boost_level `BOOST_<level>`]
133 BOOST_WARN(predicate);
134 BOOST_CHECK(predicate);
135 BOOST_REQUIRE(predicate);
138 These tools are used to validate the predicate value. The only parameter for these tools is a boolean predicate
139 value that gets validated. It could be any expression that could be evaluated and converted to boolean value. The
140 expression gets evaluated only once, so it's safe to pass complex expression for validation.
142 [bt_example example34..BOOST_<level> usage..run-fail]
146 * __BOOST_LEVEL_MESSAGE__
151 [/ ###############################################################################################]
152 [section:assertion_boost_level_bitwise_eq `BOOST_<level>_BITWISE_EQUAL`]
156 BOOST_WARN_BITWISE_EQUAL(left, right);
157 BOOST_CHECK_BITWISE_EQUAL(left, right);
158 BOOST_REQUIRE_BITWISE_EQUAL(left, right);
161 These tools are used to perform bitwise comparison of two values. The check shows all positions where left and
162 right value's bits mismatch.
164 The first parameter is the left compared value. The second parameter is the right compared value. Parameters are
165 not required to be of the same type, but warning is issued if their type's size does not coincide.
167 [bt_example example33..BOOST_<level>_BITWISE_EQUAL usage..run-fail]
171 * __BOOST_LEVEL_EQUAL__
175 [/ ###############################################################################################]
176 [section:assertion_boost_level_eq `BOOST_<level>_EQUAL`]
179 BOOST_WARN_EQUAL(left, right);
180 BOOST_CHECK_EQUAL(left, right);
181 BOOST_REQUIRE_EQUAL(left, right);
184 Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__(left == right)`.
185 The difference is that the mismatched values are reported as well.
187 [note It is bad idea to use these tools to compare floating point values. Use __BOOST_LEVEL_CLOSE__ or
188 __BOOST_LEVEL_CLOSE_FRACTION__ tools instead.
191 [bt_example example35..BOOST_<level>_EQUAL usage..run-fail]
196 * __BOOST_LEVEL_CLOSE__
198 * __BOOST_LEVEL_EQUAL_COLLECTIONS__
202 [/ ###############################################################################################]
203 [section:assertion_boost_level_eq_collections `BOOST_<level>_EQUAL_COLLECTIONS`]
206 BOOST_WARN_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
207 BOOST_CHECK_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
208 BOOST_REQUIRE_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
211 These tools are used to perform an element by element comparison of two collections. They print all mismatched
212 positions, collection elements at these positions and check that the collections have the same size. The first two
213 parameters designate begin and end of the first collection. The two last parameters designate begin and end of the
216 [bt_example example36..BOOST_<level>_EQUAL_COLLECTIONS usage..run-fail]
220 * __BOOST_LEVEL_EQUAL__
224 [/ ###############################################################################################]
225 [section:assertion_boost_level_close `BOOST_<level>_CLOSE`]
228 BOOST_WARN_CLOSE(left, right, tolerance);
229 BOOST_CHECK_CLOSE(left, right, tolerance);
230 BOOST_REQUIRE_CLOSE(left, right, tolerance);
233 These tools are used to check on closeness using strong relationship defined by the predicate
234 ``check_is_close( left, right, tolerance )``
236 To check for the weak relationship use
237 __BOOST_LEVEL_PREDICATE__ family of tools with explicit `check_is_close` invocation.
240 The first parameter is the ['left] compared value. The second parameter is the
241 ['right] compared value. Last third parameter defines the tolerance for the comparison in
242 [link boost_test.testing_tools.extended_comparison.floating_point [*percentage units]].
244 [note It is required for left and right parameters to be of the same floating point type. You will need to explicitly
245 resolve any type mismatch to select which type to use for comparison.
248 [note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`.
251 [bt_example example42..BOOST_<level>_CLOSE usage with small values..run-fail]
252 [bt_example example43..BOOST_<level>_CLOSE usage with big values..run]
256 * __BOOST_LEVEL_CLOSE_FRACTION__
257 * __BOOST_LEVEL_SMALL__
258 * __BOOST_LEVEL_EQUAL__
259 * __floating_points_testing_tools__
263 [/ ###############################################################################################]
264 [section:assertion_boost_level_close_fraction `BOOST_<level>_CLOSE_FRACTION`]
267 BOOST_WARN_CLOSE_FRACTION(left, right, tolerance);
268 BOOST_CHECK_CLOSE_FRACTION(left, right, tolerance);
269 BOOST_REQUIRE_CLOSE_FRACTION(left, right, tolerance);
272 These tools are used to check on closeness using strong relationship defined by the predicate
273 ``check_is_close(left, right, tolerance)``
275 To check for the weak relationship use __BOOST_LEVEL_PREDICATE__ family of tools with explicit `check_is_close` invocation.
277 The first parameter is the ['left] compared value. The second parameter is the
278 ['right] compared value. Last third parameter defines the tolerance for the comparison as
279 [link boost_test.testing_tools.extended_comparison.floating_point [*fraction of absolute values being compared]].
281 [note It is required for left and right parameters to be of the same floating point type. You will need to explicitly
282 resolve any type mismatch to select which type to use for comparison.]
284 [note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`.]
286 [bt_example example44..BOOST_<level>_CLOSE_FRACTION usage..run-fail]
290 * __BOOST_LEVEL_CLOSE__
291 * __BOOST_LEVEL_SMALL__
292 * __BOOST_LEVEL_EQUAL__
293 * __floating_points_testing_tools__
297 [/ ###############################################################################################]
298 [section:assertion_boost_level_ge `BOOST_<level>_GE`]
301 BOOST_WARN_GE(left, right);
302 BOOST_CHECK_GE(left, right);
303 BOOST_REQUIRE_GE(left, right);
306 Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left >= right )`.
307 The difference is that the argument values are reported as well.
309 [bt_example example57..BOOST_<level>_GE usage..run-fail]
320 [/ ###############################################################################################]
321 [section:assertion_boost_level_gt `BOOST_<level>_GT`]
325 BOOST_WARN_GT(left, right);
326 BOOST_CHECK_GT(left, right);
327 BOOST_REQUIRE_GT(left, right);
330 Check performed by these tools is the same as the one performed by __BOOST_LEVEL__`( left > right )`.
331 The difference is that the argument values are reported as well.
333 [bt_example example58..BOOST_<level>_GT usage..run-fail]
343 [/ ###############################################################################################]
344 [section:assertion_boost_level_le `BOOST_<level>_LE`]
347 BOOST_WARN_LE(left, right);
348 BOOST_CHECK_LE(left, right);
349 BOOST_REQUIRE_LE(left, right);
352 Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left <= right )`.
353 The difference is that the argument values are reported as well.
355 [bt_example example55..BOOST_<level>_LE usage..run-fail]
365 [/ ###############################################################################################]
366 [section:assertion_boost_level_lt `BOOST_<level>_LT`]
369 BOOST_WARN_LT(left, right);
370 BOOST_CHECK_LT(left, right);
371 BOOST_REQUIRE_LT(left, right);
374 Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left < right )`.
375 The difference is that the argument values are reported as well.
377 [bt_example example56..BOOST_<level>_LT usage..run-fail]
387 [/ ###############################################################################################]
388 [section:assertion_boost_level_message `BOOST_<level>_MESSAGE`]
391 BOOST_WARN_MESSAGE(predicate, message);
392 BOOST_CHECK_MESSAGE(predicate, message);
393 BOOST_REQUIRE_MESSAGE(predicate, message);
396 These tools perform exactly the same check as __BOOST_LEVEL__ tools. The only difference is that
397 instead of generating an error/confirm message these use the supplied one.
399 The first parameter is the boolean expression. The second parameter is the message reported in case of check
400 failure. The message argument can be constructed of components of any type supporting the
401 `std::ostream& operator<<(std::ostream&)`.
403 [bt_example example38..BOOST_<level>_MESSAGE usage..run]
412 [/ ###############################################################################################]
413 [section:assertion_boost_level_ne `BOOST_<level>_NE`]
417 BOOST_WARN_NE(left, right);
418 BOOST_CHECK_NE(left, right);
419 BOOST_REQUIRE_NE(left, right);
422 Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left != right )`.
423 The difference is that the matched values are reported as well.
425 [bt_example example54..BOOST_<level>_NE usage..run-fail]
429 * __BOOST_LEVEL_EQUAL__
433 [/ ###############################################################################################]
434 [section:assertion_boost_level_predicate `BOOST_<level>_PREDICATE`]
438 BOOST_WARN_PREDICATE(predicate, arguments_list);
439 BOOST_CHECK_PREDICATE(predicate, arguments_list);
440 BOOST_REQUIRE_PREDICATE(predicate, arguments_list);
443 These are generic tools used to validate an arbitrary supplied predicate functor (there is a compile time limit on
444 predicate arity defined by the configurable macro `BOOST_TEST_MAX_PREDICATE_ARITY`). To
445 validate zero arity predicate use __BOOST_LEVEL__ tools. In other cases prefer theses tools. The
446 advantage of these tools is that they show arguments values in case of predicate failure.
448 The first parameter is the predicate itself. The second parameter is the list of predicate arguments each wrapped
449 in round brackets (`BOOST_PP` sequence format).
451 [bt_example example40..BOOST_<level>_PREDICATE usage..run]
453 [note Note difference in error log from __BOOST_LEVEL__]
461 [/ ###############################################################################################]
462 [section:assertion_boost_level_no_throw `BOOST_<level>_NO_THROW`]
465 BOOST_WARN_NO_THROW(expression);
466 BOOST_CHECK_NO_THROW(expression);
467 BOOST_REQUIRE_NO_THROW(expression);
470 These assertions validate that the execution of `expression` does not throw any exception.
471 To that extent, all possible exception are caught by assertion itself and no exception is propagated to
475 It is possible to test for complex expressions with the use of constructs such as `do { /* ... */} while(0)` block.
478 [bt_example exception_nothrow..BOOST_<level>_NO_THROW usage..run-fail]
482 * __BOOST_LEVEL_THROW__
483 * [link boost_test.testing_tools.exception_correctness Exception correctness] section
487 [/ ###############################################################################################]
488 [section:assertion_boost_level_throw `BOOST_<level>_THROW`]
491 BOOST_WARN_THROW(expression, exception_type);
492 BOOST_CHECK_THROW(expression, exception_type);
493 BOOST_REQUIRE_THROW(expression, exception_type);
496 These assertions validate that the execution of `expression` raises an /expected/ exception, which means an exception of
497 the supplied `exception_type` type or of any child type.
499 * If `expression` raises an unexpected exception, this exception is not caught by `BOOST_<level>_THROW` assertion and
500 might propagate to the test body. If not caught at all, the framework will catch it and terminate the test case
501 with the status /failed/.
502 * If `expression` does not raise any exception, the the assertion fails.
504 [warning the assertion catches only the expected exceptions.]
507 It is possible to test for complex expressions with the use of constructs such as `do { /* ... */} while(0)` block.
510 [bt_example exception_check..BOOST_<level>_THROW usage..run-fail]
514 * __BOOST_LEVEL_NO_THROW__
515 * [link boost_test.testing_tools.exception_correctness Exception correctness] section
520 [/ ###############################################################################################]
521 [section:assertion_boost_level_exception `BOOST_<level>_EXCEPTION`]
524 BOOST_WARN_EXCEPTION(expression, exception_type, predicate);
525 BOOST_CHECK_EXCEPTION(expression, exception_type, predicate);
526 BOOST_REQUIRE_EXCEPTION(expression, exception_type, predicate);
529 As for __BOOST_LEVEL_THROW__, these assertions validate that `expression` raises an exception of the
530 type specified by `exception_type` or any of its child type, with additional checks on the exception instance.
532 * If an expected exception is raised by `expression`, the instance of the exception is passed to `predicate`
533 for further validation.
534 * It behaves like __BOOST_LEVEL_THROW__ if `expression` does not raise any exception, or an unrelated exception is raised.
536 `predicate` should be a unary function accepting an instance of `exception_type` or any of its child, and that should return
537 a boolean indicating a success (`true`) or a failure (`false`).
539 [warning the assertion catches only the expected exceptions.]
542 It is possible to test for complex expressions with the use of constructs such as `do { /* ... */} while(0)` block.
546 The example below checks that the exception carries the proper error code.
548 [bt_example exception_check_predicate..BOOST_<level>_EXCEPTION usage..run-fail]
552 * __BOOST_LEVEL_THROW__
553 * [link boost_test.testing_tools.exception_correctness Exception correctness] section
557 [/ ###############################################################################################]
558 [section:assertion_boost_level_small `BOOST_<level>_SMALL`]
561 BOOST_WARN_SMALL(value, tolerance);
562 BOOST_CHECK_SMALL(value, tolerance);
563 BOOST_REQUIRE_SMALL(value, tolerance);
566 These tools are used to check that supplied value is small enough. The "smallness" is defined by absolute value
567 of the tolerance supplied as a second argument. Use these tools with caution. To compare to values on closeness
568 it's preferable to use __BOOST_LEVEL_CLOSE__ tools instead.
570 The first parameter is the value to check. The second parameter is the tolerance.
572 [note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`.]
574 [bt_example example41..BOOST_<level>_SMALL usage..run-fail]
578 * __BOOST_LEVEL_CLOSE__
579 * __BOOST_LEVEL_CLOSE_FRACTION__
580 * __floating_points_testing_tools__
586 [/ ###############################################################################################]
587 [section:test_org_boost_test_case_expected_failure `BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES`]
588 Indicates the number of failures for a test case.
590 See [link boost_test.testing_tools.expected_failures here] for more details.
591 [endsect] [/ expected failures]
593 [/ ###############################################################################################]
594 [section:assertion_boost_error `BOOST_ERROR`]
597 BOOST_ERROR(message);
600 __BOOST_ERROR__ tool behave the same way as `__BOOST_TEST__(false, message)`. This tool is used for
601 an unconditional error counter increasing and message logging.
603 The tool's only parameter is an error message to log.
605 [bt_example example46..BOOST_ERROR usage..run-fail]
614 [/ ###############################################################################################]
615 [section:assertion_boost_fail `BOOST_FAIL`]
621 `__BOOST_FAIL__(message)` behave the same way as `__BOOST_TEST_REQUIRE__(false, message)`. This tool is used for an
622 unconditional error counter increasing, message logging and the current test case aborting.
624 The tool's only parameter is an error message to log.
626 [bt_example example47..BOOST_FAIL usage..run-fail]
636 [/ ###############################################################################################]
637 [section:assertion_boost_is_defined `BOOST_IS_DEFINED`]
640 BOOST_IS_DEFINED(symbol);
643 Unlike the rest of the tools in the toolbox this tool does not perform the logging itself. Its only purpose
644 is to check at runtime whether or not the supplied preprocessor symbol is defined. Use it in combination with
645 __BOOST_LEVEL__ to perform and log validation. Macros of any arity could be checked. To check the
646 macro definition with non-zero arity specify dummy arguments for it. See below for example.
648 The only tool's parameter is a preprocessor symbol that gets validated.
650 [bt_example example48..BOOST_IS_DEFINED usage..run-fail]
658 [/ ###############################################################################################]
659 [section:assertion_control_under_debugger `BOOST_TEST_TOOLS_UNDER_DEBUGGER`]
660 When defined, assertions evaluate their expression eagerly, as described [link boost_test.testing_tools.debugging here].
661 [endsect] [/ assertion_control_under_debugger]
663 [/ ###############################################################################################]
664 [section:assertion_control_under_debuggable `BOOST_TEST_TOOLS_DEBUGGABLE`]
665 When defined, test assertions are compiled in two modes (debugger-friendly and full-featured) and the version is selected at run-time, as described [link boost_test.testing_tools.debugging here].
666 [endsect] [/ assertion_control_under_debuggable]
668 [endsect] [/ testing_tool_ref]