ceph/src/boost/libs/test/doc/test_organization/test_suites.qbk

   1 [/
   2  / Copyright (c) 2003 Boost.Test contributors
   3  /
   4  / Distributed under the Boost Software License, Version 1.0. (See accompanying
   5  / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
   6  /]
   7
   8 [section:test_suite Test suite]
   9 If you consider test cases as leaves on the test tree, the test suite can be considered as branch and the master
  10 test suite as a root. Unlike real trees though, our tree in many cases consists only of leaves attached
  11 directly to the root. This is common for all test cases to reside directly in the master test suite. If you do
  12 want to construct a hierarchical test suite structure the __UTF__ provides both manual and automated
  13 test suite creation and registration facilities:
  14
  15 # Test suite with automated registration
  16 # Manually registered test suite
  17
  18 In addition the __UTF__ presents a notion of the
  19 [link boost_test.tests_organization.test_suite.master_test_suite Master Test Suite].
  20 The most important reason to learn about this component is that it provides an ability to access
  21 command line arguments supplied to a test module.
  22
  23 [#ref_BOOST_AUTO_TEST_SUITE][h3 Automated registration]
  24 The solution the __UTF__ presents for automated test suite creation and registration is designed to facilitate
  25 multiple points of definition, arbitrary test suites depth and smooth integration with automated test case creation
  26 and registration. This facility should significantly simplify a test tree construction process in comparison with
  27 manual explicit registration case.
  28
  29
  30 The implementation is based on the order of file scope variables definitions within a single compilation unit.
  31 The semantic of this facility is very similar to the namespace feature of C++, including support for test suite
  32 extension. To start test suite use the macro __BOOST_AUTO_TEST_SUITE__. To end test suite use the macro
  33 __BOOST_AUTO_TEST_SUITE_END__. The same test suite can be restarted multiple times inside the same test file or in a
  34 different test files. In a result all test units will be part of the same test suite in a constructed test tree.
  35
  36 ``
  37   BOOST_AUTO_TEST_SUITE(test_suite_name);
  38   BOOST_AUTO_TEST_SUITE_END();
  39 ``
  40
  41 Test units defined in between test suite start and end declarations become members of the test suite. A test
  42 unit always becomes the member of the closest test suite declared. Test units declared at a test file scope
  43 become members of the master test suite. There is no limit on depth of test suite inclusion.
  44
  45
  46 This example creates a test tree that matches exactly the one created in the manual test suite registration
  47 example.
  48
  49 [bt_example example12..Test suites with automated registration..run-fail]
  50
  51 As you can see test tree construction in this example is more straightforward and automated.
  52
  53 In the example below, the test suite `test_suite` consists of two parts. Their definition is remote and is separated by another
  54 test case. In fact these parts may even reside in different test files. The resulting test tree remains the same. As
  55 you can see from the output both `test_case1` and `test_case2` reside in the same test suite `test_suite`.
  56
  57 [bt_example example53..Test suite extension using automated registration facility..run-fail]
  58
  59 [h3 Test suites with manual registration]
  60 To create a test suite manually you need to
  61
  62 # create an instance of [classref boost::unit_test::test_suite] class,
  63 # register it in test tree, and
  64 # populate it with test cases (or lower level test suites).
  65
  66 [#ref_test_case_registration][h4 Test unit registration interface]
  67
  68
  69 The __UTF__ models the notion of test case container - test suite - using class [classref boost::unit_test::test_suite]. For
  70 complete class interface reference check advanced section of this documentation. Here you should only be
  71 interested in a single test unit registration interface:
  72
  73 ``
  74   void test_suite::add( test_unit* tc, counter_t expected_failures = 0, int timeout = 0 );
  75 ``
  76
  77 The first parameter is a pointer to a newly created test unit. The second optional parameter -
  78 expected_failures  - defines the number of test assertions that are expected to fail within the test unit. By
  79 default no errors are expected.
  80
  81 [caution
  82   Be careful when supplying a number of expected failures for test suites. By default the __UTF__ calculates the
  83   number of expected failures in test suite as the sum of appropriate values in all test units that constitute
  84   it. And it rarely makes sense to change this.
  85 ]
  86
  87 The third optional parameter - `timeout` - defines the timeout value for the test unit. As of now the __UTF__
  88 isn't able to set a timeout for the test suite execution, so this parameter makes sense only for test case
  89 registration. By default no timeout is set. See the method
  90 [memberref boost::execution_monitor::execute] for more details about the timeout value. [warning is the reference
  91 good? It looks to me that [memberref boost::unit_test::test_suite::add] is better]
  92
  93 To register group of test units in one function call, the [classref boost::unit_test::test_suite `test_suite`] class provides another
  94 [memberref boost::unit_test::test_suite::add `add`] interface covered in the advanced section of this documentation.
  95
  96
  97 [#ref_BOOST_TEST_SUITE][h4 Test suite instance construction]
  98
  99
 100 To create a test suite instance manually, employ the macro __BOOST_TEST_SUITE__. It hides all implementation
 101 details and you only required to specify the test suite name:
 102
 103 ``
 104   BOOST_TEST_SUITE(test_suite_name);
 105 ``
 106
 107 __BOOST_TEST_SUITE__ creates an instance of the class `boost::unit_test::test_suite` and returns a pointer to the
 108 constructed instance. Alternatively you can create an instance of class `boost::unit_test::test_suite` yourself.
 109
 110 [caution `boost::unit_test::test_suite` instances have to be allocated on the heap and the compiler won't allow you
 111       to create one on the stack.
 112 ]
 113
 114 Newly created test suite has to be registered in a parent one using add interface. Both test suite creation and
 115 registration is performed in the test module initialization function.
 116
 117 The example below creates a test tree, which can be represented by the following hierarchy:
 118
 119 [$images/class-hier.jpg]
 120
 121 [bt_example example11..Manually registered test suites..run]
 122
 123 [section:master_test_suite Master Test Suite]
 124
 125
 126 As defined in introduction section the master test suite is a root node of a test tree. Each test module built
 127 with the __UTF__ always has the master test suite defined. The __UTF__ maintain the master test suite instance
 128 internally. All other test units are registered as direct or indirect children of the master test suite.
 129
 130 ``
 131 namespace boost {
 132 namespace unit_test {
 133 class master_test_suite_t : public test_suite
 134 {
 135 public:
 136   int argc;
 137   char** argv;
 138 };
 139
 140 } // namespace unit_test
 141 } // namespace boost
 142 ``
 143
 144
 145 To access single instance of the master test suite use the following interface:
 146
 147 ``
 148 namespace boost {
 149 namespace unit_test {
 150 namespace framework {
 151
 152 master_test_suite_t& master_test_suite();
 153
 154 } // namespace framework
 155 } // namespace unit_test
 156 } // namespace boost
 157 ``
 158
 159 [h4 Command line arguments access interface]
 160
 161 Master test suite implemented as an extension to the regular test suite, since it maintains references to the
 162 command line arguments passed to the test module. To access the command line arguments use
 163
 164 ``
 165 boost::unit_test::framework::master_test_suite().argc
 166 boost::unit_test::framework::master_test_suite().argv
 167 ``
 168
 169 In below example references to the command line arguments are accessible either as an initialization function
 170 parameters or as members of the master test suite. Both references point to the same values. A test module that
 171 uses the alternative initialization function specification can only access command line arguments through the
 172 master test suite.
 173
 174
 175 Returning to the free function example, let's modify initialization function to check for absence of any
 176 test module arguments.
 177
 178 [bt_example example13..Command line access in initialization function..run]
 179
 180 [#ref_BOOST_TEST_MODULE][h4 Naming the ['Master test suite]]
 181
 182 The master test suite is created with default name ['Master Test Suite]. There are two methods two
 183 reset the name to a different value: using the macro __BOOST_TEST_MODULE__
 184 and from within the test module initialization function. Former is used for test modules that don't have the
 185 manually implemented initialization function. Following examples illustrate these methods.
 186
 187 [bt_example example14..Naming master test suite using the macro __BOOST_TEST_MODULE__..run]
 188
 189 If the macro __BOOST_TEST_MODULE__ is defined, the test module initialization
 190 function is [*automatically generated] and the
 191 macro value becomes the name of the master test suite. The name may include spaces.
 192
 193 [bt_example example15..Naming master test suite explicitly in the test module initialization function..run]
 194
 195 Without the __BOOST_TEST_MAIN__ and the __BOOST_TEST_MODULE__ flags defined, the test module initialization
 196 function has to be manually implemented. The master test suite name can be reset at any point within this function.
 197
 198 [endsect] [/ command line interface]
 199
 200 [endsect] [/ test suite]
 201
 202 [/EOF]