ceph/src/test/rgw/test_multi.md

   1 # Multi Site Test Framework
   2 This framework allows you to write and run tests against a **local** multi-cluster environment. The framework is using the `mstart.sh` script in order to setup the environment according to a configuration file, and then uses the [nose](https://nose.readthedocs.io/en/latest/) test framework to actually run the tests.
   3 Tests are written in python2.7, but can invoke shell scripts, binaries etc.
   4 ## Running Tests
   5 Entry point for all tests is `/path/to/ceph/src/test/rgw/test_multi.py`. And the actual tests are located inside the `/path/to/ceph/src/test/rgw/rgw_multi` subdirectory.
   6 So, to run all tests use:
   7 ```
   8 $ cd /path/to/ceph/src/test/rgw/
   9 $ nosetests test_multi.py
  10 ```
  11 This will assume a configuration file called `/path/to/ceph/src/test/rgw/test_multi.conf` exists.
  12 To use a different configuration file, set the `RGW_MULTI_TEST_CONF` environment variable to point to that file.
  13 Since we use the same entry point file for all tests, running specific tests is possible using the following format:
  14 ```
  15 $ nosetests test_multi.py:<specific_test_name>
  16 ```
  17 To run miltiple tests based on wildcard string, use the following format:
  18 ```
  19 $ nosetests test_multi.py -m "<wildcard string>"
  20 ```
  21 Note that the test to run, does not have to be inside the `test_multi.py` file.
  22 Note that different options for running specific and multiple tests exists in the [nose documentation](https://nose.readthedocs.io/en/latest/usage.html#options), as well as other options to control the execution of the tests.
  23 ## Configuration
  24 ### Environment Variables
  25 Following RGW environment variables are taken into consideration when running the tests:
  26  - `RGW_FRONTEND`: used to change frontend to 'civetweb' or 'beast' (default)
  27  - `RGW_VALGRIND`: used to run the radosgw under valgrind. e.g. RGW_VALGRIND=yes
  28 Other environment variables used to configure elements other than RGW can also be used as they are used in vstart.sh. E.g. MON, OSD, MGR, MSD
  29 The configuration file for the run has 3 sections:
  30 ### Default
  31 This section holds the following parameters:
  32  - `num_zonegroups`: number of zone groups (integer, default 1)
  33  - `num_zones`: number of regular zones in each group (integer, default 3)
  34  - `num_ps_zones`: number of pubsub zones in each group (integer, default 0)
  35  - `num_az_zones`: number of archive zones (integer, default 0, max value 1)
  36  - `gateways_per_zone`: number of RADOS gateways per zone (integer, default 2)
  37  - `no_bootstrap`: whether to assume that the cluster is already up and does not need to be setup again. If set to "false", it will try to re-run the cluster, so, `mstop.sh` must be called beforehand. Should be set to false, anytime the configuration is changed. Otherwise, and assuming the cluster is already up, it should be set to "true" to save on execution time (boolean, default false)
  38  - `log_level`: console log level of the logs in the tests, note that any program invoked from the test my emit logs regardless of that setting (integer, default 20)
  39     - 20 and up -> DEBUG
  40     - 10 and up -> INFO
  41     - 5 and up -> WARNING
  42     - 1 and up -> ERROR
  43     - CRITICAL is always logged
  44 - `log_file`: log file name. If not set, only console logs exists (string, default None)
  45 - `file_log_level`: file log level of the logs in the tests. Similar to `log_level`
  46 - `tenant`: name of tenant (string, default None)
  47 - `checkpoint_retries`: *TODO* (integer, default 60)
  48 - `checkpoint_delay`: *TODO* (integer, default 5)
  49 - `reconfigure_delay`: *TODO* (integer, default 5)
  50 ### Elasticsearch
  51 *TODO*
  52 ### Cloud
  53 *TODO*
  54 ### PubSub
  55 *TODO*
  56 ## Writing Tests
  57 New tests should be added into the `/path/to/ceph/src/test/rgw/rgw_multi` subdirectory.
  58 - Base classes are in: `/path/to/ceph/src/test/rgw/rgw_multi/multisite.py`
  59 - `/path/to/ceph/src/test/rgw/rgw_multi/tests.py` holds the majority of the tests, but also many utility and infrastructure functions that could be used in other tests files