ceph/qa/README

   1 ceph-qa-suite
   2 -------------
   3
   4 clusters/    - some predefined cluster layouts
   5 suites/      - set suite
   6
   7 The suites directory has a hierarchical collection of tests.  This can be
   8 freeform, but generally follows the convention of
   9
  10   suites/<test suite name>/<test group>/...
  11
  12 A test is described by a yaml fragment.
  13
  14 A test can exist as a single .yaml file in the directory tree.  For example:
  15
  16  suites/foo/one.yaml
  17  suites/foo/two.yaml
  18
  19 is a simple group of two tests.
  20
  21 A directory with a magic '+' file represents a test that combines all
  22 other items in the directory into a single yaml fragment.  For example:
  23
  24  suites/foo/bar/+
  25  suites/foo/bar/a.yaml
  26  suites/foo/bar/b.yaml
  27  suites/foo/bar/c.yaml
  28
  29 is a single test consisting of a + b + c.
  30
  31 A directory with a magic '%' file represents a test matrix formed from
  32 all other items in the directory.  For example,
  33
  34  suites/baz/%
  35  suites/baz/a.yaml
  36  suites/baz/b/b1.yaml
  37  suites/baz/b/b2.yaml
  38  suites/baz/c.yaml
  39  suites/baz/d/d1.yaml
  40  suites/baz/d/d2.yaml
  41
  42 is a 4-dimensional test matrix.  Two dimensions (a, c) are trivial (1
  43 item), so this is really 2x2 = 4 tests, which are
  44
  45   a + b1 + c + d1
  46   a + b1 + c + d2
  47   a + b2 + c + d1
  48   a + b2 + c + d2
  49
  50 A directory with a magic '$' file, or a directory whose name ends with '$',
  51 represents a test where one of the non-magic items is chosen randomly.  For
  52 example, both
  53
  54  suites/foo/$
  55  suites/foo/a.yaml
  56  suites/foo/b.yaml
  57  suites/foo/c.yaml
  58
  59 and
  60
  61  suites/foo$/a.yaml
  62  suites/foo$/b.yaml
  63  suites/foo$/c.yaml
  64
  65 is a single test, either a, b or c.  This can be used in conjunction with the
  66 '%' file in the same (see below) or other directories to run a series of tests
  67 without causing an unwanted increase in the total number of jobs run.
  68
  69 Symlinks are okay.
  70
  71 One particular use of symlinks is to combine '%' and the latter form of '$'
  72 feature.  Consider supported_distros directory containing fragments that define
  73 os_type and os_version:
  74
  75  supported_distros/%
  76  supported_distros/centos.yaml
  77  supported_distros/rhel.yaml
  78  supported_distros/ubuntu.yaml
  79
  80 A test that links supported_distros as distros (a name that doesn't end with
  81 '$') will be run three times: on centos, rhel and ubuntu.  A test that links
  82 supported_distros as distros$ will be run just once: either on centos, rhel or
  83 ubuntu, chosen randomly.
  84
  85 The teuthology code can be found in https://github.com/ceph/teuthology.git