doc/developer/topotests-markers.rst

   1 .. _topotests-markers:
   2
   3 Markers
   4 --------
   5
   6 To allow for automated selective testing on large scale continuous integration
   7 systems, all tests must be marked with at least one of the following markers:
   8
   9 * babeld
  10 * bfdd
  11 * bgpd
  12 * eigrpd
  13 * isisd
  14 * ldpd
  15 * nhrpd
  16 * ospf6d
  17 * ospfd
  18 * pathd
  19 * pbrd
  20 * pimd
  21 * ripd
  22 * ripngd
  23 * sharpd
  24 * staticd
  25 * vrrpd
  26
  27 The markers corespond to the daemon subdirectories in FRR's source code and have
  28 to be added to tests on a module level depending on which daemons are used
  29 during the test.
  30
  31 The goal is to have continuous integration systems scan code submissions, detect
  32 changes to files in a daemons subdirectory and select only tests using that
  33 daemon to run to shorten developers waiting times for test results and save test
  34 infrastructure resources.
  35
  36 Newly written modules and code changes on tests, which do not contain any or
  37 incorrect markers will be rejected by reviewers.
  38
  39
  40 Registering markers
  41 ^^^^^^^^^^^^^^^^^^^
  42 The Registration of new markers takes place in the file
  43 ``tests/topotests/pytest.ini``:
  44
  45 .. code:: python3
  46
  47     # tests/topotests/pytest.ini
  48     [pytest]
  49     ...
  50     markers =
  51         babeld: Tests that run against BABELD
  52         bfdd: Tests that run against BFDD
  53         ...
  54         vrrpd: Tests that run against VRRPD
  55
  56
  57 Adding markers to tests
  58 ^^^^^^^^^^^^^^^^^^^^^^^
  59 Markers are added to a test by placing a global variable in the test module.
  60
  61 Adding a single marker:
  62
  63 .. code:: python3
  64
  65     import pytest
  66     ...
  67
  68     # add after imports, before defining classes or functions:
  69     pytestmark = pytest.mark.bfdd
  70
  71     ...
  72
  73     def test_using_bfdd():
  74
  75
  76 Adding multiple markers:
  77
  78 .. code:: python3
  79
  80     import pytest
  81     ...
  82
  83     # add after imports, before defining classes or functions:
  84     pytestmark = [
  85         pytest.mark.bgpd,
  86         pytest.mark.ospfd,
  87         pytest.mark.ospf6d
  88     ]
  89
  90     ...
  91
  92     def test_using_bgpd_ospfd_ospf6d():
  93
  94
  95 Selecting marked modules for testing
  96 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  97 Selecting by a single marker:
  98
  99 .. code:: bash
 100
 101     pytest -v -m isisd
 102
 103 Selecting by multiple markers:
 104
 105 .. code:: bash
 106
 107     pytest -v -m "isisd or ldpd or nhrpd"
 108
 109
 110 Further Information
 111 ^^^^^^^^^^^^^^^^^^^
 112 The `online pytest documentation <https://docs.pytest.org/en/stable/example/markers.html>`_
 113 provides further information and usage examples for pytest markers.
 114