tests/topotests/README.md

   1 # FRRouting Topology Tests with Mininet
   2
   3 ## Running tests with docker
   4
   5 There is a docker image which allows to run topotests. Instructions can be
   6 found [here](docker/README.md).
   7
   8 ## Guidelines
   9
  10 Instructions for  use, write or debug topologies can be found in the
  11 [guidelines](GUIDELINES.md). To learn/remember common code snippets see
  12 [here](SNIPPETS.md).
  13
  14 Before creating a new topology, make sure that there isn't one already
  15 that does what you need. If nothing is similar, then you may create a
  16 new topology, preferably, using the newest
  17 [template](example-test/test_template.py).
  18
  19 ## Installation of Mininet for running tests
  20 Only tested with Ubuntu 16.04 and Ubuntu 18.04 (which uses Mininet 2.2.x)
  21
  22 Instructions are the same for all setups (ie ExaBGP is only used for BGP
  23 tests)
  24
  25 ### Installing Mininet Infrastructure:
  26
  27 1. apt-get install mininet
  28 2. apt-get install python-pip
  29 3. apt-get install iproute
  30 4. pip install ipaddr
  31 5. pip install pytest
  32 6. pip install exabgp==3.4.17
  33    (Newer 4.0 version of exabgp is not yet supported)
  34 7. useradd -d /var/run/exabgp/ -s /bin/false exabgp
  35
  36 ### Enable Coredumps
  37 Optional, will give better output
  38
  39 1. apt-get install gdb
  40 2. disable apport (which move core files)
  41
  42         Set `enabled=0` in `/etc/default/apport`
  43
  44 3. Update security limits
  45
  46         Add/change `/etc/security/limits.conf` to
  47
  48                 #<domain>      <type>  <item>         <value>
  49                 *               soft    core          unlimited
  50                 root            soft    core          unlimited
  51                 *               hard    core          unlimited
  52                 root            hard    core          unlimited
  53
  54 4. reboot (for options to take effect)
  55
  56 ## FRRouting (FRR) Installation
  57 FRR needs to be installed separatly. It is assume to be configured
  58 like the standard Ubuntu Packages:
  59
  60 - Binaries in /usr/lib/frr
  61 - State Directory /var/run/frr
  62 - Running under user frr, group frr
  63 - vtygroup: frrvty
  64 - config directory: /etc/frr
  65 - For FRR Packages, install the dbg package as well for coredump decoding
  66
  67 No FRR config needs to be done and no FRR daemons should be run ahead
  68 of the test. They are all started as part of the test
  69
  70 #### Manual FRRouting (FRR) build
  71
  72 If you prefer to manually build FRR, then use the following suggested config:
  73
  74         ./configure \
  75                 --prefix=/usr \
  76                 --localstatedir=/var/run/frr \
  77                 --sbindir=/usr/lib/frr \
  78                 --sysconfdir=/etc/frr \
  79                 --enable-vtysh \
  80                 --enable-pimd \
  81                 --enable-multipath=64 \
  82                 --enable-user=frr \
  83                 --enable-group=frr \
  84                 --enable-vty-group=frrvty \
  85                 --with-pkg-extra-version=-my-manual-build
  86
  87 And create frr User and frrvty group as follows:
  88
  89         addgroup --system --gid 92 frr
  90         addgroup --system --gid 85 frrvty
  91         adduser --system --ingroup frr --home /var/run/frr/ \
  92            --gecos "FRRouting suite" --shell /bin/false frr
  93         usermod -G frrvty frr
  94
  95 ## Executing Tests
  96
  97 #### Execute all tests with output to console
  98
  99         py.test -s -v --tb=no
 100
 101 All test_* scripts in subdirectories are detected and executed (unless
 102 disabled in `pytest.ini` file)
 103
 104 `--tb=no` disables the python traceback which might be irrelevant unless the
 105 test script itself is debugged
 106
 107 #### Execute single test
 108
 109         cd test_to_be_run
 110         ./test_to_be_run.py
 111
 112 For further options, refer to pytest documentation
 113
 114 Test will set exit code which can be used with `git bisect`
 115
 116 For the simulated topology, see the description in the python file
 117
 118 If you need to clear the mininet setup between tests (if it isn't cleanly
 119 shutdown), then use the `mn -c` command to clean up the environment
 120
 121 #### (Optional) StdErr log from daemos after exit
 122
 123 To enable the reporting of any messages seen on StdErr after the
 124 daemons exit, the following env variable can be set.
 125
 126         export TOPOTESTS_CHECK_STDERR=Yes
 127
 128 (The value doesn't matter at this time. The check is if the env variable
 129 exists or not)
 130 There is no pass/fail on this reporting. The Output will be reported to
 131 the console
 132
 133         export TOPOTESTS_CHECK_MEMLEAK="/home/mydir/memleak_"
 134
 135 This will enable the check and output to console and the writing of
 136 the information to files with the given prefix (followed by testname),
 137 ie `/home/mydir/memcheck_test_bgp_multiview_topo1.txt` in case of a
 138 memory leak.
 139
 140 #### (Optional) Collect Memory Leak Information
 141
 142 FreeRangeRouting processes have the capabilities to report remaining memory
 143 allocations upon exit. To enable the reporting of the memory, define an
 144 enviroment variable `TOPOTESTS_CHECK_MEMLEAK` with the file prefix, ie
 145
 146         export TOPOTESTS_CHECK_MEMLEAK="/home/mydir/memleak_"
 147
 148 This will enable the check and output to console and the writing of
 149 the information to files with the given prefix (followed by testname),
 150 ie `/home/mydir/memcheck_test_bgp_multiview_topo1.txt` in case of a
 151 memory leak.
 152
 153 #### (Optional) Run topotests with GCC AddressSanitizer enabled
 154
 155 Topotests can be run with the GCC AddressSanitizer. It requires GCC 4.8 or
 156 newer. (Ubuntu 16.04 as suggested here is fine with GCC 5 as default)
 157 For more information on AddressSanitizer, see
 158 https://github.com/google/sanitizers/wiki/AddressSanitizer
 159
 160 The checks are done automatically in the library call of `checkRouterRunning`
 161 (ie at beginning of tests when there is a check for all daemons running).
 162 No changes or extra configuration for topotests is required beside compiling
 163 the suite with AddressSanitizer enabled.
 164
 165 If a daemon crashed, then the errorlog is checked for AddressSanitizer
 166 output. If found, then this is added with context (calling test) to
 167 `/tmp/AddressSanitizer.txt` in markdown compatible format.
 168
 169 Compiling for GCC AddressSanitizer requires to use gcc as a linker as well
 170 (instead of ld). Here is a suggest way to compile frr with AddressSanitizer
 171 for `stable/3.0` branch:
 172
 173         git clone https://github.com/FRRouting/frr.git
 174         cd frr
 175         git checkout stable/3.0
 176         ./bootstrap.sh
 177         export CC=gcc
 178         export CFLAGS="-O1 -g -fsanitize=address -fno-omit-frame-pointer"
 179         export LD=gcc
 180         export LDFLAGS="-g -fsanitize=address -ldl"
 181         ./configure --enable-shared=no \
 182                 --prefix=/usr/lib/frr --sysconfdir=/etc/frr \
 183                 --localstatedir=/var/run/frr \
 184                 --sbindir=/usr/lib/frr --bindir=/usr/lib/frr \
 185                 --enable-exampledir=/usr/lib/frr/examples \
 186                 --with-moduledir=/usr/lib/frr/modules \
 187                 --enable-multipath=0 --enable-rtadv \
 188                 --enable-tcp-zebra --enable-fpm --enable-pimd
 189         make
 190         sudo make install
 191         # Create symlink for vtysh, so topotest finds it in /usr/lib/frr
 192         sudo ln -s /usr/lib/frr/vtysh /usr/bin/
 193
 194 and create `frr` user and `frrvty` group as shown above
 195
 196 ## License
 197
 198 All the configs and scripts are licensed under a ISC-style license. See
 199 Python scripts for details.