]>
Commit | Line | Data |
---|---|---|
370c8e07 QY |
1 | .. _topotests: |
2 | ||
3 | Topotests | |
4 | ========= | |
5 | ||
6 | Topotests is a suite of topology tests for FRR built on top of Mininet. | |
7 | ||
8 | Installation and Setup | |
9 | ---------------------- | |
10 | ||
11 | Only tested with Ubuntu 16.04 and Ubuntu 18.04 (which uses Mininet 2.2.x). | |
12 | ||
13 | Instructions are the same for all setups (i.e. ExaBGP is only used for BGP | |
14 | tests). | |
15 | ||
16 | Installing Mininet Infrastructure | |
17 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
18 | ||
19 | .. code:: shell | |
20 | ||
21 | apt-get install mininet | |
22 | apt-get install python-pip | |
23 | apt-get install iproute | |
a9153049 | 24 | apt-get install iperf |
370c8e07 | 25 | pip install ipaddr |
fbcf1146 | 26 | pip install "pytest<5" |
bf17e2df | 27 | pip install "scapy>=2.4.2" |
370c8e07 QY |
28 | pip install exabgp==3.4.17 (Newer 4.0 version of exabgp is not yet |
29 | supported) | |
30 | useradd -d /var/run/exabgp/ -s /bin/false exabgp | |
31 | ||
32 | Enable Coredumps | |
33 | """""""""""""""" | |
34 | ||
35 | Optional, will give better output. | |
36 | ||
37 | .. code:: shell | |
38 | ||
39 | apt-get install gdb | |
40 | disable apport (which move core files) | |
41 | ||
42 | Set ``enabled=0`` in ``/etc/default/apport``. | |
43 | ||
44 | Next, update security limits by changing :file:`/etc/security/limits.conf` to:: | |
45 | ||
46 | #<domain> <type> <item> <value> | |
47 | * soft core unlimited | |
48 | root soft core unlimited | |
49 | * hard core unlimited | |
50 | root hard core unlimited | |
51 | ||
52 | Reboot for options to take effect. | |
53 | ||
b638685c PR |
54 | SNMP Utilities Installation |
55 | """"""""""""""""""""""""""" | |
56 | ||
57 | To run SNMP test you need to install SNMP utilities and MIBs. Unfortunately | |
58 | there are some errors in the upstream MIBS which need to be patched up. The | |
59 | following steps will get you there on Ubuntu 20.04. | |
60 | ||
61 | .. code:: shell | |
62 | ||
63 | apt install snmpd snmp | |
64 | apt install snmp-mibs-downloader | |
65 | download-mibs | |
66 | wget http://www.iana.org/assignments/ianaippmmetricsregistry-mib/ianaippmmetricsregistry-mib -O /usr/share/snmp/mibs/iana/IANA-IPPM-METRICS-REGISTRY-MIB | |
67 | wget http://pastebin.com/raw.php?i=p3QyuXzZ -O /usr/share/snmp/mibs/ietf/SNMPv2-PDU | |
68 | wget http://pastebin.com/raw.php?i=gG7j8nyk -O /usr/share/snmp/mibs/ietf/IPATM-IPMC-MIB | |
69 | edit /etc/snmp/snmp.conf to look like this | |
70 | # As the snmp packages come without MIB files due to license reasons, loading | |
71 | # of MIBs is disabled by default. If you added the MIBs you can reenable | |
72 | # loading them by commenting out the following line. | |
73 | mibs +ALL | |
74 | ||
75 | ||
370c8e07 QY |
76 | FRR Installation |
77 | ^^^^^^^^^^^^^^^^ | |
78 | ||
79 | FRR needs to be installed separately. It is assume to be configured like the | |
80 | standard Ubuntu Packages: | |
81 | ||
82 | - Binaries in :file:`/usr/lib/frr` | |
83 | - State Directory :file:`/var/run/frr` | |
84 | - Running under user ``frr``, group ``frr`` | |
85 | - vtygroup: ``frrvty`` | |
86 | - config directory: :file:`/etc/frr` | |
87 | - For FRR Packages, install the dbg package as well for coredump decoding | |
88 | ||
89 | No FRR config needs to be done and no FRR daemons should be run ahead of the | |
90 | test. They are all started as part of the test. | |
91 | ||
92 | Manual FRR build | |
93 | """""""""""""""" | |
94 | ||
95 | If you prefer to manually build FRR, then use the following suggested config: | |
96 | ||
97 | .. code:: shell | |
98 | ||
99 | ./configure \ | |
100 | --prefix=/usr \ | |
101 | --localstatedir=/var/run/frr \ | |
102 | --sbindir=/usr/lib/frr \ | |
103 | --sysconfdir=/etc/frr \ | |
104 | --enable-vtysh \ | |
105 | --enable-pimd \ | |
dc935a51 | 106 | --enable-sharpd \ |
370c8e07 QY |
107 | --enable-multipath=64 \ |
108 | --enable-user=frr \ | |
109 | --enable-group=frr \ | |
110 | --enable-vty-group=frrvty \ | |
b638685c | 111 | --enable-snmp=agentx \ |
370c8e07 QY |
112 | --with-pkg-extra-version=-my-manual-build |
113 | ||
114 | And create ``frr`` user and ``frrvty`` group as follows: | |
115 | ||
116 | .. code:: shell | |
117 | ||
118 | addgroup --system --gid 92 frr | |
119 | addgroup --system --gid 85 frrvty | |
120 | adduser --system --ingroup frr --home /var/run/frr/ \ | |
121 | --gecos "FRRouting suite" --shell /bin/false frr | |
122 | usermod -G frrvty frr | |
123 | ||
124 | Executing Tests | |
125 | --------------- | |
126 | ||
127 | Execute all tests with output to console | |
128 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
129 | ||
130 | .. code:: shell | |
131 | ||
132 | py.test -s -v --tb=no | |
133 | ||
44f3760c | 134 | The above command must be executed from inside the topotests directory. |
135 | ||
370c8e07 QY |
136 | All test\_\* scripts in subdirectories are detected and executed (unless |
137 | disabled in ``pytest.ini`` file). | |
138 | ||
139 | ``--tb=no`` disables the python traceback which might be irrelevant unless the | |
140 | test script itself is debugged. | |
141 | ||
142 | Execute single test | |
143 | ^^^^^^^^^^^^^^^^^^^ | |
144 | ||
145 | .. code:: shell | |
146 | ||
147 | cd test_to_be_run | |
148 | ./test_to_be_run.py | |
149 | ||
44f3760c | 150 | For example, and assuming you are inside the frr directory: |
151 | ||
152 | .. code:: shell | |
153 | ||
154 | cd tests/topotests/bgp_l3vpn_to_bgp_vrf | |
155 | ./test_bgp_l3vpn_to_bgp_vrf.py | |
156 | ||
370c8e07 QY |
157 | For further options, refer to pytest documentation. |
158 | ||
159 | Test will set exit code which can be used with ``git bisect``. | |
160 | ||
161 | For the simulated topology, see the description in the python file. | |
162 | ||
163 | If you need to clear the mininet setup between tests (if it isn't cleanly | |
164 | shutdown), then use the ``mn -c`` command to clean up the environment. | |
165 | ||
166 | StdErr log from daemos after exit | |
167 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
168 | ||
169 | To enable the reporting of any messages seen on StdErr after the daemons exit, | |
170 | the following env variable can be set:: | |
171 | ||
172 | export TOPOTESTS_CHECK_STDERR=Yes | |
173 | ||
e60aaed9 MS |
174 | (The value doesn't matter at this time. The check is whether the env |
175 | variable exists or not.) There is no pass/fail on this reporting; the | |
176 | Output will be reported to the console. | |
370c8e07 QY |
177 | |
178 | Collect Memory Leak Information | |
179 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
180 | ||
e60aaed9 MS |
181 | FRR processes can report unfreed memory allocations upon exit. To |
182 | enable the reporting of memory leaks, define an environment variable | |
370c8e07 QY |
183 | ``TOPOTESTS_CHECK_MEMLEAK`` with the file prefix, i.e.:: |
184 | ||
185 | export TOPOTESTS_CHECK_MEMLEAK="/home/mydir/memleak_" | |
186 | ||
e60aaed9 MS |
187 | This will enable the check and output to console and the writing of |
188 | the information to files with the given prefix (followed by testname), | |
189 | ie :file:`/home/mydir/memcheck_test_bgp_multiview_topo1.txt` in case | |
190 | of a memory leak. | |
370c8e07 QY |
191 | |
192 | Running Topotests with AddressSanitizer | |
193 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
194 | ||
195 | Topotests can be run with AddressSanitizer. It requires GCC 4.8 or newer. | |
196 | (Ubuntu 16.04 as suggested here is fine with GCC 5 as default). For more | |
197 | information on AddressSanitizer, see | |
198 | https://github.com/google/sanitizers/wiki/AddressSanitizer. | |
199 | ||
200 | The checks are done automatically in the library call of ``checkRouterRunning`` | |
201 | (ie at beginning of tests when there is a check for all daemons running). No | |
202 | changes or extra configuration for topotests is required beside compiling the | |
203 | suite with AddressSanitizer enabled. | |
204 | ||
205 | If a daemon crashed, then the errorlog is checked for AddressSanitizer output. | |
206 | If found, then this is added with context (calling test) to | |
207 | :file:`/tmp/AddressSanitizer.txt` in Markdown compatible format. | |
208 | ||
209 | Compiling for GCC AddressSanitizer requires to use ``gcc`` as a linker as well | |
210 | (instead of ``ld``). Here is a suggest way to compile frr with AddressSanitizer | |
a191719e | 211 | for ``master`` branch: |
370c8e07 QY |
212 | |
213 | .. code:: shell | |
214 | ||
215 | git clone https://github.com/FRRouting/frr.git | |
216 | cd frr | |
370c8e07 | 217 | ./bootstrap.sh |
4260f165 RZ |
218 | ./configure \ |
219 | --enable-address-sanitizer \ | |
370c8e07 QY |
220 | --prefix=/usr/lib/frr --sysconfdir=/etc/frr \ |
221 | --localstatedir=/var/run/frr \ | |
222 | --sbindir=/usr/lib/frr --bindir=/usr/lib/frr \ | |
223 | --enable-exampledir=/usr/lib/frr/examples \ | |
224 | --with-moduledir=/usr/lib/frr/modules \ | |
225 | --enable-multipath=0 --enable-rtadv \ | |
a191719e LB |
226 | --enable-tcp-zebra --enable-fpm --enable-pimd \ |
227 | --enable-sharpd | |
370c8e07 QY |
228 | make |
229 | sudo make install | |
230 | # Create symlink for vtysh, so topotest finds it in /usr/lib/frr | |
231 | sudo ln -s /usr/lib/frr/vtysh /usr/bin/ | |
232 | ||
233 | and create ``frr`` user and ``frrvty`` group as shown above. | |
234 | ||
235 | .. _topotests_docker: | |
236 | ||
237 | Running Tests with Docker | |
238 | ------------------------- | |
239 | ||
240 | There is a Docker image which allows to run topotests. | |
241 | ||
242 | Quickstart | |
243 | ^^^^^^^^^^ | |
244 | ||
245 | If you have Docker installed, you can run the topotests in Docker. The easiest | |
246 | way to do this, is to use the make targets from this repository. | |
247 | ||
248 | Your current user needs to have access to the Docker daemon. Alternatively you | |
249 | can run these commands as root. | |
250 | ||
251 | .. code:: console | |
252 | ||
253 | make topotests | |
254 | ||
255 | This command will pull the most recent topotests image from Dockerhub, compile | |
256 | FRR inside of it, and run the topotests. | |
257 | ||
258 | Advanced Usage | |
259 | ^^^^^^^^^^^^^^ | |
260 | ||
261 | Internally, the topotests make target uses a shell script to pull the image and | |
262 | spawn the Docker container. | |
263 | ||
264 | There are several environment variables which can be used to modify the | |
265 | behavior of the script, these can be listed by calling it with ``-h``: | |
266 | ||
267 | .. code:: console | |
268 | ||
269 | ./tests/topotests/docker/frr-topotests.sh -h | |
270 | ||
271 | For example, a volume is used to cache build artifacts between multiple runs of | |
272 | the image. If you need to force a complete recompile, you can set | |
273 | ``TOPOTEST_CLEAN``: | |
274 | ||
275 | .. code:: console | |
276 | ||
277 | TOPOTEST_CLEAN=1 ./tests/topotests/docker/frr-topotests.sh | |
278 | ||
279 | By default, ``frr-topotests.sh`` will build frr and run pytest. If you append | |
280 | arguments and the first one starts with ``/`` or ``./``, they will replace the | |
281 | call to pytest. If the appended arguments do not match this patttern, they will | |
282 | be provided to pytest as arguments. So, to run a specific test with more | |
283 | verbose logging: | |
284 | ||
285 | .. code:: console | |
286 | ||
287 | ./tests/topotests/docker/frr-topotests.sh -vv -s all-protocol-startup/test_all_protocol_startup.py | |
288 | ||
289 | And to compile FRR but drop into a shell instead of running pytest: | |
290 | ||
291 | .. code:: console | |
292 | ||
293 | ./tests/topotests/docker/frr-topotests.sh /bin/bash | |
294 | ||
295 | Development | |
296 | ^^^^^^^^^^^ | |
297 | ||
298 | The Docker image just includes all the components to run the topotests, but not | |
299 | the topotests themselves. So if you just want to write tests and don't want to | |
300 | make changes to the environment provided by the Docker image. You don't need to | |
301 | build your own Docker image if you do not want to. | |
302 | ||
303 | When developing new tests, there is one caveat though: The startup script of | |
304 | the container will run a ``git-clean`` on its copy of the FRR tree to avoid any | |
305 | pollution of the container with build artefacts from the host. This will also | |
306 | result in your newly written tests being unavailable in the container unless at | |
307 | least added to the index with ``git-add``. | |
308 | ||
309 | If you do want to test changes to the Docker image, you can locally build the | |
310 | image and run the tests without pulling from the registry using the following | |
311 | commands: | |
312 | ||
313 | .. code:: console | |
314 | ||
315 | make topotests-build | |
316 | TOPOTEST_PULL=0 make topotests | |
317 | ||
318 | ||
319 | .. _topotests-guidelines: | |
320 | ||
321 | Guidelines | |
322 | ---------- | |
323 | ||
324 | Executing Tests | |
325 | ^^^^^^^^^^^^^^^ | |
326 | ||
327 | To run the whole suite of tests the following commands must be executed at the | |
328 | top level directory of topotest: | |
329 | ||
330 | .. code:: shell | |
331 | ||
332 | $ # Change to the top level directory of topotests. | |
333 | $ cd path/to/topotests | |
334 | $ # Tests must be run as root, since Mininet requires it. | |
335 | $ sudo pytest | |
336 | ||
337 | In order to run a specific test, you can use the following command: | |
338 | ||
339 | .. code:: shell | |
340 | ||
341 | $ # running a specific topology | |
342 | $ sudo pytest ospf-topo1/ | |
343 | $ # or inside the test folder | |
344 | $ cd ospf-topo1 | |
345 | $ sudo pytest # to run all tests inside the directory | |
346 | $ sudo pytest test_ospf_topo1.py # to run a specific test | |
347 | $ # or outside the test folder | |
348 | $ cd .. | |
349 | $ sudo pytest ospf-topo1/test_ospf_topo1.py # to run a specific one | |
350 | ||
351 | The output of the tested daemons will be available at the temporary folder of | |
352 | your machine: | |
353 | ||
354 | .. code:: shell | |
355 | ||
356 | $ ls /tmp/topotest/ospf-topo1.test_ospf-topo1/r1 | |
357 | ... | |
358 | zebra.err # zebra stderr output | |
359 | zebra.log # zebra log file | |
360 | zebra.out # zebra stdout output | |
361 | ... | |
362 | ||
363 | You can also run memory leak tests to get reports: | |
364 | ||
365 | .. code:: shell | |
366 | ||
367 | $ # Set the environment variable to apply to a specific test... | |
368 | $ sudo env TOPOTESTS_CHECK_MEMLEAK="/tmp/memleak_report_" pytest ospf-topo1/test_ospf_topo1.py | |
369 | $ # ...or apply to all tests adding this line to the configuration file | |
370 | $ echo 'memleak_path = /tmp/memleak_report_' >> pytest.ini | |
371 | $ # You can also use your editor | |
372 | $ $EDITOR pytest.ini | |
373 | $ # After running tests you should see your files: | |
374 | $ ls /tmp/memleak_report_* | |
375 | memleak_report_test_ospf_topo1.txt | |
376 | ||
377 | Writing a New Test | |
378 | ^^^^^^^^^^^^^^^^^^ | |
379 | ||
380 | This section will guide you in all recommended steps to produce a standard | |
381 | topology test. | |
382 | ||
383 | This is the recommended test writing routine: | |
384 | ||
385 | - Write a topology (Graphviz recommended) | |
386 | - Obtain configuration files | |
387 | - Write the test itself | |
9dd78258 | 388 | - Format the new code using `black <https://github.com/psf/black>`_ |
370c8e07 QY |
389 | - Create a Pull Request |
390 | ||
b43be6b8 MS |
391 | Some things to keep in mind: |
392 | ||
393 | - BGP tests MUST use generous convergence timeouts - you must ensure | |
394 | that any test involving BGP uses a convergence timeout of at least | |
395 | 130 seconds. | |
396 | - Topotests are run on a range of Linux versions: if your test | |
397 | requires some OS-specific capability (like mpls support, or vrf | |
398 | support), there are test functions available in the libraries that | |
399 | will help you determine whether your test should run or be skipped. | |
400 | - Avoid including unstable data in your test: don't rely on link-local | |
401 | addresses or ifindex values, for example, because these can change | |
402 | from run to run. | |
d9c43f8f | 403 | |
d9c43f8f | 404 | |
370c8e07 QY |
405 | Topotest File Hierarchy |
406 | """"""""""""""""""""""" | |
407 | ||
408 | Before starting to write any tests one must know the file hierarchy. The | |
409 | repository hierarchy looks like this: | |
410 | ||
411 | .. code:: shell | |
412 | ||
413 | $ cd path/to/topotest | |
414 | $ find ./* | |
415 | ... | |
416 | ./README.md # repository read me | |
417 | ./GUIDELINES.md # this file | |
418 | ./conftest.py # test hooks - pytest related functions | |
419 | ./example-test # example test folder | |
420 | ./example-test/__init__.py # python package marker - must always exist. | |
421 | ./example-test/test_template.jpg # generated topology picture - see next section | |
422 | ./example-test/test_template.dot # Graphviz dot file | |
423 | ./example-test/test_template.py # the topology plus the test | |
424 | ... | |
425 | ./ospf-topo1 # the ospf topology test | |
426 | ./ospf-topo1/r1 # router 1 configuration files | |
427 | ./ospf-topo1/r1/zebra.conf # zebra configuration file | |
428 | ./ospf-topo1/r1/ospfd.conf # ospf configuration file | |
429 | ./ospf-topo1/r1/ospfroute.txt # 'show ip ospf' output reference file | |
430 | # removed other for shortness sake | |
431 | ... | |
432 | ./lib # shared test/topology functions | |
433 | ./lib/topogen.py # topogen implementation | |
434 | ./lib/topotest.py # topotest implementation | |
435 | ||
436 | Guidelines for creating/editing topotest: | |
437 | ||
438 | - New topologies that don't fit the existing directories should create its own | |
439 | - Always remember to add the ``__init__.py`` to new folders, this makes auto | |
440 | complete engines and pylint happy | |
441 | - Router (Quagga/FRR) specific code should go on topotest.py | |
442 | - Generic/repeated router actions should have an abstraction in | |
443 | topogen.TopoRouter. | |
444 | - Generic/repeated non-router code should go to topotest.py | |
445 | - pytest related code should go to conftest.py (e.g. specialized asserts) | |
446 | ||
447 | Defining the Topology | |
448 | """"""""""""""""""""" | |
449 | ||
450 | The first step to write a new test is to define the topology. This step can be | |
451 | done in many ways, but the recommended is to use Graphviz to generate a drawing | |
452 | of the topology. It allows us to see the topology graphically and to see the | |
56f0bea7 | 453 | names of equipment, links and addresses. |
370c8e07 QY |
454 | |
455 | Here is an example of Graphviz dot file that generates the template topology | |
456 | :file:`tests/topotests/example-test/test_template.dot` (the inlined code might | |
457 | get outdated, please see the linked file):: | |
458 | ||
459 | graph template { | |
460 | label="template"; | |
461 | ||
462 | # Routers | |
463 | r1 [ | |
464 | shape=doubleoctagon, | |
465 | label="r1", | |
466 | fillcolor="#f08080", | |
467 | style=filled, | |
468 | ]; | |
469 | r2 [ | |
470 | shape=doubleoctagon, | |
471 | label="r2", | |
472 | fillcolor="#f08080", | |
473 | style=filled, | |
474 | ]; | |
475 | ||
476 | # Switches | |
477 | s1 [ | |
478 | shape=oval, | |
479 | label="s1\n192.168.0.0/24", | |
480 | fillcolor="#d0e0d0", | |
481 | style=filled, | |
482 | ]; | |
483 | s2 [ | |
484 | shape=oval, | |
485 | label="s2\n192.168.1.0/24", | |
486 | fillcolor="#d0e0d0", | |
487 | style=filled, | |
488 | ]; | |
489 | ||
490 | # Connections | |
491 | r1 -- s1 [label="eth0\n.1"]; | |
492 | ||
493 | r1 -- s2 [label="eth1\n.100"]; | |
494 | r2 -- s2 [label="eth0\n.1"]; | |
495 | } | |
496 | ||
497 | Here is the produced graph: | |
498 | ||
499 | .. graphviz:: | |
500 | ||
501 | graph template { | |
502 | label="template"; | |
503 | ||
504 | # Routers | |
505 | r1 [ | |
506 | shape=doubleoctagon, | |
507 | label="r1", | |
508 | fillcolor="#f08080", | |
509 | style=filled, | |
510 | ]; | |
511 | r2 [ | |
512 | shape=doubleoctagon, | |
513 | label="r2", | |
514 | fillcolor="#f08080", | |
515 | style=filled, | |
516 | ]; | |
517 | ||
518 | # Switches | |
519 | s1 [ | |
520 | shape=oval, | |
521 | label="s1\n192.168.0.0/24", | |
522 | fillcolor="#d0e0d0", | |
523 | style=filled, | |
524 | ]; | |
525 | s2 [ | |
526 | shape=oval, | |
527 | label="s2\n192.168.1.0/24", | |
528 | fillcolor="#d0e0d0", | |
529 | style=filled, | |
530 | ]; | |
531 | ||
532 | # Connections | |
533 | r1 -- s1 [label="eth0\n.1"]; | |
534 | ||
535 | r1 -- s2 [label="eth1\n.100"]; | |
536 | r2 -- s2 [label="eth0\n.1"]; | |
537 | } | |
538 | ||
539 | Generating / Obtaining Configuration Files | |
540 | """""""""""""""""""""""""""""""""""""""""" | |
541 | ||
542 | In order to get the configuration files or command output for each router, we | |
543 | need to run the topology and execute commands in ``vtysh``. The quickest way to | |
544 | achieve that is writing the topology building code and running the topology. | |
545 | ||
546 | To bootstrap your test topology, do the following steps: | |
547 | ||
548 | - Copy the template test | |
549 | ||
550 | .. code:: shell | |
551 | ||
552 | $ mkdir new-topo/ | |
553 | $ touch new-topo/__init__.py | |
554 | $ cp example-test/test_template.py new-topo/test_new_topo.py | |
555 | ||
556 | - Modify the template according to your dot file | |
557 | ||
558 | Here is the template topology described in the previous section in python code: | |
559 | ||
560 | .. code:: py | |
561 | ||
562 | class TemplateTopo(Topo): | |
563 | "Test topology builder" | |
564 | def build(self, *_args, **_opts): | |
565 | "Build function" | |
566 | tgen = get_topogen(self) | |
567 | ||
568 | # Create 2 routers | |
569 | for routern in range(1, 3): | |
570 | tgen.add_router('r{}'.format(routern)) | |
571 | ||
572 | # Create a switch with just one router connected to it to simulate a | |
573 | # empty network. | |
574 | switch = tgen.add_switch('s1') | |
575 | switch.add_link(tgen.gears['r1']) | |
576 | ||
577 | # Create a connection between r1 and r2 | |
578 | switch = tgen.add_switch('s2') | |
579 | switch.add_link(tgen.gears['r1']) | |
580 | switch.add_link(tgen.gears['r2']) | |
581 | ||
582 | - Run the topology | |
583 | ||
584 | Topogen allows us to run the topology without running any tests, you can do | |
585 | that using the following example commands: | |
586 | ||
587 | .. code:: shell | |
588 | ||
589 | $ # Running your bootstraped topology | |
590 | $ sudo pytest -s --topology-only new-topo/test_new_topo.py | |
591 | $ # Running the test_template.py topology | |
592 | $ sudo pytest -s --topology-only example-test/test_template.py | |
593 | $ # Running the ospf_topo1.py topology | |
594 | $ sudo pytest -s --topology-only ospf-topo1/test_ospf_topo1.py | |
595 | ||
596 | Parameters explanation: | |
597 | ||
598 | .. program:: pytest | |
599 | ||
600 | .. option:: -s | |
601 | ||
602 | Actives input/output capture. This is required by mininet in order to show | |
603 | the interactive shell. | |
604 | ||
605 | .. option:: --topology-only | |
606 | ||
607 | Don't run any tests, just build the topology. | |
608 | ||
609 | After executing the commands above, you should get the following terminal | |
610 | output: | |
611 | ||
612 | .. code:: shell | |
613 | ||
614 | === test session starts === | |
615 | platform linux2 -- Python 2.7.12, pytest-3.1.2, py-1.4.34, pluggy-0.4.0 | |
616 | rootdir: /media/sf_src/topotests, inifile: pytest.ini | |
617 | collected 3 items | |
618 | ||
619 | ospf-topo1/test_ospf_topo1.py *** Starting controller | |
620 | ||
621 | *** Starting 6 switches | |
622 | switch1 switch2 switch3 switch4 switch5 switch6 ... | |
623 | r2: frr zebra started | |
624 | r2: frr ospfd started | |
625 | r3: frr zebra started | |
626 | r3: frr ospfd started | |
627 | r1: frr zebra started | |
628 | r1: frr ospfd started | |
629 | r4: frr zebra started | |
630 | r4: frr ospfd started | |
631 | *** Starting CLI: | |
632 | mininet> | |
633 | ||
634 | The last line shows us that we are now using the Mininet CLI (Command Line | |
635 | Interface), from here you can call your router ``vtysh`` or even bash. | |
636 | ||
637 | Here are some commands example: | |
638 | ||
639 | .. code:: shell | |
640 | ||
641 | mininet> r1 ping 10.0.3.1 | |
642 | PING 10.0.3.1 (10.0.3.1) 56(84) bytes of data. | |
643 | 64 bytes from 10.0.3.1: icmp_seq=1 ttl=64 time=0.576 ms | |
644 | 64 bytes from 10.0.3.1: icmp_seq=2 ttl=64 time=0.083 ms | |
645 | 64 bytes from 10.0.3.1: icmp_seq=3 ttl=64 time=0.088 ms | |
646 | ^C | |
647 | --- 10.0.3.1 ping statistics --- | |
648 | 3 packets transmitted, 3 received, 0% packet loss, time 1998ms | |
649 | rtt min/avg/max/mdev = 0.083/0.249/0.576/0.231 ms | |
650 | ||
651 | ||
652 | ||
653 | mininet> r1 ping 10.0.3.3 | |
654 | PING 10.0.3.3 (10.0.3.3) 56(84) bytes of data. | |
655 | 64 bytes from 10.0.3.3: icmp_seq=1 ttl=64 time=2.87 ms | |
656 | 64 bytes from 10.0.3.3: icmp_seq=2 ttl=64 time=0.080 ms | |
657 | 64 bytes from 10.0.3.3: icmp_seq=3 ttl=64 time=0.091 ms | |
658 | ^C | |
659 | --- 10.0.3.3 ping statistics --- | |
660 | 3 packets transmitted, 3 received, 0% packet loss, time 2003ms | |
661 | rtt min/avg/max/mdev = 0.080/1.014/2.872/1.313 ms | |
662 | ||
663 | ||
664 | ||
665 | mininet> r3 vtysh | |
666 | ||
667 | Hello, this is FRRouting (version 3.1-devrzalamena-build). | |
668 | Copyright 1996-2005 Kunihiro Ishiguro, et al. | |
669 | ||
670 | frr-1# show running-config | |
671 | Building configuration... | |
672 | ||
673 | Current configuration: | |
674 | ! | |
675 | frr version 3.1-devrzalamena-build | |
676 | frr defaults traditional | |
677 | hostname r3 | |
678 | no service integrated-vtysh-config | |
679 | ! | |
680 | log file zebra.log | |
681 | ! | |
682 | log file ospfd.log | |
683 | ! | |
684 | interface r3-eth0 | |
685 | ip address 10.0.3.1/24 | |
686 | ! | |
687 | interface r3-eth1 | |
688 | ip address 10.0.10.1/24 | |
689 | ! | |
690 | interface r3-eth2 | |
691 | ip address 172.16.0.2/24 | |
692 | ! | |
693 | router ospf | |
694 | ospf router-id 10.0.255.3 | |
695 | redistribute kernel | |
696 | redistribute connected | |
697 | redistribute static | |
698 | network 10.0.3.0/24 area 0 | |
699 | network 10.0.10.0/24 area 0 | |
700 | network 172.16.0.0/24 area 1 | |
701 | ! | |
702 | line vty | |
703 | ! | |
704 | end | |
705 | frr-1# | |
706 | ||
707 | After you successfully configured your topology, you can obtain the | |
708 | configuration files (per-daemon) using the following commands: | |
709 | ||
710 | .. code:: shell | |
711 | ||
712 | mininet> r3 vtysh -d ospfd | |
713 | ||
714 | Hello, this is FRRouting (version 3.1-devrzalamena-build). | |
715 | Copyright 1996-2005 Kunihiro Ishiguro, et al. | |
716 | ||
717 | frr-1# show running-config | |
718 | Building configuration... | |
719 | ||
720 | Current configuration: | |
721 | ! | |
722 | frr version 3.1-devrzalamena-build | |
723 | frr defaults traditional | |
724 | no service integrated-vtysh-config | |
725 | ! | |
726 | log file ospfd.log | |
727 | ! | |
728 | router ospf | |
729 | ospf router-id 10.0.255.3 | |
730 | redistribute kernel | |
731 | redistribute connected | |
732 | redistribute static | |
733 | network 10.0.3.0/24 area 0 | |
734 | network 10.0.10.0/24 area 0 | |
735 | network 172.16.0.0/24 area 1 | |
736 | ! | |
737 | line vty | |
738 | ! | |
739 | end | |
740 | frr-1# | |
741 | ||
742 | Writing Tests | |
743 | """"""""""""" | |
744 | ||
745 | Test topologies should always be bootstrapped from | |
746 | :file:`tests/topotests/example-test/test_template.py` because it contains | |
747 | important boilerplate code that can't be avoided, like: | |
748 | ||
749 | - imports: os, sys, pytest, topotest/topogen and mininet topology class | |
750 | - The global variable CWD (Current Working directory): which is most likely | |
751 | going to be used to reference the routers configuration file location | |
752 | ||
753 | Example: | |
754 | ||
755 | .. code:: py | |
756 | ||
757 | # For all registered routers, load the zebra configuration file | |
e5f0ed14 | 758 | for rname, router in router_list.items(): |
370c8e07 QY |
759 | router.load_config( |
760 | TopoRouter.RD_ZEBRA, | |
761 | os.path.join(CWD, '{}/zebra.conf'.format(rname)) | |
762 | ) | |
763 | # os.path.join() joins the CWD string with arguments adding the necessary | |
764 | # slashes ('/'). Arguments must not begin with '/'. | |
765 | ||
766 | - The topology class that inherits from Mininet Topo class: | |
767 | ||
768 | .. code:: py | |
769 | ||
770 | class TemplateTopo(Topo): | |
771 | def build(self, *_args, **_opts): | |
772 | tgen = get_topogen(self) | |
773 | # topology build code | |
774 | ||
775 | - pytest ``setup_module()`` and ``teardown_module()`` to start the topology | |
776 | ||
777 | .. code:: py | |
778 | ||
779 | def setup_module(_m): | |
780 | tgen = Topogen(TemplateTopo) | |
781 | tgen.start_topology('debug') | |
782 | ||
783 | def teardown_module(_m): | |
784 | tgen = get_topogen() | |
785 | tgen.stop_topology() | |
786 | ||
787 | - ``__main__`` initialization code (to support running the script directly) | |
788 | ||
789 | .. code:: py | |
790 | ||
791 | if __name__ == '__main__': | |
792 | sys.exit(pytest.main(["-s"])) | |
793 | ||
794 | Requirements: | |
795 | ||
796 | - Test code should always be declared inside functions that begin with the | |
797 | ``test_`` prefix. Functions beginning with different prefixes will not be run | |
798 | by pytest. | |
799 | - Configuration files and long output commands should go into separated files | |
800 | inside folders named after the equipment. | |
801 | - Tests must be able to run without any interaction. To make sure your test | |
802 | conforms with this, run it without the :option:`-s` parameter. | |
9dd78258 DA |
803 | - Use `black <https://github.com/psf/black>`_ code formatter before creating |
804 | a pull request. This ensures we have a unified code style. | |
0f84d138 | 805 | - Mark test modules with pytest markers depending on the daemons used during the |
b43be6b8 | 806 | tests (see :ref:`topotests-markers`) |
370c8e07 QY |
807 | |
808 | Tips: | |
809 | ||
810 | - Keep results in stack variables, so people inspecting code with ``pdb`` can | |
811 | easily print their values. | |
812 | ||
813 | Don't do this: | |
814 | ||
815 | .. code:: py | |
816 | ||
817 | assert foobar(router1, router2) | |
818 | ||
819 | Do this instead: | |
820 | ||
821 | .. code:: py | |
822 | ||
823 | result = foobar(router1, router2) | |
824 | assert result | |
825 | ||
826 | - Use ``assert`` messages to indicate where the test failed. | |
827 | ||
828 | Example: | |
829 | ||
830 | .. code:: py | |
831 | ||
832 | for router in router_list: | |
833 | # ... | |
834 | assert condition, 'Router "{}" condition failed'.format(router.name) | |
835 | ||
836 | Debugging Execution | |
837 | ^^^^^^^^^^^^^^^^^^^ | |
838 | ||
839 | The most effective ways to inspect topology tests are: | |
840 | ||
841 | - Run pytest with ``--pdb`` option. This option will cause a pdb shell to | |
842 | appear when an assertion fails | |
843 | ||
844 | Example: ``pytest -s --pdb ospf-topo1/test_ospf_topo1.py`` | |
845 | ||
846 | - Set a breakpoint in the test code with ``pdb`` | |
847 | ||
848 | Example: | |
849 | ||
850 | .. code:: py | |
851 | ||
852 | # Add the pdb import at the beginning of the file | |
853 | import pdb | |
854 | # ... | |
855 | ||
856 | # Add a breakpoint where you think the problem is | |
857 | def test_bla(): | |
858 | # ... | |
859 | pdb.set_trace() | |
860 | # ... | |
861 | ||
862 | The `Python Debugger <https://docs.python.org/2.7/library/pdb.html>`__ (pdb) | |
863 | shell allows us to run many useful operations like: | |
864 | ||
865 | - Setting breaking point on file/function/conditions (e.g. ``break``, | |
866 | ``condition``) | |
867 | - Inspecting variables (e.g. ``p`` (print), ``pp`` (pretty print)) | |
868 | - Running python code | |
869 | ||
870 | .. tip:: | |
871 | ||
872 | The TopoGear (equipment abstraction class) implements the ``__str__`` method | |
873 | that allows the user to inspect equipment information. | |
874 | ||
875 | Example of pdb usage: | |
876 | ||
877 | .. code:: shell | |
878 | ||
879 | > /media/sf_src/topotests/ospf-topo1/test_ospf_topo1.py(121)test_ospf_convergence() | |
880 | -> for rnum in range(1, 5): | |
881 | (Pdb) help | |
882 | Documented commands (type help <topic>): | |
883 | ======================================== | |
884 | EOF bt cont enable jump pp run unt | |
885 | a c continue exit l q s until | |
886 | alias cl d h list quit step up | |
887 | args clear debug help n r tbreak w | |
888 | b commands disable ignore next restart u whatis | |
889 | break condition down j p return unalias where | |
890 | ||
891 | Miscellaneous help topics: | |
892 | ========================== | |
893 | exec pdb | |
894 | ||
895 | Undocumented commands: | |
896 | ====================== | |
897 | retval rv | |
898 | ||
899 | (Pdb) list | |
900 | 116 title2="Expected output") | |
901 | 117 | |
902 | 118 def test_ospf_convergence(): | |
903 | 119 "Test OSPF daemon convergence" | |
904 | 120 pdb.set_trace() | |
905 | 121 -> for rnum in range(1, 5): | |
906 | 122 router = 'r{}'.format(rnum) | |
907 | 123 | |
908 | 124 # Load expected results from the command | |
909 | 125 reffile = os.path.join(CWD, '{}/ospfroute.txt'.format(router)) | |
910 | 126 expected = open(reffile).read() | |
911 | (Pdb) step | |
912 | > /media/sf_src/topotests/ospf-topo1/test_ospf_topo1.py(122)test_ospf_convergence() | |
913 | -> router = 'r{}'.format(rnum) | |
914 | (Pdb) step | |
915 | > /media/sf_src/topotests/ospf-topo1/test_ospf_topo1.py(125)test_ospf_convergence() | |
916 | -> reffile = os.path.join(CWD, '{}/ospfroute.txt'.format(router)) | |
917 | (Pdb) print rnum | |
918 | 1 | |
919 | (Pdb) print router | |
920 | r1 | |
921 | (Pdb) tgen = get_topogen() | |
922 | (Pdb) pp tgen.gears[router] | |
923 | <lib.topogen.TopoRouter object at 0x7f74e06c9850> | |
924 | (Pdb) pp str(tgen.gears[router]) | |
925 | 'TopoGear<name="r1",links=["r1-eth0"<->"s1-eth0","r1-eth1"<->"s3-eth0"]> TopoRouter<>' | |
926 | (Pdb) l 125 | |
927 | 120 pdb.set_trace() | |
928 | 121 for rnum in range(1, 5): | |
929 | 122 router = 'r{}'.format(rnum) | |
930 | 123 | |
931 | 124 # Load expected results from the command | |
932 | 125 -> reffile = os.path.join(CWD, '{}/ospfroute.txt'.format(router)) | |
933 | 126 expected = open(reffile).read() | |
934 | 127 | |
935 | 128 # Run test function until we get an result. Wait at most 60 seconds. | |
936 | 129 test_func = partial(compare_show_ip_ospf, router, expected) | |
937 | 130 result, diff = topotest.run_and_expect(test_func, '', | |
938 | (Pdb) router1 = tgen.gears[router] | |
939 | (Pdb) router1.vtysh_cmd('show ip ospf route') | |
940 | '============ OSPF network routing table ============\r\nN 10.0.1.0/24 [10] area: 0.0.0.0\r\n directly attached to r1-eth0\r\nN 10.0.2.0/24 [20] area: 0.0.0.0\r\n via 10.0.3.3, r1-eth1\r\nN 10.0.3.0/24 [10] area: 0.0.0.0\r\n directly attached to r1-eth1\r\nN 10.0.10.0/24 [20] area: 0.0.0.0\r\n via 10.0.3.1, r1-eth1\r\nN IA 172.16.0.0/24 [20] area: 0.0.0.0\r\n via 10.0.3.1, r1-eth1\r\nN IA 172.16.1.0/24 [30] area: 0.0.0.0\r\n via 10.0.3.1, r1-eth1\r\n\r\n============ OSPF router routing table =============\r\nR 10.0.255.2 [10] area: 0.0.0.0, ASBR\r\n via 10.0.3.3, r1-eth1\r\nR 10.0.255.3 [10] area: 0.0.0.0, ABR, ASBR\r\n via 10.0.3.1, r1-eth1\r\nR 10.0.255.4 IA [20] area: 0.0.0.0, ASBR\r\n via 10.0.3.1, r1-eth1\r\n\r\n============ OSPF external routing table ===========\r\n\r\n\r\n' | |
941 | (Pdb) tgen.mininet_cli() | |
942 | *** Starting CLI: | |
943 | mininet> | |
944 | ||
945 | To enable more debug messages in other Topogen subsystems (like Mininet), more | |
946 | logging messages can be displayed by modifying the test configuration file | |
947 | ``pytest.ini``: | |
948 | ||
949 | .. code:: ini | |
950 | ||
951 | [topogen] | |
952 | # Change the default verbosity line from 'info'... | |
953 | #verbosity = info | |
954 | # ...to 'debug' | |
955 | verbosity = debug | |
956 | ||
957 | Instructions for use, write or debug topologies can be found in :ref:`topotests-guidelines`. | |
958 | To learn/remember common code snippets see :ref:`topotests-snippets`. | |
959 | ||
960 | Before creating a new topology, make sure that there isn't one already that | |
961 | does what you need. If nothing is similar, then you may create a new topology, | |
962 | preferably, using the newest template | |
963 | (:file:`tests/topotests/example-test/test_template.py`). | |
964 | ||
0f84d138 DS |
965 | .. include:: topotests-markers.rst |
966 | ||
370c8e07 QY |
967 | .. include:: topotests-snippets.rst |
968 | ||
969 | License | |
970 | ------- | |
971 | ||
972 | All the configs and scripts are licensed under a ISC-style license. See Python | |
973 | scripts for details. |