]>
Commit | Line | Data |
---|---|---|
370c8e07 QY |
1 | .. _topotests: |
2 | ||
3 | Topotests | |
4 | ========= | |
5 | ||
6 | Topotests is a suite of topology tests for FRR built on top of Mininet. | |
7 | ||
8 | Installation and Setup | |
9 | ---------------------- | |
10 | ||
11 | Only tested with Ubuntu 16.04 and Ubuntu 18.04 (which uses Mininet 2.2.x). | |
12 | ||
13 | Instructions are the same for all setups (i.e. ExaBGP is only used for BGP | |
14 | tests). | |
15 | ||
16 | Installing Mininet Infrastructure | |
17 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
18 | ||
19 | .. code:: shell | |
20 | ||
21 | apt-get install mininet | |
22 | apt-get install python-pip | |
23 | apt-get install iproute | |
24 | pip install ipaddr | |
25 | pip install pytest | |
26 | pip install exabgp==3.4.17 (Newer 4.0 version of exabgp is not yet | |
27 | supported) | |
28 | useradd -d /var/run/exabgp/ -s /bin/false exabgp | |
29 | ||
30 | Enable Coredumps | |
31 | """""""""""""""" | |
32 | ||
33 | Optional, will give better output. | |
34 | ||
35 | .. code:: shell | |
36 | ||
37 | apt-get install gdb | |
38 | disable apport (which move core files) | |
39 | ||
40 | Set ``enabled=0`` in ``/etc/default/apport``. | |
41 | ||
42 | Next, update security limits by changing :file:`/etc/security/limits.conf` to:: | |
43 | ||
44 | #<domain> <type> <item> <value> | |
45 | * soft core unlimited | |
46 | root soft core unlimited | |
47 | * hard core unlimited | |
48 | root hard core unlimited | |
49 | ||
50 | Reboot for options to take effect. | |
51 | ||
52 | FRR Installation | |
53 | ^^^^^^^^^^^^^^^^ | |
54 | ||
55 | FRR needs to be installed separately. It is assume to be configured like the | |
56 | standard Ubuntu Packages: | |
57 | ||
58 | - Binaries in :file:`/usr/lib/frr` | |
59 | - State Directory :file:`/var/run/frr` | |
60 | - Running under user ``frr``, group ``frr`` | |
61 | - vtygroup: ``frrvty`` | |
62 | - config directory: :file:`/etc/frr` | |
63 | - For FRR Packages, install the dbg package as well for coredump decoding | |
64 | ||
65 | No FRR config needs to be done and no FRR daemons should be run ahead of the | |
66 | test. They are all started as part of the test. | |
67 | ||
68 | Manual FRR build | |
69 | """""""""""""""" | |
70 | ||
71 | If you prefer to manually build FRR, then use the following suggested config: | |
72 | ||
73 | .. code:: shell | |
74 | ||
75 | ./configure \ | |
76 | --prefix=/usr \ | |
77 | --localstatedir=/var/run/frr \ | |
78 | --sbindir=/usr/lib/frr \ | |
79 | --sysconfdir=/etc/frr \ | |
80 | --enable-vtysh \ | |
81 | --enable-pimd \ | |
82 | --enable-multipath=64 \ | |
83 | --enable-user=frr \ | |
84 | --enable-group=frr \ | |
85 | --enable-vty-group=frrvty \ | |
86 | --with-pkg-extra-version=-my-manual-build | |
87 | ||
88 | And create ``frr`` user and ``frrvty`` group as follows: | |
89 | ||
90 | .. code:: shell | |
91 | ||
92 | addgroup --system --gid 92 frr | |
93 | addgroup --system --gid 85 frrvty | |
94 | adduser --system --ingroup frr --home /var/run/frr/ \ | |
95 | --gecos "FRRouting suite" --shell /bin/false frr | |
96 | usermod -G frrvty frr | |
97 | ||
98 | Executing Tests | |
99 | --------------- | |
100 | ||
101 | Execute all tests with output to console | |
102 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
103 | ||
104 | .. code:: shell | |
105 | ||
106 | py.test -s -v --tb=no | |
107 | ||
108 | All test\_\* scripts in subdirectories are detected and executed (unless | |
109 | disabled in ``pytest.ini`` file). | |
110 | ||
111 | ``--tb=no`` disables the python traceback which might be irrelevant unless the | |
112 | test script itself is debugged. | |
113 | ||
114 | Execute single test | |
115 | ^^^^^^^^^^^^^^^^^^^ | |
116 | ||
117 | .. code:: shell | |
118 | ||
119 | cd test_to_be_run | |
120 | ./test_to_be_run.py | |
121 | ||
122 | For further options, refer to pytest documentation. | |
123 | ||
124 | Test will set exit code which can be used with ``git bisect``. | |
125 | ||
126 | For the simulated topology, see the description in the python file. | |
127 | ||
128 | If you need to clear the mininet setup between tests (if it isn't cleanly | |
129 | shutdown), then use the ``mn -c`` command to clean up the environment. | |
130 | ||
131 | StdErr log from daemos after exit | |
132 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
133 | ||
134 | To enable the reporting of any messages seen on StdErr after the daemons exit, | |
135 | the following env variable can be set:: | |
136 | ||
137 | export TOPOTESTS_CHECK_STDERR=Yes | |
138 | ||
139 | (The value doesn't matter at this time. The check is if the env variable exists | |
140 | or not) There is no pass/fail on this reporting. The Output will be reported to | |
141 | the console:: | |
142 | ||
143 | export TOPOTESTS_CHECK_MEMLEAK="/home/mydir/memleak_" | |
144 | ||
145 | This will enable the check and output to console and the writing of the | |
146 | information to files with the given prefix (followed by testname), ie | |
147 | :file:`/home/mydir/memcheck_test_bgp_multiview_topo1.txt` in case of a memory | |
148 | leak. | |
149 | ||
150 | Collect Memory Leak Information | |
151 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
152 | ||
153 | FRR processes have the capabilities to report remaining memory allocations upon | |
56f0bea7 | 154 | exit. To enable the reporting of the memory, define an environment variable |
370c8e07 QY |
155 | ``TOPOTESTS_CHECK_MEMLEAK`` with the file prefix, i.e.:: |
156 | ||
157 | export TOPOTESTS_CHECK_MEMLEAK="/home/mydir/memleak_" | |
158 | ||
159 | This will enable the check and output to console and the writing of the | |
160 | information to files with the given prefix (followed by testname), ie | |
161 | :file:`/home/mydir/memcheck_test_bgp_multiview_topo1.txt` in case of a memory | |
162 | leak. | |
163 | ||
164 | Running Topotests with AddressSanitizer | |
165 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
166 | ||
167 | Topotests can be run with AddressSanitizer. It requires GCC 4.8 or newer. | |
168 | (Ubuntu 16.04 as suggested here is fine with GCC 5 as default). For more | |
169 | information on AddressSanitizer, see | |
170 | https://github.com/google/sanitizers/wiki/AddressSanitizer. | |
171 | ||
172 | The checks are done automatically in the library call of ``checkRouterRunning`` | |
173 | (ie at beginning of tests when there is a check for all daemons running). No | |
174 | changes or extra configuration for topotests is required beside compiling the | |
175 | suite with AddressSanitizer enabled. | |
176 | ||
177 | If a daemon crashed, then the errorlog is checked for AddressSanitizer output. | |
178 | If found, then this is added with context (calling test) to | |
179 | :file:`/tmp/AddressSanitizer.txt` in Markdown compatible format. | |
180 | ||
181 | Compiling for GCC AddressSanitizer requires to use ``gcc`` as a linker as well | |
182 | (instead of ``ld``). Here is a suggest way to compile frr with AddressSanitizer | |
a191719e | 183 | for ``master`` branch: |
370c8e07 QY |
184 | |
185 | .. code:: shell | |
186 | ||
187 | git clone https://github.com/FRRouting/frr.git | |
188 | cd frr | |
370c8e07 QY |
189 | ./bootstrap.sh |
190 | export CC=gcc | |
191 | export CFLAGS="-O1 -g -fsanitize=address -fno-omit-frame-pointer" | |
192 | export LD=gcc | |
193 | export LDFLAGS="-g -fsanitize=address -ldl" | |
194 | ./configure --enable-shared=no \ | |
195 | --prefix=/usr/lib/frr --sysconfdir=/etc/frr \ | |
196 | --localstatedir=/var/run/frr \ | |
197 | --sbindir=/usr/lib/frr --bindir=/usr/lib/frr \ | |
198 | --enable-exampledir=/usr/lib/frr/examples \ | |
199 | --with-moduledir=/usr/lib/frr/modules \ | |
200 | --enable-multipath=0 --enable-rtadv \ | |
a191719e LB |
201 | --enable-tcp-zebra --enable-fpm --enable-pimd \ |
202 | --enable-sharpd | |
370c8e07 QY |
203 | make |
204 | sudo make install | |
205 | # Create symlink for vtysh, so topotest finds it in /usr/lib/frr | |
206 | sudo ln -s /usr/lib/frr/vtysh /usr/bin/ | |
207 | ||
208 | and create ``frr`` user and ``frrvty`` group as shown above. | |
209 | ||
210 | .. _topotests_docker: | |
211 | ||
212 | Running Tests with Docker | |
213 | ------------------------- | |
214 | ||
215 | There is a Docker image which allows to run topotests. | |
216 | ||
217 | Quickstart | |
218 | ^^^^^^^^^^ | |
219 | ||
220 | If you have Docker installed, you can run the topotests in Docker. The easiest | |
221 | way to do this, is to use the make targets from this repository. | |
222 | ||
223 | Your current user needs to have access to the Docker daemon. Alternatively you | |
224 | can run these commands as root. | |
225 | ||
226 | .. code:: console | |
227 | ||
228 | make topotests | |
229 | ||
230 | This command will pull the most recent topotests image from Dockerhub, compile | |
231 | FRR inside of it, and run the topotests. | |
232 | ||
233 | Advanced Usage | |
234 | ^^^^^^^^^^^^^^ | |
235 | ||
236 | Internally, the topotests make target uses a shell script to pull the image and | |
237 | spawn the Docker container. | |
238 | ||
239 | There are several environment variables which can be used to modify the | |
240 | behavior of the script, these can be listed by calling it with ``-h``: | |
241 | ||
242 | .. code:: console | |
243 | ||
244 | ./tests/topotests/docker/frr-topotests.sh -h | |
245 | ||
246 | For example, a volume is used to cache build artifacts between multiple runs of | |
247 | the image. If you need to force a complete recompile, you can set | |
248 | ``TOPOTEST_CLEAN``: | |
249 | ||
250 | .. code:: console | |
251 | ||
252 | TOPOTEST_CLEAN=1 ./tests/topotests/docker/frr-topotests.sh | |
253 | ||
254 | By default, ``frr-topotests.sh`` will build frr and run pytest. If you append | |
255 | arguments and the first one starts with ``/`` or ``./``, they will replace the | |
256 | call to pytest. If the appended arguments do not match this patttern, they will | |
257 | be provided to pytest as arguments. So, to run a specific test with more | |
258 | verbose logging: | |
259 | ||
260 | .. code:: console | |
261 | ||
262 | ./tests/topotests/docker/frr-topotests.sh -vv -s all-protocol-startup/test_all_protocol_startup.py | |
263 | ||
264 | And to compile FRR but drop into a shell instead of running pytest: | |
265 | ||
266 | .. code:: console | |
267 | ||
268 | ./tests/topotests/docker/frr-topotests.sh /bin/bash | |
269 | ||
270 | Development | |
271 | ^^^^^^^^^^^ | |
272 | ||
273 | The Docker image just includes all the components to run the topotests, but not | |
274 | the topotests themselves. So if you just want to write tests and don't want to | |
275 | make changes to the environment provided by the Docker image. You don't need to | |
276 | build your own Docker image if you do not want to. | |
277 | ||
278 | When developing new tests, there is one caveat though: The startup script of | |
279 | the container will run a ``git-clean`` on its copy of the FRR tree to avoid any | |
280 | pollution of the container with build artefacts from the host. This will also | |
281 | result in your newly written tests being unavailable in the container unless at | |
282 | least added to the index with ``git-add``. | |
283 | ||
284 | If you do want to test changes to the Docker image, you can locally build the | |
285 | image and run the tests without pulling from the registry using the following | |
286 | commands: | |
287 | ||
288 | .. code:: console | |
289 | ||
290 | make topotests-build | |
291 | TOPOTEST_PULL=0 make topotests | |
292 | ||
293 | ||
294 | .. _topotests-guidelines: | |
295 | ||
296 | Guidelines | |
297 | ---------- | |
298 | ||
299 | Executing Tests | |
300 | ^^^^^^^^^^^^^^^ | |
301 | ||
302 | To run the whole suite of tests the following commands must be executed at the | |
303 | top level directory of topotest: | |
304 | ||
305 | .. code:: shell | |
306 | ||
307 | $ # Change to the top level directory of topotests. | |
308 | $ cd path/to/topotests | |
309 | $ # Tests must be run as root, since Mininet requires it. | |
310 | $ sudo pytest | |
311 | ||
312 | In order to run a specific test, you can use the following command: | |
313 | ||
314 | .. code:: shell | |
315 | ||
316 | $ # running a specific topology | |
317 | $ sudo pytest ospf-topo1/ | |
318 | $ # or inside the test folder | |
319 | $ cd ospf-topo1 | |
320 | $ sudo pytest # to run all tests inside the directory | |
321 | $ sudo pytest test_ospf_topo1.py # to run a specific test | |
322 | $ # or outside the test folder | |
323 | $ cd .. | |
324 | $ sudo pytest ospf-topo1/test_ospf_topo1.py # to run a specific one | |
325 | ||
326 | The output of the tested daemons will be available at the temporary folder of | |
327 | your machine: | |
328 | ||
329 | .. code:: shell | |
330 | ||
331 | $ ls /tmp/topotest/ospf-topo1.test_ospf-topo1/r1 | |
332 | ... | |
333 | zebra.err # zebra stderr output | |
334 | zebra.log # zebra log file | |
335 | zebra.out # zebra stdout output | |
336 | ... | |
337 | ||
338 | You can also run memory leak tests to get reports: | |
339 | ||
340 | .. code:: shell | |
341 | ||
342 | $ # Set the environment variable to apply to a specific test... | |
343 | $ sudo env TOPOTESTS_CHECK_MEMLEAK="/tmp/memleak_report_" pytest ospf-topo1/test_ospf_topo1.py | |
344 | $ # ...or apply to all tests adding this line to the configuration file | |
345 | $ echo 'memleak_path = /tmp/memleak_report_' >> pytest.ini | |
346 | $ # You can also use your editor | |
347 | $ $EDITOR pytest.ini | |
348 | $ # After running tests you should see your files: | |
349 | $ ls /tmp/memleak_report_* | |
350 | memleak_report_test_ospf_topo1.txt | |
351 | ||
352 | Writing a New Test | |
353 | ^^^^^^^^^^^^^^^^^^ | |
354 | ||
355 | This section will guide you in all recommended steps to produce a standard | |
356 | topology test. | |
357 | ||
358 | This is the recommended test writing routine: | |
359 | ||
360 | - Write a topology (Graphviz recommended) | |
361 | - Obtain configuration files | |
362 | - Write the test itself | |
363 | - Create a Pull Request | |
364 | ||
365 | Topotest File Hierarchy | |
366 | """"""""""""""""""""""" | |
367 | ||
368 | Before starting to write any tests one must know the file hierarchy. The | |
369 | repository hierarchy looks like this: | |
370 | ||
371 | .. code:: shell | |
372 | ||
373 | $ cd path/to/topotest | |
374 | $ find ./* | |
375 | ... | |
376 | ./README.md # repository read me | |
377 | ./GUIDELINES.md # this file | |
378 | ./conftest.py # test hooks - pytest related functions | |
379 | ./example-test # example test folder | |
380 | ./example-test/__init__.py # python package marker - must always exist. | |
381 | ./example-test/test_template.jpg # generated topology picture - see next section | |
382 | ./example-test/test_template.dot # Graphviz dot file | |
383 | ./example-test/test_template.py # the topology plus the test | |
384 | ... | |
385 | ./ospf-topo1 # the ospf topology test | |
386 | ./ospf-topo1/r1 # router 1 configuration files | |
387 | ./ospf-topo1/r1/zebra.conf # zebra configuration file | |
388 | ./ospf-topo1/r1/ospfd.conf # ospf configuration file | |
389 | ./ospf-topo1/r1/ospfroute.txt # 'show ip ospf' output reference file | |
390 | # removed other for shortness sake | |
391 | ... | |
392 | ./lib # shared test/topology functions | |
393 | ./lib/topogen.py # topogen implementation | |
394 | ./lib/topotest.py # topotest implementation | |
395 | ||
396 | Guidelines for creating/editing topotest: | |
397 | ||
398 | - New topologies that don't fit the existing directories should create its own | |
399 | - Always remember to add the ``__init__.py`` to new folders, this makes auto | |
400 | complete engines and pylint happy | |
401 | - Router (Quagga/FRR) specific code should go on topotest.py | |
402 | - Generic/repeated router actions should have an abstraction in | |
403 | topogen.TopoRouter. | |
404 | - Generic/repeated non-router code should go to topotest.py | |
405 | - pytest related code should go to conftest.py (e.g. specialized asserts) | |
406 | ||
407 | Defining the Topology | |
408 | """"""""""""""""""""" | |
409 | ||
410 | The first step to write a new test is to define the topology. This step can be | |
411 | done in many ways, but the recommended is to use Graphviz to generate a drawing | |
412 | of the topology. It allows us to see the topology graphically and to see the | |
56f0bea7 | 413 | names of equipment, links and addresses. |
370c8e07 QY |
414 | |
415 | Here is an example of Graphviz dot file that generates the template topology | |
416 | :file:`tests/topotests/example-test/test_template.dot` (the inlined code might | |
417 | get outdated, please see the linked file):: | |
418 | ||
419 | graph template { | |
420 | label="template"; | |
421 | ||
422 | # Routers | |
423 | r1 [ | |
424 | shape=doubleoctagon, | |
425 | label="r1", | |
426 | fillcolor="#f08080", | |
427 | style=filled, | |
428 | ]; | |
429 | r2 [ | |
430 | shape=doubleoctagon, | |
431 | label="r2", | |
432 | fillcolor="#f08080", | |
433 | style=filled, | |
434 | ]; | |
435 | ||
436 | # Switches | |
437 | s1 [ | |
438 | shape=oval, | |
439 | label="s1\n192.168.0.0/24", | |
440 | fillcolor="#d0e0d0", | |
441 | style=filled, | |
442 | ]; | |
443 | s2 [ | |
444 | shape=oval, | |
445 | label="s2\n192.168.1.0/24", | |
446 | fillcolor="#d0e0d0", | |
447 | style=filled, | |
448 | ]; | |
449 | ||
450 | # Connections | |
451 | r1 -- s1 [label="eth0\n.1"]; | |
452 | ||
453 | r1 -- s2 [label="eth1\n.100"]; | |
454 | r2 -- s2 [label="eth0\n.1"]; | |
455 | } | |
456 | ||
457 | Here is the produced graph: | |
458 | ||
459 | .. graphviz:: | |
460 | ||
461 | graph template { | |
462 | label="template"; | |
463 | ||
464 | # Routers | |
465 | r1 [ | |
466 | shape=doubleoctagon, | |
467 | label="r1", | |
468 | fillcolor="#f08080", | |
469 | style=filled, | |
470 | ]; | |
471 | r2 [ | |
472 | shape=doubleoctagon, | |
473 | label="r2", | |
474 | fillcolor="#f08080", | |
475 | style=filled, | |
476 | ]; | |
477 | ||
478 | # Switches | |
479 | s1 [ | |
480 | shape=oval, | |
481 | label="s1\n192.168.0.0/24", | |
482 | fillcolor="#d0e0d0", | |
483 | style=filled, | |
484 | ]; | |
485 | s2 [ | |
486 | shape=oval, | |
487 | label="s2\n192.168.1.0/24", | |
488 | fillcolor="#d0e0d0", | |
489 | style=filled, | |
490 | ]; | |
491 | ||
492 | # Connections | |
493 | r1 -- s1 [label="eth0\n.1"]; | |
494 | ||
495 | r1 -- s2 [label="eth1\n.100"]; | |
496 | r2 -- s2 [label="eth0\n.1"]; | |
497 | } | |
498 | ||
499 | Generating / Obtaining Configuration Files | |
500 | """""""""""""""""""""""""""""""""""""""""" | |
501 | ||
502 | In order to get the configuration files or command output for each router, we | |
503 | need to run the topology and execute commands in ``vtysh``. The quickest way to | |
504 | achieve that is writing the topology building code and running the topology. | |
505 | ||
506 | To bootstrap your test topology, do the following steps: | |
507 | ||
508 | - Copy the template test | |
509 | ||
510 | .. code:: shell | |
511 | ||
512 | $ mkdir new-topo/ | |
513 | $ touch new-topo/__init__.py | |
514 | $ cp example-test/test_template.py new-topo/test_new_topo.py | |
515 | ||
516 | - Modify the template according to your dot file | |
517 | ||
518 | Here is the template topology described in the previous section in python code: | |
519 | ||
520 | .. code:: py | |
521 | ||
522 | class TemplateTopo(Topo): | |
523 | "Test topology builder" | |
524 | def build(self, *_args, **_opts): | |
525 | "Build function" | |
526 | tgen = get_topogen(self) | |
527 | ||
528 | # Create 2 routers | |
529 | for routern in range(1, 3): | |
530 | tgen.add_router('r{}'.format(routern)) | |
531 | ||
532 | # Create a switch with just one router connected to it to simulate a | |
533 | # empty network. | |
534 | switch = tgen.add_switch('s1') | |
535 | switch.add_link(tgen.gears['r1']) | |
536 | ||
537 | # Create a connection between r1 and r2 | |
538 | switch = tgen.add_switch('s2') | |
539 | switch.add_link(tgen.gears['r1']) | |
540 | switch.add_link(tgen.gears['r2']) | |
541 | ||
542 | - Run the topology | |
543 | ||
544 | Topogen allows us to run the topology without running any tests, you can do | |
545 | that using the following example commands: | |
546 | ||
547 | .. code:: shell | |
548 | ||
549 | $ # Running your bootstraped topology | |
550 | $ sudo pytest -s --topology-only new-topo/test_new_topo.py | |
551 | $ # Running the test_template.py topology | |
552 | $ sudo pytest -s --topology-only example-test/test_template.py | |
553 | $ # Running the ospf_topo1.py topology | |
554 | $ sudo pytest -s --topology-only ospf-topo1/test_ospf_topo1.py | |
555 | ||
556 | Parameters explanation: | |
557 | ||
558 | .. program:: pytest | |
559 | ||
560 | .. option:: -s | |
561 | ||
562 | Actives input/output capture. This is required by mininet in order to show | |
563 | the interactive shell. | |
564 | ||
565 | .. option:: --topology-only | |
566 | ||
567 | Don't run any tests, just build the topology. | |
568 | ||
569 | After executing the commands above, you should get the following terminal | |
570 | output: | |
571 | ||
572 | .. code:: shell | |
573 | ||
574 | === test session starts === | |
575 | platform linux2 -- Python 2.7.12, pytest-3.1.2, py-1.4.34, pluggy-0.4.0 | |
576 | rootdir: /media/sf_src/topotests, inifile: pytest.ini | |
577 | collected 3 items | |
578 | ||
579 | ospf-topo1/test_ospf_topo1.py *** Starting controller | |
580 | ||
581 | *** Starting 6 switches | |
582 | switch1 switch2 switch3 switch4 switch5 switch6 ... | |
583 | r2: frr zebra started | |
584 | r2: frr ospfd started | |
585 | r3: frr zebra started | |
586 | r3: frr ospfd started | |
587 | r1: frr zebra started | |
588 | r1: frr ospfd started | |
589 | r4: frr zebra started | |
590 | r4: frr ospfd started | |
591 | *** Starting CLI: | |
592 | mininet> | |
593 | ||
594 | The last line shows us that we are now using the Mininet CLI (Command Line | |
595 | Interface), from here you can call your router ``vtysh`` or even bash. | |
596 | ||
597 | Here are some commands example: | |
598 | ||
599 | .. code:: shell | |
600 | ||
601 | mininet> r1 ping 10.0.3.1 | |
602 | PING 10.0.3.1 (10.0.3.1) 56(84) bytes of data. | |
603 | 64 bytes from 10.0.3.1: icmp_seq=1 ttl=64 time=0.576 ms | |
604 | 64 bytes from 10.0.3.1: icmp_seq=2 ttl=64 time=0.083 ms | |
605 | 64 bytes from 10.0.3.1: icmp_seq=3 ttl=64 time=0.088 ms | |
606 | ^C | |
607 | --- 10.0.3.1 ping statistics --- | |
608 | 3 packets transmitted, 3 received, 0% packet loss, time 1998ms | |
609 | rtt min/avg/max/mdev = 0.083/0.249/0.576/0.231 ms | |
610 | ||
611 | ||
612 | ||
613 | mininet> r1 ping 10.0.3.3 | |
614 | PING 10.0.3.3 (10.0.3.3) 56(84) bytes of data. | |
615 | 64 bytes from 10.0.3.3: icmp_seq=1 ttl=64 time=2.87 ms | |
616 | 64 bytes from 10.0.3.3: icmp_seq=2 ttl=64 time=0.080 ms | |
617 | 64 bytes from 10.0.3.3: icmp_seq=3 ttl=64 time=0.091 ms | |
618 | ^C | |
619 | --- 10.0.3.3 ping statistics --- | |
620 | 3 packets transmitted, 3 received, 0% packet loss, time 2003ms | |
621 | rtt min/avg/max/mdev = 0.080/1.014/2.872/1.313 ms | |
622 | ||
623 | ||
624 | ||
625 | mininet> r3 vtysh | |
626 | ||
627 | Hello, this is FRRouting (version 3.1-devrzalamena-build). | |
628 | Copyright 1996-2005 Kunihiro Ishiguro, et al. | |
629 | ||
630 | frr-1# show running-config | |
631 | Building configuration... | |
632 | ||
633 | Current configuration: | |
634 | ! | |
635 | frr version 3.1-devrzalamena-build | |
636 | frr defaults traditional | |
637 | hostname r3 | |
638 | no service integrated-vtysh-config | |
639 | ! | |
640 | log file zebra.log | |
641 | ! | |
642 | log file ospfd.log | |
643 | ! | |
644 | interface r3-eth0 | |
645 | ip address 10.0.3.1/24 | |
646 | ! | |
647 | interface r3-eth1 | |
648 | ip address 10.0.10.1/24 | |
649 | ! | |
650 | interface r3-eth2 | |
651 | ip address 172.16.0.2/24 | |
652 | ! | |
653 | router ospf | |
654 | ospf router-id 10.0.255.3 | |
655 | redistribute kernel | |
656 | redistribute connected | |
657 | redistribute static | |
658 | network 10.0.3.0/24 area 0 | |
659 | network 10.0.10.0/24 area 0 | |
660 | network 172.16.0.0/24 area 1 | |
661 | ! | |
662 | line vty | |
663 | ! | |
664 | end | |
665 | frr-1# | |
666 | ||
667 | After you successfully configured your topology, you can obtain the | |
668 | configuration files (per-daemon) using the following commands: | |
669 | ||
670 | .. code:: shell | |
671 | ||
672 | mininet> r3 vtysh -d ospfd | |
673 | ||
674 | Hello, this is FRRouting (version 3.1-devrzalamena-build). | |
675 | Copyright 1996-2005 Kunihiro Ishiguro, et al. | |
676 | ||
677 | frr-1# show running-config | |
678 | Building configuration... | |
679 | ||
680 | Current configuration: | |
681 | ! | |
682 | frr version 3.1-devrzalamena-build | |
683 | frr defaults traditional | |
684 | no service integrated-vtysh-config | |
685 | ! | |
686 | log file ospfd.log | |
687 | ! | |
688 | router ospf | |
689 | ospf router-id 10.0.255.3 | |
690 | redistribute kernel | |
691 | redistribute connected | |
692 | redistribute static | |
693 | network 10.0.3.0/24 area 0 | |
694 | network 10.0.10.0/24 area 0 | |
695 | network 172.16.0.0/24 area 1 | |
696 | ! | |
697 | line vty | |
698 | ! | |
699 | end | |
700 | frr-1# | |
701 | ||
702 | Writing Tests | |
703 | """"""""""""" | |
704 | ||
705 | Test topologies should always be bootstrapped from | |
706 | :file:`tests/topotests/example-test/test_template.py` because it contains | |
707 | important boilerplate code that can't be avoided, like: | |
708 | ||
709 | - imports: os, sys, pytest, topotest/topogen and mininet topology class | |
710 | - The global variable CWD (Current Working directory): which is most likely | |
711 | going to be used to reference the routers configuration file location | |
712 | ||
713 | Example: | |
714 | ||
715 | .. code:: py | |
716 | ||
717 | # For all registered routers, load the zebra configuration file | |
718 | for rname, router in router_list.iteritems(): | |
719 | router.load_config( | |
720 | TopoRouter.RD_ZEBRA, | |
721 | os.path.join(CWD, '{}/zebra.conf'.format(rname)) | |
722 | ) | |
723 | # os.path.join() joins the CWD string with arguments adding the necessary | |
724 | # slashes ('/'). Arguments must not begin with '/'. | |
725 | ||
726 | - The topology class that inherits from Mininet Topo class: | |
727 | ||
728 | .. code:: py | |
729 | ||
730 | class TemplateTopo(Topo): | |
731 | def build(self, *_args, **_opts): | |
732 | tgen = get_topogen(self) | |
733 | # topology build code | |
734 | ||
735 | - pytest ``setup_module()`` and ``teardown_module()`` to start the topology | |
736 | ||
737 | .. code:: py | |
738 | ||
739 | def setup_module(_m): | |
740 | tgen = Topogen(TemplateTopo) | |
741 | tgen.start_topology('debug') | |
742 | ||
743 | def teardown_module(_m): | |
744 | tgen = get_topogen() | |
745 | tgen.stop_topology() | |
746 | ||
747 | - ``__main__`` initialization code (to support running the script directly) | |
748 | ||
749 | .. code:: py | |
750 | ||
751 | if __name__ == '__main__': | |
752 | sys.exit(pytest.main(["-s"])) | |
753 | ||
754 | Requirements: | |
755 | ||
756 | - Test code should always be declared inside functions that begin with the | |
757 | ``test_`` prefix. Functions beginning with different prefixes will not be run | |
758 | by pytest. | |
759 | - Configuration files and long output commands should go into separated files | |
760 | inside folders named after the equipment. | |
761 | - Tests must be able to run without any interaction. To make sure your test | |
762 | conforms with this, run it without the :option:`-s` parameter. | |
763 | ||
764 | Tips: | |
765 | ||
766 | - Keep results in stack variables, so people inspecting code with ``pdb`` can | |
767 | easily print their values. | |
768 | ||
769 | Don't do this: | |
770 | ||
771 | .. code:: py | |
772 | ||
773 | assert foobar(router1, router2) | |
774 | ||
775 | Do this instead: | |
776 | ||
777 | .. code:: py | |
778 | ||
779 | result = foobar(router1, router2) | |
780 | assert result | |
781 | ||
782 | - Use ``assert`` messages to indicate where the test failed. | |
783 | ||
784 | Example: | |
785 | ||
786 | .. code:: py | |
787 | ||
788 | for router in router_list: | |
789 | # ... | |
790 | assert condition, 'Router "{}" condition failed'.format(router.name) | |
791 | ||
792 | Debugging Execution | |
793 | ^^^^^^^^^^^^^^^^^^^ | |
794 | ||
795 | The most effective ways to inspect topology tests are: | |
796 | ||
797 | - Run pytest with ``--pdb`` option. This option will cause a pdb shell to | |
798 | appear when an assertion fails | |
799 | ||
800 | Example: ``pytest -s --pdb ospf-topo1/test_ospf_topo1.py`` | |
801 | ||
802 | - Set a breakpoint in the test code with ``pdb`` | |
803 | ||
804 | Example: | |
805 | ||
806 | .. code:: py | |
807 | ||
808 | # Add the pdb import at the beginning of the file | |
809 | import pdb | |
810 | # ... | |
811 | ||
812 | # Add a breakpoint where you think the problem is | |
813 | def test_bla(): | |
814 | # ... | |
815 | pdb.set_trace() | |
816 | # ... | |
817 | ||
818 | The `Python Debugger <https://docs.python.org/2.7/library/pdb.html>`__ (pdb) | |
819 | shell allows us to run many useful operations like: | |
820 | ||
821 | - Setting breaking point on file/function/conditions (e.g. ``break``, | |
822 | ``condition``) | |
823 | - Inspecting variables (e.g. ``p`` (print), ``pp`` (pretty print)) | |
824 | - Running python code | |
825 | ||
826 | .. tip:: | |
827 | ||
828 | The TopoGear (equipment abstraction class) implements the ``__str__`` method | |
829 | that allows the user to inspect equipment information. | |
830 | ||
831 | Example of pdb usage: | |
832 | ||
833 | .. code:: shell | |
834 | ||
835 | > /media/sf_src/topotests/ospf-topo1/test_ospf_topo1.py(121)test_ospf_convergence() | |
836 | -> for rnum in range(1, 5): | |
837 | (Pdb) help | |
838 | Documented commands (type help <topic>): | |
839 | ======================================== | |
840 | EOF bt cont enable jump pp run unt | |
841 | a c continue exit l q s until | |
842 | alias cl d h list quit step up | |
843 | args clear debug help n r tbreak w | |
844 | b commands disable ignore next restart u whatis | |
845 | break condition down j p return unalias where | |
846 | ||
847 | Miscellaneous help topics: | |
848 | ========================== | |
849 | exec pdb | |
850 | ||
851 | Undocumented commands: | |
852 | ====================== | |
853 | retval rv | |
854 | ||
855 | (Pdb) list | |
856 | 116 title2="Expected output") | |
857 | 117 | |
858 | 118 def test_ospf_convergence(): | |
859 | 119 "Test OSPF daemon convergence" | |
860 | 120 pdb.set_trace() | |
861 | 121 -> for rnum in range(1, 5): | |
862 | 122 router = 'r{}'.format(rnum) | |
863 | 123 | |
864 | 124 # Load expected results from the command | |
865 | 125 reffile = os.path.join(CWD, '{}/ospfroute.txt'.format(router)) | |
866 | 126 expected = open(reffile).read() | |
867 | (Pdb) step | |
868 | > /media/sf_src/topotests/ospf-topo1/test_ospf_topo1.py(122)test_ospf_convergence() | |
869 | -> router = 'r{}'.format(rnum) | |
870 | (Pdb) step | |
871 | > /media/sf_src/topotests/ospf-topo1/test_ospf_topo1.py(125)test_ospf_convergence() | |
872 | -> reffile = os.path.join(CWD, '{}/ospfroute.txt'.format(router)) | |
873 | (Pdb) print rnum | |
874 | 1 | |
875 | (Pdb) print router | |
876 | r1 | |
877 | (Pdb) tgen = get_topogen() | |
878 | (Pdb) pp tgen.gears[router] | |
879 | <lib.topogen.TopoRouter object at 0x7f74e06c9850> | |
880 | (Pdb) pp str(tgen.gears[router]) | |
881 | 'TopoGear<name="r1",links=["r1-eth0"<->"s1-eth0","r1-eth1"<->"s3-eth0"]> TopoRouter<>' | |
882 | (Pdb) l 125 | |
883 | 120 pdb.set_trace() | |
884 | 121 for rnum in range(1, 5): | |
885 | 122 router = 'r{}'.format(rnum) | |
886 | 123 | |
887 | 124 # Load expected results from the command | |
888 | 125 -> reffile = os.path.join(CWD, '{}/ospfroute.txt'.format(router)) | |
889 | 126 expected = open(reffile).read() | |
890 | 127 | |
891 | 128 # Run test function until we get an result. Wait at most 60 seconds. | |
892 | 129 test_func = partial(compare_show_ip_ospf, router, expected) | |
893 | 130 result, diff = topotest.run_and_expect(test_func, '', | |
894 | (Pdb) router1 = tgen.gears[router] | |
895 | (Pdb) router1.vtysh_cmd('show ip ospf route') | |
896 | '============ OSPF network routing table ============\r\nN 10.0.1.0/24 [10] area: 0.0.0.0\r\n directly attached to r1-eth0\r\nN 10.0.2.0/24 [20] area: 0.0.0.0\r\n via 10.0.3.3, r1-eth1\r\nN 10.0.3.0/24 [10] area: 0.0.0.0\r\n directly attached to r1-eth1\r\nN 10.0.10.0/24 [20] area: 0.0.0.0\r\n via 10.0.3.1, r1-eth1\r\nN IA 172.16.0.0/24 [20] area: 0.0.0.0\r\n via 10.0.3.1, r1-eth1\r\nN IA 172.16.1.0/24 [30] area: 0.0.0.0\r\n via 10.0.3.1, r1-eth1\r\n\r\n============ OSPF router routing table =============\r\nR 10.0.255.2 [10] area: 0.0.0.0, ASBR\r\n via 10.0.3.3, r1-eth1\r\nR 10.0.255.3 [10] area: 0.0.0.0, ABR, ASBR\r\n via 10.0.3.1, r1-eth1\r\nR 10.0.255.4 IA [20] area: 0.0.0.0, ASBR\r\n via 10.0.3.1, r1-eth1\r\n\r\n============ OSPF external routing table ===========\r\n\r\n\r\n' | |
897 | (Pdb) tgen.mininet_cli() | |
898 | *** Starting CLI: | |
899 | mininet> | |
900 | ||
901 | To enable more debug messages in other Topogen subsystems (like Mininet), more | |
902 | logging messages can be displayed by modifying the test configuration file | |
903 | ``pytest.ini``: | |
904 | ||
905 | .. code:: ini | |
906 | ||
907 | [topogen] | |
908 | # Change the default verbosity line from 'info'... | |
909 | #verbosity = info | |
910 | # ...to 'debug' | |
911 | verbosity = debug | |
912 | ||
913 | Instructions for use, write or debug topologies can be found in :ref:`topotests-guidelines`. | |
914 | To learn/remember common code snippets see :ref:`topotests-snippets`. | |
915 | ||
916 | Before creating a new topology, make sure that there isn't one already that | |
917 | does what you need. If nothing is similar, then you may create a new topology, | |
918 | preferably, using the newest template | |
919 | (:file:`tests/topotests/example-test/test_template.py`). | |
920 | ||
921 | .. include:: topotests-snippets.rst | |
922 | ||
923 | License | |
924 | ------- | |
925 | ||
926 | All the configs and scripts are licensed under a ISC-style license. See Python | |
927 | scripts for details. |