]>
Commit | Line | Data |
---|---|---|
8a6b34c2 | 1 | .. _topotests-json: |
eb0a7b65 | 2 | |
8a6b34c2 QY |
3 | Topotests with JSON |
4 | =================== | |
eb0a7b65 AP |
5 | |
6 | Overview | |
8a6b34c2 | 7 | -------- |
eb0a7b65 AP |
8 | |
9 | On top of current topotests framework following enhancements are done: | |
10 | ||
11 | ||
8a6b34c2 QY |
12 | * Creating the topology and assigning IPs to router' interfaces dynamically. |
13 | It is achieved by using json file, in which user specify the number of | |
14 | routers, links to each router, interfaces for the routers and protocol | |
15 | configurations for all routers. | |
eb0a7b65 | 16 | |
8a6b34c2 QY |
17 | * Creating the configurations dynamically. It is achieved by using |
18 | :file:`/usr/lib/frr/frr-reload.py` utility, which takes running configuration | |
19 | and the newly created configuration for any particular router and creates a | |
20 | delta file(diff file) and loads it to router. | |
eb0a7b65 AP |
21 | |
22 | ||
23 | Logging of test case executions | |
8a6b34c2 | 24 | ------------------------------- |
eb0a7b65 | 25 | |
36c19497 CH |
26 | * The execution log for each test is saved in the test specific directory create |
27 | under `/tmp/topotests` (e.g., | |
28 | `/tmp/topotests/<testdirname.testfilename>/exec.log`) | |
eb0a7b65 | 29 | |
36c19497 CH |
30 | * Additionally all test logs are captured in the `topotest.xml` results file. |
31 | This file will be saved in `/tmp/topotests/topotests.xml`. In order to extract | |
32 | the logs for a particular test one can use the `analyze.py` utility found in | |
33 | the topotests base directory. | |
b41f3f0a | 34 | |
36c19497 CH |
35 | * Router's current configuration, as it is changed during the test, can be |
36 | displayed on console or sent to logs by adding ``show_router_config = True`` in | |
37 | :file:`pytest.ini`. | |
eb0a7b65 AP |
38 | |
39 | Note: directory "/tmp/topotests/" is created by topotests by default, making | |
40 | use of same directory to save execution logs. | |
41 | ||
eb0a7b65 | 42 | Guidelines |
8a6b34c2 | 43 | ---------- |
eb0a7b65 AP |
44 | |
45 | Writing New Tests | |
8a6b34c2 | 46 | ^^^^^^^^^^^^^^^^^ |
eb0a7b65 | 47 | |
b41f3f0a QY |
48 | This section will guide you in all recommended steps to produce a standard |
49 | topology test. | |
eb0a7b65 AP |
50 | |
51 | This is the recommended test writing routine: | |
52 | ||
36c19497 CH |
53 | * Create a json file which will have routers and protocol configurations |
54 | * Write and debug the tests | |
d9c43f8f | 55 | * Format the new code using `black <https://github.com/psf/black>`_ |
eb0a7b65 AP |
56 | * Create a Pull Request |
57 | ||
d9c43f8f MS |
58 | .. Note:: |
59 | ||
36c19497 CH |
60 | BGP tests MUST use generous convergence timeouts - you must ensure that any |
61 | test involving BGP uses a convergence timeout that is proportional to the | |
62 | configured BGP timers. If the timers are not reduced from their defaults this | |
63 | means 130 seconds; however, it is highly recommended that timers be reduced | |
64 | from the default values unless the test requires they not be. | |
8a6b34c2 | 65 | |
eb0a7b65 | 66 | File Hierarchy |
8a6b34c2 | 67 | ^^^^^^^^^^^^^^ |
eb0a7b65 AP |
68 | |
69 | Before starting to write any tests one must know the file hierarchy. The | |
70 | repository hierarchy looks like this: | |
71 | ||
8a6b34c2 QY |
72 | .. code-block:: console |
73 | ||
36c19497 | 74 | $ cd frr/tests/topotests |
8a6b34c2 QY |
75 | $ find ./* |
76 | ... | |
36c19497 CH |
77 | ./example_test/ |
78 | ./example_test/test_template_json.json # input json file, having topology, interfaces, bgp and other configuration | |
79 | ./example_test/test_template_json.py # test script to write and execute testcases | |
8a6b34c2 QY |
80 | ... |
81 | ./lib # shared test/topology functions | |
36c19497 CH |
82 | ./lib/topojson.py # library to create topology and configurations dynamically from json file |
83 | ./lib/common_config.py # library to create protocol's common configurations ex- static_routes, prefix_lists, route_maps etc. | |
84 | ./lib/bgp.py # library to create and test bgp configurations | |
eb0a7b65 AP |
85 | |
86 | Defining the Topology and initial configuration in JSON file | |
b41f3f0a | 87 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
eb0a7b65 AP |
88 | |
89 | The first step to write a new test is to define the topology and initial | |
90 | configuration. User has to define topology and initial configuration in JSON | |
b41f3f0a QY |
91 | file. Here is an example of JSON file:: |
92 | ||
0dc861fa | 93 | BGP neighborship with single phy-link, sample JSON file: |
b41f3f0a QY |
94 | { |
95 | "ipv4base": "192.168.0.0", | |
96 | "ipv4mask": 30, | |
97 | "ipv6base": "fd00::", | |
98 | "ipv6mask": 64, | |
99 | "link_ip_start": {"ipv4": "192.168.0.0", "v4mask": 30, "ipv6": "fd00::", "v6mask": 64}, | |
100 | "lo_prefix": {"ipv4": "1.0.", "v4mask": 32, "ipv6": "2001:DB8:F::", "v6mask": 128}, | |
101 | "routers": { | |
102 | "r1": { | |
103 | "links": { | |
104 | "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, | |
105 | "r2": {"ipv4": "auto", "ipv6": "auto"}, | |
106 | "r3": {"ipv4": "auto", "ipv6": "auto"} | |
107 | }, | |
108 | "bgp": { | |
109 | "local_as": "64512", | |
110 | "address_family": { | |
111 | "ipv4": { | |
112 | "unicast": { | |
113 | "neighbor": { | |
114 | "r2": { | |
115 | "dest_link": { | |
116 | "r1": {} | |
117 | } | |
118 | }, | |
119 | "r3": { | |
120 | "dest_link": { | |
121 | "r1": {} | |
122 | } | |
123 | } | |
124 | } | |
125 | } | |
126 | } | |
127 | } | |
128 | } | |
129 | }, | |
130 | "r2": { | |
131 | "links": { | |
132 | "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, | |
133 | "r1": {"ipv4": "auto", "ipv6": "auto"}, | |
134 | "r3": {"ipv4": "auto", "ipv6": "auto"} | |
135 | }, | |
136 | "bgp": { | |
137 | "local_as": "64512", | |
138 | "address_family": { | |
139 | "ipv4": { | |
140 | "unicast": { | |
141 | "redistribute": [ | |
142 | { | |
143 | "redist_type": "static" | |
144 | } | |
145 | ], | |
146 | "neighbor": { | |
147 | "r1": { | |
148 | "dest_link": { | |
149 | "r2": {} | |
150 | } | |
151 | }, | |
152 | "r3": { | |
153 | "dest_link": { | |
154 | "r2": {} | |
155 | } | |
156 | } | |
157 | } | |
158 | } | |
159 | } | |
160 | } | |
161 | } | |
162 | } | |
163 | ... | |
164 | ||
165 | ||
166 | BGP neighboship with loopback interface, sample JSON file:: | |
167 | ||
168 | { | |
169 | "ipv4base": "192.168.0.0", | |
170 | "ipv4mask": 30, | |
171 | "ipv6base": "fd00::", | |
172 | "ipv6mask": 64, | |
173 | "link_ip_start": {"ipv4": "192.168.0.0", "v4mask": 30, "ipv6": "fd00::", "v6mask": 64}, | |
174 | "lo_prefix": {"ipv4": "1.0.", "v4mask": 32, "ipv6": "2001:DB8:F::", "v6mask": 128}, | |
175 | "routers": { | |
176 | "r1": { | |
177 | "links": { | |
178 | "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback", | |
179 | "add_static_route":"yes"}, | |
180 | "r2": {"ipv4": "auto", "ipv6": "auto"} | |
181 | }, | |
182 | "bgp": { | |
183 | "local_as": "64512", | |
184 | "address_family": { | |
185 | "ipv4": { | |
186 | "unicast": { | |
187 | "neighbor": { | |
188 | "r2": { | |
189 | "dest_link": { | |
190 | "lo": { | |
191 | "source_link": "lo" | |
192 | } | |
193 | } | |
194 | } | |
195 | } | |
196 | } | |
197 | } | |
198 | } | |
199 | }, | |
200 | "static_routes": [ | |
201 | { | |
202 | "network": "1.0.2.17/32", | |
203 | "next_hop": "192.168.0.1 | |
204 | } | |
205 | ] | |
206 | }, | |
207 | "r2": { | |
208 | "links": { | |
209 | "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback", | |
210 | "add_static_route":"yes"}, | |
211 | "r1": {"ipv4": "auto", "ipv6": "auto"}, | |
212 | "r3": {"ipv4": "auto", "ipv6": "auto"} | |
213 | }, | |
214 | "bgp": { | |
215 | "local_as": "64512", | |
216 | "address_family": { | |
217 | "ipv4": { | |
218 | "unicast": { | |
219 | "redistribute": [ | |
220 | { | |
221 | "redist_type": "static" | |
222 | } | |
223 | ], | |
224 | "neighbor": { | |
225 | "r1": { | |
226 | "dest_link": { | |
227 | "lo": { | |
228 | "source_link": "lo" | |
229 | } | |
230 | } | |
231 | }, | |
232 | "r3": { | |
233 | "dest_link": { | |
234 | "lo": { | |
235 | "source_link": "lo" | |
236 | } | |
237 | } | |
238 | } | |
239 | } | |
240 | } | |
241 | } | |
242 | } | |
243 | }, | |
244 | "static_routes": [ | |
245 | { | |
246 | "network": "192.0.20.1/32", | |
247 | "no_of_ip": 9, | |
248 | "admin_distance": 100, | |
249 | "next_hop": "192.168.0.1", | |
250 | "tag": 4001 | |
251 | } | |
252 | ], | |
253 | } | |
254 | ... | |
255 | ||
256 | BGP neighborship with Multiple phy-links, sample JSON file:: | |
257 | ||
258 | { | |
259 | "ipv4base": "192.168.0.0", | |
260 | "ipv4mask": 30, | |
261 | "ipv6base": "fd00::", | |
262 | "ipv6mask": 64, | |
263 | "link_ip_start": {"ipv4": "192.168.0.0", "v4mask": 30, "ipv6": "fd00::", "v6mask": 64}, | |
264 | "lo_prefix": {"ipv4": "1.0.", "v4mask": 32, "ipv6": "2001:DB8:F::", "v6mask": 128}, | |
265 | "routers": { | |
266 | "r1": { | |
267 | "links": { | |
268 | "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, | |
269 | "r2-link1": {"ipv4": "auto", "ipv6": "auto"}, | |
270 | "r2-link2": {"ipv4": "auto", "ipv6": "auto"} | |
271 | }, | |
272 | "bgp": { | |
273 | "local_as": "64512", | |
274 | "address_family": { | |
275 | "ipv4": { | |
276 | "unicast": { | |
277 | "neighbor": { | |
278 | "r2": { | |
279 | "dest_link": { | |
280 | "r1-link1": {} | |
281 | } | |
282 | } | |
283 | } | |
284 | } | |
285 | } | |
286 | } | |
287 | } | |
288 | }, | |
289 | "r2": { | |
290 | "links": { | |
291 | "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, | |
292 | "r1-link1": {"ipv4": "auto", "ipv6": "auto"}, | |
293 | "r1-link2": {"ipv4": "auto", "ipv6": "auto"}, | |
294 | "r3-link1": {"ipv4": "auto", "ipv6": "auto"}, | |
295 | "r3-link2": {"ipv4": "auto", "ipv6": "auto"} | |
296 | }, | |
297 | "bgp": { | |
298 | "local_as": "64512", | |
299 | "address_family": { | |
300 | "ipv4": { | |
301 | "unicast": { | |
302 | "redistribute": [ | |
303 | { | |
304 | "redist_type": "static" | |
305 | } | |
306 | ], | |
307 | "neighbor": { | |
308 | "r1": { | |
309 | "dest_link": { | |
310 | "r2-link1": {} | |
311 | } | |
312 | }, | |
313 | "r3": { | |
314 | "dest_link": { | |
315 | "r2-link1": {} | |
316 | } | |
317 | } | |
318 | } | |
319 | } | |
320 | } | |
321 | } | |
322 | } | |
323 | } | |
324 | ... | |
325 | ||
326 | ||
8a6b34c2 | 327 | JSON File Explained |
eb0a7b65 AP |
328 | """"""""""""""""""" |
329 | ||
330 | Mandatory keywords/options in JSON: | |
331 | ||
b41f3f0a QY |
332 | * ``ipv4base`` : base ipv4 address to generate ips, ex - 192.168.0.0 |
333 | * ``ipv4mask`` : mask for ipv4 address, ex - 30 | |
334 | * ``ipv6base`` : base ipv6 address to generate ips, ex - fd00: | |
335 | * ``ipv6mask`` : mask for ipv6 address, ex - 64 | |
336 | * ``link_ip_start`` : physical interface base ipv4 and ipv6 address | |
337 | * ``lo_prefix`` : loopback interface base ipv4 and ipv6 address | |
338 | * ``routers`` : user can add number of routers as per topology, router's name | |
eb0a7b65 | 339 | can be any logical name, ex- r1 or a0. |
b41f3f0a QY |
340 | * ``r1`` : name of the router |
341 | * ``lo`` : loopback interface dict, ipv4 and/or ipv6 addresses generated automatically | |
342 | * ``type`` : type of interface, to identify loopback interface | |
343 | * ``links`` : physical interfaces dict, ipv4 and/or ipv6 addresses generated | |
eb0a7b65 | 344 | automatically |
b41f3f0a | 345 | * ``r2-link1`` : it will be used when routers have multiple links. 'r2' is router |
eb0a7b65 | 346 | name, 'link' is any logical name, '1' is to identify link number, |
b41f3f0a | 347 | router name and link must be seperated by hyphen (``-``), ex- a0-peer1 |
eb0a7b65 AP |
348 | |
349 | Optional keywords/options in JSON: | |
350 | ||
b41f3f0a QY |
351 | * ``bgp`` : bgp configuration |
352 | * ``local_as`` : Local AS number | |
353 | * ``unicast`` : All SAFI configuration | |
354 | * ``neighbor``: All neighbor details | |
355 | * ``dest_link`` : Destination link to which router will connect | |
356 | * ``router_id`` : bgp router-id | |
357 | * ``source_link`` : if user wants to establish bgp neighborship with loopback | |
358 | interface, add ``source_link``: ``lo`` | |
359 | * ``keepalivetimer`` : Keep alive timer for BGP neighbor | |
360 | * ``holddowntimer`` : Hold down timer for BGP neighbor | |
361 | * ``static_routes`` : create static routes for routers | |
362 | * ``redistribute`` : redistribute static and/or connected routes | |
363 | * ``prefix_lists`` : create Prefix-lists for routers | |
eb0a7b65 AP |
364 | |
365 | Building topology and configurations | |
366 | """""""""""""""""""""""""""""""""""" | |
367 | ||
36c19497 CH |
368 | Topology and initial configuration as well as teardown are invoked through the |
369 | use of a pytest fixture:: | |
eb0a7b65 | 370 | |
77f3acb4 | 371 | |
36c19497 | 372 | from lib import fixtures |
eb0a7b65 | 373 | |
36c19497 | 374 | tgen = pytest.fixture(fixtures.tgen_json, scope="module") |
eb0a7b65 | 375 | |
eb0a7b65 | 376 | |
36c19497 CH |
377 | # tgen is defined above |
378 | # topo is a fixture defined in ../conftest.py and automatically available | |
379 | def test_bgp_convergence(tgen, topo): | |
380 | bgp_convergence = bgp.verify_bgp_convergence(tgen, topo) | |
381 | assert bgp_convergence | |
eb0a7b65 | 382 | |
36c19497 CH |
383 | The `fixtures.topo_json` function calls `topojson.setup_module_from_json()` to |
384 | create and return a new `topogen.Topogen()` object using the JSON config file | |
385 | with the same base filename as the test (i.e., `test_file.py` -> | |
386 | `test_file.json`). Additionally, the fixture calls `tgen.stop_topology()` after | |
387 | all the tests have run to cleanup. The function is only invoked once per | |
388 | file/module (scope="module"), but the resulting object is passed to each | |
389 | function that has `tgen` as an argument. | |
eb0a7b65 | 390 | |
36c19497 | 391 | For more info on the powerful pytest fixtures feature please see `FIXTURES`_. |
eb0a7b65 | 392 | |
36c19497 | 393 | .. _FIXTURES: https://docs.pytest.org/en/6.2.x/fixture.html |
eb0a7b65 AP |
394 | |
395 | Creating configuration files | |
396 | """""""""""""""""""""""""""" | |
397 | ||
398 | Router's configuration would be saved in config file frr_json.conf. Common | |
399 | configurations are like, static routes, prefixlists and route maps etc configs, | |
400 | these configs can be used by any other protocols as it is. | |
401 | BGP config will be specific to BGP protocol testing. | |
402 | ||
77f3acb4 CH |
403 | * json file is passed to API Topogen() which saves the JSON object in |
404 | `self.json_topo` | |
405 | * The Topogen object is then passed to API build_config_from_json(), which looks | |
406 | for configuration tags in new JSON object. | |
407 | * If tag is found in the JSON object, configuration is created as per input and | |
408 | written to file frr_json.conf | |
eb0a7b65 AP |
409 | * Once JSON parsing is over, frr_json.conf is loaded onto respective router. |
410 | Config loading is done using 'vtysh -f <file>'. Initial config at this point | |
411 | is also saved frr_json_initial.conf. This file can be used to reset | |
412 | configuration on router, during the course of execution. | |
413 | * Reset of configuration is done using frr "reload.py" utility, which | |
414 | calculates the difference between router's running config and user's config | |
415 | and loads delta file to router. API used - reset_config_on_router() | |
416 | ||
417 | Writing Tests | |
418 | """"""""""""" | |
419 | ||
420 | Test topologies should always be bootstrapped from the | |
36c19497 CH |
421 | `example_test/test_template_json.py` when possible in order to take advantage of |
422 | the most recent infrastructure support code. | |
eb0a7b65 AP |
423 | |
424 | Example: | |
425 | ||
426 | ||
36c19497 CH |
427 | * Define a module scoped fixture to setup/teardown and supply the tests with the |
428 | `Topogen` object. | |
eb0a7b65 | 429 | |
36c19497 | 430 | .. code-block:: python |
eb0a7b65 | 431 | |
36c19497 CH |
432 | import pytest |
433 | from lib import fixtures | |
eb0a7b65 | 434 | |
36c19497 | 435 | tgen = pytest.fixture(fixtures.tgen_json, scope="module") |
eb0a7b65 | 436 | |
eb0a7b65 | 437 | |
36c19497 | 438 | * Define test functions using pytest fixtures |
eb0a7b65 | 439 | |
36c19497 | 440 | .. code-block:: python |
eb0a7b65 | 441 | |
36c19497 | 442 | from lib import bgp |
eb0a7b65 | 443 | |
36c19497 CH |
444 | # tgen is defined above |
445 | # topo is a global available fixture defined in ../conftest.py | |
446 | def test_bgp_convergence(tgen, topo): | |
447 | "Test for BGP convergence." | |
8a6b34c2 | 448 | |
36c19497 CH |
449 | # Don't run this test if we have any failure. |
450 | if tgen.routers_have_failure(): | |
451 | pytest.skip(tgen.errors) | |
eb0a7b65 | 452 | |
36c19497 CH |
453 | bgp_convergence = bgp.verify_bgp_convergence(tgen, topo) |
454 | assert bgp_convergence |