bgp_l3vpn_to_bgp_vrf: skip tests when mpls not available

Signed-off-by: Lou Berger <lberger@labn.net>

lib: add script name to summary output

Signed-off-by: Lou Berger <lberger@labn.net>

lib: add luLast to get last found luCommand regex match

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_direct: allow for some ping loss

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_vrf: allow for some ping loss

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_vrf: add basic data path checks prior to adjacency check

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_direct: add basic data path checks prior to adjacency check

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_xxx: match ltemplate start/stop changes

Signed-off-by: Lou Berger <lberger@labn.net>

lib: ltemplate simplify start/stop

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_xxx: avoid collision of customize.py match lutil and ltemplate changes

Signed-off-by: Lou Berger <lberger@labn.net>

lib: scope life of Lutil and _lt to a single test run reorg to support regression environment

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_vrf: just test adjacencies now - for dev/integration

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_vrf: don't set vrf-policy, wait for new format

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_vrf: add ce4 off of r4 for inter-vrf route leaking

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_vrf: in progress version Adjacenies up, MPLS configured on interfaces -- right now provides testbed for code changes

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_vrf: clone bgp_l3vpn_to_bgp_direct as starting point

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_direct: 2nd try at skipping tests when MPLS not available

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_direct: skip tests when mpls not available

Signed-off-by: Lou Berger <lberger@labn.net>

lib: add router.has_mpls check

Signed-off-by: Lou Berger <lberger@labn.net>

topogen: fix assertion on `routers_have_failure`

`errors != ''` is never going to be false so the assert was never being
triggered.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

topogen: import pytest assert function

Use the pytest assert function to be able to report to pytest failures
that happened inside the API.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

isis-topo1: check IPv6 linux route install

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

isis-topo1: simplify dictionary key removal code

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

isis-topo1: check ipv6 zebra route installation

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

isis-topo1: skip test on FRR 2.0

It was noticed that this topology doesn't converge when there are IPv6
configurations, so this commit makes the tests to be skipped when the
version is 2.X.Y.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org

isis-topo1: add IPv6 ISIS topology test

Configure IPv6 addresses and ISIS distribution, then test the topology
for the newly configured addresses.

This commit also adds support for parsing IPv6 data from the ISIS
topology output while updating the routers topology JSON files.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

isis-topo1: add support for stable versions

Allow isis-topo1 tests to run on stable versions (2.0.2 and 3.0.2).

There are two differences between the stable versions and master:
1. ISIS nexthops don't show interfaceName nor interfaceIndex
   (on 'show ip route json')
2. New FRR version install ISIS routes in Linux with a different proto
   type identification.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

isis-topo1: check ISIS kernel route installation

Check if the ISIS learned routes are being installed in the underlying
OS.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

isis-topo1: test ISIS route installation

Check the 'show ip route json' output for installed ISIS routes.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

isis-topo1: test ISIS topology convergence

Add function to parse 'show isis topology' and expect the correct
convergence result.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

isis-topo1: add topology graph

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

isis-topo1: import topology

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

ldp-vpls-topo1: new format for ospf neighbor json

Accept new format in ospf neighbor json output in addition to
old version. New format was introduced with PR1659

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

lib: ltemplate - fix handling of CallOnFail (take 2)

Signed-off-by: Lou Berger <lberger@labn.net>

lib: ltemplate - fix handling of CallOnFail

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_direct: fix hook routines for ltemplate

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_l3vpn_to_bgp_direct: rename bgp_direct_to_bgp_vpn Rename BGP L3VPN test to have L3VPN at start of test name Also remove unused/pre-ltemplate files

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_direct_to_bgp_vpn: enable mpls forwarding, add more details on adjacencies

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_direct_to_bgp_vpn: convert to lib/ltemplate

Signed-off-by: Lou Berger <lberger@labn.net>

lib: add LabN testing template

Signed-off-by: Lou Berger <lberger@labn.net>

ospf-topo1: Allow IPv6 ECMP version to pass

Check ospf IPv6 routes for ECMP version first and fallback
to previous version if needed.

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

ospf-topo1: Add IPv6 addresses to diagram

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

lib: lutil - make it easier to spot command output and null output

Signed-off-by: Lou Berger <lberger@labn.net>

lib: lutil - make requested error logging change

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_direct_to_bgp_vpn: scripts - avoid race condition, and test for predicable results

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_direct_to_bgp_vpn: add some progress output

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_direct_to_bgp_vpn: general cleanup use moved lutil.py improve error information adjust to new luCommand default timeout sdd support for CliOnFail, remove no_test_call_mininet_cli scripts - update to correct stable values use topo test's log dir

Signed-off-by: Lou Berger <lberger@labn.net>

lib: lutil cleanup move utilities/lutil.py -> lib/lutil.py add: luShowFail, CallOnFail, luStart parameter to set log dir change default wait to 10sec

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_direct_to_bgp_vpn: convert to new template, skip tests on pre-3.1 versions

Signed-off-by: Lou Berger <lberger@labn.net>

bgp_direct_to_bgp_vpn: initial version

Signed-off-by: Lou Berger <lberger@labn.net>

lutil: initial version

Signed-off-by: Lou Berger <lberger@labn.net>

topotest: remove pid files

Remove PID files after a SIGKILL or confirmation that the pid file no
longer contains a running PID.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

topotest: improve informational messages

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

topotest: show the user that we are waiting

Add a log message to tell the user that we are waiting for zebra.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>

lib: speed up router shutdown stopRouter: report when a process is being killed only sleep if actually killed a process add option to not sleep or conduct -7 kill topogen stop: add wait parameter (defaults to old/wait) when stopping routers, first stop all without waiting then do a second pass where will wait if needed

Signed-off-by: Lou Berger <lberger@labn.net>

topotest: always show diff on json_cmp failure

Append diffs on all json_cmp failures so the reader can compare JSON
field differences.

topotest: specify why we are waiting

This shows the user why the router teardown is taking 2 seconds long.

doc: update README and add code snippets

Add some pointers in the README documentation that might help users get
ready to use topotest.

topotest/topogen: demote some messages level

The messages level of this commit were downgraded because some of them
happen on 'non-error' situations. This should help diminish the error
log verbosity on the CI-system run.

topotest: fix a regression in version_cmp

It was found a regression on an edge case when the second number in the
comparison was (at least) 2 numbers longer the comparison would fail
with a wrong return value. It succeeded for some cases because the
first comparison in the exception was correct, but not the second.

bgp-ecmp-topo1: Workaround for version check

temp workaround for Issue #43

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

ldp-vpls-topo1: Allow ospf neighbor json command with list (new) and dict (old)

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

ldp-vpls-topo1: fix a json_cmp result assert

Use the 'right' assert to show more detailed failure report and remove
a unused variable.

Fix output to allow it on generic Debian

- Generic Debian only has a single space in front of “proto” in the linux shell routing outptu

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

eigrp-topo1: solve output ordering problem

Transform 'show ip eigrp topo' output into data structures and compare
using json_cmp() to avoid expecting output order.

ldp-topo1: Remove check for protocol in installed LSPs

PR #1213 in FRR changed the protocol of installed LSPs. To avoid breaking
older outstanding Pull Requests, remove the protocol check.

Signed-off-by: Renato Westphal <renato@opensourcerouting.org>

eigrp-topo1: Remove check for EIGRP distance

- Distance used to be wrong (0), new commit fixes this (90). To avoid breaking older outstanding Pull Requests, remove the distance check

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

topotest: improve json error messages

Show a diff of the JSON values instead of dumping the whole
data structures.

topotest: json_cmp_result split error lines

When the API user wanted to show newlines we have to break them manually
to get the propper format.

topotest: allow passing options to difflib

Extend the topotest diff functions to allow receiving difflib options.

topotest: remove unused json_cmp unused parameter

all-protocol-startup/isis: Fix check for "show isis interface" to allow different circuit ids than 1

topogen: fix diagnostics failure

Don't try to log to file if directory doesn't exist.

topogen: save zebra version output

Keep the zebra version output for later problem diagnostics.

topogen: log diagnostics to file

eigrp-topo1: Cleanup leftover comments from previous commit

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

eigrp-topo1: Change Route verification to use json

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

all-protocol-starup: Fix triple-output of same vtysh stderr/stdout messages.

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

lib: Use SIGTERM for killing daemons first and only SIGBUS (7) if the daemons fail to exit.

- This allows daemons to free up memory and avoid false memory leak reports

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

multiple_topos: Don't report about StdErr Output if there is no output

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

eigrp-topo1: Correct eigrp test results.

The values saved for the route table and eigrp topo
were a bit off because they have been corrected
in the eigrp daemon

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>

Enable eigrp-topo1

Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>

lib: check for eigrpd available and return warning if missing instead of failing

eigrp-topo1: convert to topogen

topogen: add support for eigrpd and nhrp

topogen: router startup failure triggers errors

Makes `router_have_failure()` detect frr startup errors.

topogen: treat some corner cases on vtysh failure

Don't throw unneeded tracebacks when 'show version' doesn't show us what
we expect, instead gracefully fail.

topogen: show pretty output for skipped messages

Format them properly instead of printing a dictionary.

ldp_vpls_topo1: set an error instead of exit

It will have the same effect and code will not have to be repeated since
all functions must call `routers_have_failure()`.

ldp_vpls_topo1: don't use pytest.mark.skipif

This feature from pytest can not be used to detect runtime errors.

Also, remove test_router_running() since all tests should actually check
if all daemons are up and running.

Signed-off-by: Renato Westphal <renato@opensourcerouting.org>

ldp_vpls_topo1: skip all tests on stable/2.0

Signed-off-by: Renato Westphal <renato@opensourcerouting.org>

ldp_vpls_topo1: add new topology for ldp pseudowires

This is a very simple topology for testing LDP pseudowires.

Signed-off-by: Renato Westphal <renato@opensourcerouting.org>

Basic EIGRP topo test working

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>

Exclude eigrp test from running normally yet

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>

Add a basic EIGRP topology.

Nothing fancy here, just add 3 routers in a row
attempt to let eigrp come up and start a mininet
xterm to debug, since eigrp doesn't work yet.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>

Allow topotests to work with eigrp and nhrp

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>

ldp-topo1: Use 'label implicit-null' for implicit labels

Signed-off-by: Martin Winter <mwinter@netdef.org>

topotests: change from "context" to "unified" diff

context diff:
*** before.py

topotests: add topology name to assert output

Make it easy to identify which topology test we are running.

topotests: make asserts show up in stderr

Code was based on the pytest default makereport code:
https://github.com/pytest-dev/pytest/blob/c92760dca8637251eb9e7b9ea4819b32bc0b8042/_pytest/runner.py#L264