ovs-macros: An option to suspend test execution on error

Origins for this patch are captured at
https://mail.openvswitch.org/pipermail/ovs-discuss/2019-June/048923.html.

Summarizing here, when a test fails, it would be good to pause test execution
and let the developer poke around the system to see current status of system.

As part of this patch, made a small tweaks to ovs-macros.at, so that when test
suite fails, ovs_on_exit() function will be called. And in this function, a check
is made to see if an environment variable to OVS_PAUSE_TEST is set. If it is
set, then test suite is paused and will continue to wait for user input
Ctrl-D. Meanwhile user can poke around the system to see why test case has
failed. Once done with investigation, user can press ctrl-d to cleanup the
test suite.

For example, to re-run test case 139:

export OVS_PAUSE_TEST=1
cd tests/system-userspace-testsuite.dir/139
sudo -E ./run

When error occurs, above command would display something like this:
=====================================================
Set environment variable to use various ovs utilities
export OVS_RUNDIR=/opt/vdasari/Developer/ovs/_build-gcc/tests/system-userspace-testsuite.dir/139
Press ENTER to continue:

=====================================================
And from another window, one can execute ovs-xxx commands like:
export OVS_RUNDIR=/opt/vdasari/Developer/ovs/_build-gcc/tests/system-userspace-testsuite.dir/139
$ ovs-ofctl dump-ports br0
.
.

To be able to pause while performing `make check`, one can do:
$ OVS_PAUSE_TEST=1 make check TESTSUITEFLAGS='-v'

Acked-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Vasu Dasari <vdasari@gmail.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>

diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst
index a8892e1c1c33a77c97f230d0605a2cd7c4d763b5..eef61e5d5847b44bdf700bae999a5f56b1aa6b47 100644 (file)
--- a/Documentation/topics/testing.rst
+++ b/Documentation/topics/testing.rst
@@ -90,6 +90,30 @@ report test failures as bugs and include the ``testsuite.log`` in your report.
 
       $ make check TESTSUITEFLAGS=-j8 RECHECK=yes
 
+Debugging unit tests
+++++++++++++++++++++
+
+To initiate debugging from artifacts generated from `make check` run, set the
+``OVS_PAUSE_TEST`` environment variable to 1.  For example, to run test case
+139 and pause on error::
+
+  $ OVS_PAUSE_TEST=1 make check TESTSUITEFLAGS='-v 139'
+
+When error occurs, above command would display something like this::
+
+   Set environment variable to use various ovs utilities
+   export OVS_RUNDIR=<dir>/ovs/_build-gcc/tests/testsuite.dir/0139
+   Press ENTER to continue:
+
+And from another window, one can execute ovs-xxx commands like::
+
+   export OVS_RUNDIR=/opt/vdasari/Developer/ovs/_build-gcc/tests/testsuite.dir/0139
+   $ ovs-ofctl dump-ports br0
+   .
+   .
+
+Once done with investigation, press ENTER to perform cleanup operation.
+
 .. _testing-coverage:
 
 Coverage
@@ -297,6 +321,9 @@ To invoke the datapath testsuite with the userspace datapath, run::
 
 The results of the testsuite are in ``tests/system-userspace-testsuite.dir``.
 
+All the features documented under `Unit Tests`_ are available for the userspace
+datapath testsuite.
+
 DPDK datapath
 '''''''''''''
 
@@ -328,6 +355,9 @@ The phy test will skip if no compatible physical device is available.
 
 .. _Configure hugepages: http://doc.dpdk.org/guides/linux_gsg/sys_reqs.html
 
+All the features documented under `Unit Tests`_ are available for the DPDK
+datapath testsuite.
+
 Kernel datapath
 '''''''''''''''
 
@@ -348,6 +378,9 @@ testsuite against that kernel module::
 
 The results of the testsuite are in ``tests/system-kmod-testsuite.dir``.
 
+All the features documented under `Unit Tests`_ are available for the kernel
+datapath testsuite.
+
 .. _testing-static-analysis:
 
 Static Code Analysis

diff --git a/tests/ovs-macros.at b/tests/ovs-macros.at
index 10593429d9f7dc24b3f416d35bc9da7dd6abd0ba..b6add7fda5e15551bde9bd5644be64765880161f 100644 (file)
--- a/tests/ovs-macros.at
+++ b/tests/ovs-macros.at
@@ -35,11 +35,33 @@ m4_divert_push([PREPARE_TESTS])
 # directory.
 ovs_init() {
     ovs_base=`pwd`
-    trap '. "$ovs_base/cleanup"' 0
+    trap ovs_on_exit 0
     : > cleanup
     ovs_setenv
 }
 
+# Catch testsuite error condition and cleanup test environment by tearing down
+# all interfaces and processes spawned.
+# User has an option to leave the test environment in error state so that system
+# can be poked around to get more information. User can enable this option by setting
+# environment variable OVS_PAUSE_TEST=1. User needs to press CTRL-D to resume the
+# cleanup operation.
+ovs_pause() {
+    echo "====================================================="
+    echo "Set following environment variable to use various ovs utilities"
+    echo "export OVS_RUNDIR=$ovs_base"
+    echo "Press ENTER to continue: "
+    read
+}
+
+ovs_on_exit () {
+    if [ ! -z "${OVS_PAUSE_TEST}" ] && [ -z $at_verbose ]; then
+        trap '' INT
+        ovs_pause
+    fi
+    . "$ovs_base/cleanup"
+}
+
 # With no parameter or an empty parameter, sets the OVS_*DIR
 # environment variables to point to $ovs_base, the base directory in
 # which the test is running.