2 Licensed under the Apache License, Version 2.0 (the "License"); you may
3 not use this file except in compliance with the License. You may obtain
4 a copy of the License at
6 http://www.apache.org/licenses/LICENSE-2.0
8 Unless required by applicable law or agreed to in writing, software
9 distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
10 WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
11 License for the specific language governing permissions and limitations
14 Convention for heading levels in Open vSwitch documentation:
16 ======= Heading 0 (reserved for the title in a document)
22 Avoid deeper levels because they do not render well.
28 It is possible to test Open vSwitch using both tooling provided with Open
29 vSwitch and using a variety of third party tooling.
34 Open vSwitch provides a number of different test suites and other tooling for
35 validating basic functionality of OVS. Before running any of the tests
36 described here, you must bootstrap, configure and build Open vSwitch as
37 described in :doc:`/intro/install/general`. You do not need to install Open
38 vSwitch or to build or load the kernel module to run these test suites. You do
39 not need supervisor privilege to run these test suites.
44 Open vSwitch includes a suite of self-tests. Before you submit patches
45 upstream, we advise that you run the tests and ensure that they pass. If you
46 add new features to Open vSwitch, then adding tests for those features will
47 ensure your features don't break as developers modify other areas of Open
50 To run all the unit tests in Open vSwitch, one at a time, run::
54 This takes under 5 minutes on a modern desktop system.
56 To run all the unit tests in Open vSwitch in parallel, run::
58 $ make check TESTSUITEFLAGS=-j8
60 You can run up to eight threads. This takes under a minute on a modern 4-core
63 To see a list of all the available tests, run::
65 $ make check TESTSUITEFLAGS=--list
67 To run only a subset of tests, e.g. test 123 and tests 477 through 484, run::
69 $ make check TESTSUITEFLAGS='123 477-484'
71 Tests do not have inter-dependencies, so you may run any subset.
73 To run tests matching a keyword, e.g. ``ovsdb``, run::
75 $ make check TESTSUITEFLAGS='-k ovsdb'
77 To see a complete list of test options, run::
79 $ make check TESTSUITEFLAGS=--help
81 The results of a testing run are reported in ``tests/testsuite.log``. Report
82 report test failures as bugs and include the ``testsuite.log`` in your report.
85 Sometimes a few tests may fail on some runs but not others. This is usually a
86 bug in the testsuite, not a bug in Open vSwitch itself. If you find that a
87 test fails intermittently, please report it, since the developers may not
88 have noticed. You can make the testsuite automatically rerun tests that fail,
89 by adding ``RECHECK=yes`` to the ``make`` command line, e.g.::
91 $ make check TESTSUITEFLAGS=-j8 RECHECK=yes
98 If the build was configured with ``--enable-coverage`` and the ``lcov`` utility
99 is installed, you can run the testsuite and generate a code coverage report by
100 using the ``check-lcov`` target::
104 All the same options are available via TESTSUITEFLAGS. For example::
106 $ make check-lcov TESTSUITEFLAGS='-j8 -k ovn'
108 .. _testing-valgrind:
113 If you have ``valgrind`` installed, you can run the testsuite under
114 valgrind by using the ``check-valgrind`` target::
116 $ make check-valgrind
118 When you do this, the "valgrind" results for test ``<N>`` are reported in files
119 named ``tests/testsuite.dir/<N>/valgrind.*``.
121 To test the testsuite of kernel datapath under valgrind, you can use the
122 ``check-kernel-valgrind`` target and find the "valgrind" results under
123 directory ``tests/system-kmod-testsuite.dir/``.
125 All the same options are available via TESTSUITEFLAGS.
128 You may find that the valgrind results are easier to interpret if you put
129 ``-q`` in ``~/.valgrindrc``, since that reduces the amount of output.
134 OFTest is an OpenFlow protocol testing suite. Open vSwitch includes a Makefile
135 target to run OFTest with Open vSwitch in "dummy mode". In this mode of
136 testing, no packets travel across physical or virtual networks. Instead, Unix
137 domain sockets stand in as simulated networks. This simulation is imperfect,
138 but it is much easier to set up, does not require extra physical or virtual
139 hardware, and does not require supervisor privileges.
141 To run OFTest with Open vSwitch, you must obtain a copy of OFTest and install
142 its prerequisites. You need a copy of OFTest that includes commit 406614846c5
143 (make ovs-dummy platform work again). This commit was merged into the OFTest
144 repository on Feb 1, 2013, so any copy of OFTest more recent than that should
145 work. Testing OVS in dummy mode does not require root privilege, so you may
146 ignore that requirement.
148 Optionally, add the top-level OFTest directory (containing the ``oft`` program)
149 to your ``$PATH``. This slightly simplifies running OFTest later.
151 To run OFTest in dummy mode, run the following command from your Open vSwitch
154 $ make check-oftest OFT=<oft-binary>
156 where ``<oft-binary>`` is the absolute path to the ``oft`` program in OFTest.
157 If you added "oft" to your $PATH, you may omit the OFT variable
160 By default, ``check-oftest`` passes ``oft`` just enough options to enable dummy
161 mode. You can use ``OFTFLAGS`` to pass additional options. For example, to run
162 just the ``basic.Echo`` test instead of all tests (the default) and enable
163 verbose logging, run::
165 $ make check-oftest OFT=<oft-binary> OFTFLAGS='--verbose -T basic.Echo'
167 If you use OFTest that does not include commit 4d1f3eb2c792 (oft: change
168 default port to 6653), merged into the OFTest repository in October 2013, then
169 you need to add an option to use the IETF-assigned controller port::
171 $ make check-oftest OFT=<oft-binary> OFTFLAGS='--port=6653'
173 Interpret OFTest results cautiously. Open vSwitch can fail a given test in
174 OFTest for many reasons, including bugs in Open vSwitch, bugs in OFTest, bugs
175 in the "dummy mode" integration, and differing interpretations of the OpenFlow
176 standard and other standards.
179 Open vSwitch has not been validated against OFTest. Report test failures that
180 you believe to represent bugs in Open vSwitch. Include the precise versions
181 of Open vSwitch and OFTest in your bug report, plus any other information
182 needed to reproduce the problem.
187 Ryu is an OpenFlow controller written in Python that includes an extensive
188 OpenFlow testsuite. Open vSwitch includes a Makefile target to run Ryu in
189 "dummy mode". See `OFTest`_ above for an explanation of dummy mode.
191 To run Ryu tests with Open vSwitch, first read and follow the instructions
192 under **Testing** above. Second, obtain a copy of Ryu, install its
193 prerequisites, and build it. You do not need to install Ryu (some of the tests
194 do not get installed, so it does not help).
196 To run Ryu tests, run the following command from your Open vSwitch build
199 $ make check-ryu RYUDIR=<ryu-source-dir>
201 where ``<ryu-source-dir>`` is the absolute path to the root of the Ryu source
202 distribution. The default ``<ryu-source-dir>`` is ``$srcdir/../ryu``
203 where ``$srcdir`` is your Open vSwitch source directory. If this is correct,
207 Open vSwitch has not been validated against Ryu. Report test failures that
208 you believe to represent bugs in Open vSwitch. Include the precise versions
209 of Open vSwitch and Ryu in your bug report, plus any other information
210 needed to reproduce the problem.
212 .. _datapath-testing:
217 Open vSwitch includes a suite of tests specifically for datapath functionality,
218 which can be run against the userspace or kernel datapaths. If you are
219 developing datapath features, it is recommended that you use these tests and
220 build upon them to verify your implementation.
222 The datapath tests make some assumptions about the environment. They must be
223 run under root privileges on a Linux system with support for network
224 namespaces. For ease of use, the OVS source tree includes a vagrant box to
225 invoke these tests. Running the tests inside Vagrant provides kernel isolation,
226 protecting your development host from kernel panics or configuration conflicts
227 in the testsuite. If you wish to run the tests without using the vagrant box,
228 there are further instructions below.
235 Requires Vagrant (version 1.7.0 or later) and a compatible hypervisor
238 You must bootstrap and configure the sources (see
239 doc:`/intro/install/general`) before you run the steps described
242 A Vagrantfile is provided allowing to compile and provision the source tree as
243 found locally in a virtual machine using the following command::
247 This will bring up a Fedora 23 VM by default. If you wish to use a different
248 box or a vagrant backend not supported by the default box, the ``Vagrantfile``
249 can be modified to use a different box as base.
251 The VM can be reprovisioned at any time::
255 OVS out-of-tree compilation environment can be set up with::
258 $ vagrant provision --provision-with configure_ovs,build_ovs
260 This will set up an out-of-tree build environment inside the VM in
261 ``/root/build``. The source code can be found in ``/vagrant``.
263 To recompile and reinstall OVS in the VM using RPM::
266 $ vagrant provision --provision-with configure_ovs,install_rpm
268 Two provisioners are included to run system tests with the OVS kernel module or
269 with a userspace datapath. This tests are different from the self-tests
270 mentioned above. To run them::
273 $ vagrant provision --provision-with \
274 configure_ovs,test_ovs_kmod,test_ovs_system_userspace
276 The results of the testsuite reside in the VM root user's home directory::
286 The datapath testsuite as invoked by Vagrant above may also be run manually on
287 a Linux system with root privileges. Make sure, no other Open vSwitch instance
288 is running on the test suite. These tests may take several minutes to complete,
289 and cannot be run in parallel.
294 To invoke the datapath testsuite with the userspace datapath, run::
296 $ make check-system-userspace
298 The results of the testsuite are in ``tests/system-userspace-testsuite.dir``.
303 Make targets are also provided for testing the Linux kernel module. Note that
304 these tests operate by inserting modules into the running Linux kernel, so if
305 the tests are able to trigger a bug in the OVS kernel module or in the upstream
306 kernel then the kernel may panic.
308 To run the testsuite against the kernel module which is currently installed on
313 To install the kernel module from the current build directory and run the
314 testsuite against that kernel module::
318 The results of the testsuite are in ``tests/system-kmod-testsuite.dir``.
320 .. _testing-static-analysis:
325 Static Analysis is a method of debugging Software by examining code rather than
326 actually executing it. This can be done through 'scan-build' commandline
327 utility which internally uses clang (or) gcc to compile the code and also
328 invokes a static analyzer to do the code analysis. At the end of the build, the
329 reports are aggregated in to a common folder and can later be analyzed using
332 Open vSwitch includes a Makefile target to trigger static code analysis::
335 $ ./configure CC=clang # clang
337 $ ./configure CC=gcc CFLAGS="-std=gnu99" # gcc
340 You should invoke scan-view to view analysis results. The last line of output
341 from ``clang-analyze`` will list the command (containing results directory)
342 that you should invoke to view the results on a browser.
344 Continuous Integration with Travis CI
345 -------------------------------------
347 A .travis.yml file is provided to automatically build Open vSwitch with various
348 build configurations and run the testsuite using Travis CI. Builds will be
349 performed with gcc, sparse and clang with the -Werror compiler flag included,
350 therefore the build will fail if a new warning has been introduced.
352 The CI build is triggered via git push (regardless of the specific branch) or
353 pull request against any Open vSwitch GitHub repository that is linked to
356 Instructions to setup travis-ci for your GitHub repository:
358 1. Go to https://travis-ci.org/ and sign in using your GitHub ID.
359 2. Go to the "Repositories" tab and enable the ovs repository. You may disable
360 builds for pushes or pull requests.
361 3. In order to avoid forks sending build failures to the upstream mailing list,
362 the notification email recipient is encrypted. If you want to receive email
363 notification for build failures, replace the the encrypted string:
365 1. Install the travis-ci CLI (Requires ruby >=2.0): gem install travis
366 2. In your Open vSwitch repository: travis encrypt mylist@mydomain.org
367 3. Add/replace the notifications section in .travis.yml and fill in the
368 secure string as returned by travis encrypt::
376 You may remove/omit the notifications section to fall back to default
377 notification behaviour which is to send an email directly to the author and
378 committer of the failing commit. Note that the email is only sent if the
379 author/committer have commit rights for the particular GitHub repository.
381 4. Pushing a commit to the repository which breaks the build or the
382 testsuite will now trigger a email sent to mylist@mydomain.org
387 The vsperf project aims to develop a vSwitch test framework that can be used to
388 validate the suitability of different vSwitch implementations in a telco
389 deployment environment. More information can be found on the `OPNFV wiki`_.
391 .. _OPNFV wiki: https://wiki.opnfv.org/display/vsperf/VSperf+Home
396 Proof of Concepts are documentation materialized into Ansible recipes
397 executed in VirtualBox or Libvirt environments orchastrated by Vagrant.
398 Proof of Concepts allow developers to create small virtualized setups that
399 demonstrate how certain Open vSwitch features are intended to work avoiding
400 user introduced errors by overlooking instructions. Proof of Concepts
401 are also helpful when integrating with thirdparty software, because standard
402 unit tests with make check are limited.
404 Vagrant by default uses VirtualBox provider. However, if Libvirt is your
405 choice of virtualization technology, then you can use it by installing Libvirt
408 $ vagrant plugin install vagrant-libvirt
410 And then appending --provider=libvirt flag to vagrant commands.
412 The host where Vagrant runs does not need to have any special software
413 installed besides vagrant, virtualbox (or libvirt and libvirt-dev) and
416 The following Proof of Concepts are supported:
421 This particular Proof of Concept demonstrates integration with Debian and RPM
428 Once that command finished you can get packages from /var/www/html
429 directory. Since those hosts are also configured as repositories then
430 you can add them to /etc/apt/sources.list.d or /etc/yum.repos.d
431 configuration files on another host to retrieve packages with yum or
434 When you have made changes to OVS source code and want to rebuild packages run:
436 $ vagrant rsync && vagrant provision
438 Whenever packages are rebuilt the Open vSwitch release number increases
439 by one and you can simply upgrade Open vSwitch by running yum or apt-get
442 Once you are done with experimenting you can tear down setup with:
446 Sometimes deployment of Proof of Concept may fail, if, for example, VMs
447 don't have network reachability to the Internet.