[ceph.git] / ceph / src / dpdk / doc / guides / sample_app_ug / dist_app.rst

..  BSD LICENSE
    Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
    All rights reserved.

    Redistribution and use in source and binary forms, with or without
    modification, are permitted provided that the following conditions
    are met:

    * Redistributions of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
    notice, this list of conditions and the following disclaimer in
    the documentation and/or other materials provided with the
    distribution.
    * Neither the name of Intel Corporation nor the names of its
    contributors may be used to endorse or promote products derived
    from this software without specific prior written permission.

    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Distributor Sample Application
==============================

The distributor sample application is a simple example of packet distribution
to cores using the Data Plane Development Kit (DPDK).

Overview
--------

The distributor application performs the distribution of packets that are received
on an RX_PORT to different cores. When processed by the cores, the destination
port of a packet is the port from the enabled port mask adjacent to the one on
which the packet was received, that is, if the first four ports are enabled
(port mask 0xf), ports 0 and 1 RX/TX into each other, and ports 2 and 3 RX/TX
into each other.

This application can be used to benchmark performance using the traffic
generator as shown in the figure below.

.. _figure_dist_perf:

.. figure:: img/dist_perf.*

   Performance Benchmarking Setup (Basic Environment)


Compiling the Application
-------------------------

#.  Go to the sample application directory:

    ..  code-block:: console

        export RTE_SDK=/path/to/rte_sdk
        cd ${RTE_SDK}/examples/distributor

#.  Set the target (a default target is used if not specified). For example:

    ..  code-block:: console

        export RTE_TARGET=x86_64-native-linuxapp-gcc

    See the DPDK Getting Started Guide for possible RTE_TARGET values.

#.  Build the application:

    ..  code-block:: console

        make

Running the Application
-----------------------

#. The application has a number of command line options:

   ..  code-block:: console

       ./build/distributor_app [EAL options] -- -p PORTMASK

   where,

   *   -p PORTMASK: Hexadecimal bitmask of ports to configure

#. To run the application in linuxapp environment with 10 lcores, 4 ports,
   issue the command:

   ..  code-block:: console

       $ ./build/distributor_app -c 0x4003fe -n 4 -- -p f

#. Refer to the DPDK Getting Started Guide for general information on running
   applications and the Environment Abstraction Layer (EAL) options.

Explanation
-----------

The distributor application consists of three types of threads: a receive
thread (lcore_rx()), a set of worker threads(lcore_worker())
and a transmit thread(lcore_tx()). How these threads work together is shown
in :numref:`figure_dist_app` below. The main() function launches  threads of these three types.
Each thread has a while loop which will be doing processing and which is
terminated only upon SIGINT or ctrl+C. The receive and transmit threads
communicate using a software ring (rte_ring structure).

The receive thread receives the packets using rte_eth_rx_burst() and gives
them to  the distributor (using rte_distributor_process() API) which will
be called in context of the receive thread itself. The distributor distributes
the packets to workers threads based on the tagging of the packet -
indicated by the hash field in the mbuf. For IP traffic, this field is
automatically filled by the NIC with the "usr" hash value for the packet,
which works as a per-flow tag.

More than one worker thread can exist as part of the application, and these
worker threads do simple packet processing by requesting packets from
the distributor, doing a simple XOR operation on the input port mbuf field
(to indicate the output port which will be used later for packet transmission)
and then finally returning the packets back to the distributor in the RX thread.

Meanwhile, the receive thread will call the distributor api
rte_distributor_returned_pkts() to get the packets processed, and will enqueue
them to a ring for transfer to the TX thread for transmission on the output port.
The transmit thread will dequeue the packets from the ring and transmit them on
the output port specified in packet mbuf.

Users who wish to terminate the running of the application have to press ctrl+C
(or send SIGINT to the app). Upon this signal, a signal handler provided
in the application will terminate all running threads gracefully and print
final statistics to the user.

.. _figure_dist_app:

.. figure:: img/dist_app.*

   Distributor Sample Application Layout


Debug Logging Support
---------------------

Debug logging is provided as part of the application; the user needs to uncomment
the line "#define DEBUG" defined in start of the application in main.c to enable debug logs.

Statistics
----------

Upon SIGINT (or) ctrl+C, the print_stats() function displays the count of packets
processed at the different stages in the application.

Application Initialization
--------------------------

Command line parsing is done in the same way as it is done in the L2 Forwarding Sample
Application. See :ref:`l2_fwd_app_cmd_arguments`.

Mbuf pool initialization is done in the same way as it is done in the L2 Forwarding
Sample Application. See :ref:`l2_fwd_app_mbuf_init`.

Driver Initialization is done in same way as it is done in the L2 Forwarding Sample
Application. See :ref:`l2_fwd_app_dvr_init`.

RX queue initialization is done in the same way as it is done in the L2 Forwarding
Sample Application. See :ref:`l2_fwd_app_rx_init`.

TX queue initialization is done in the same way as it is done in the L2 Forwarding
Sample Application. See :ref:`l2_fwd_app_tx_init`.
Commit	Line	Data
7c673cae FG	1	.. BSD LICENSE
	2	Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
	3	All rights reserved.
	4
	5	Redistribution and use in source and binary forms, with or without
	6	modification, are permitted provided that the following conditions
	7	are met:
	8
	9	* Redistributions of source code must retain the above copyright
	10	notice, this list of conditions and the following disclaimer.
	11	* Redistributions in binary form must reproduce the above copyright
	12	notice, this list of conditions and the following disclaimer in
	13	the documentation and/or other materials provided with the
	14	distribution.
	15	* Neither the name of Intel Corporation nor the names of its
	16	contributors may be used to endorse or promote products derived
	17	from this software without specific prior written permission.
	18
	19	THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
	20	"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
	21	LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
	22	A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
	23	OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
	24	SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
	25	LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
	26	DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
	27	THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
	28	(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
	29	OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
	30
	31	Distributor Sample Application
	32	==============================
	33
	34	The distributor sample application is a simple example of packet distribution
	35	to cores using the Data Plane Development Kit (DPDK).
	36
	37	Overview
	38	--------
	39
	40	The distributor application performs the distribution of packets that are received
	41	on an RX_PORT to different cores. When processed by the cores, the destination
	42	port of a packet is the port from the enabled port mask adjacent to the one on
	43	which the packet was received, that is, if the first four ports are enabled
	44	(port mask 0xf), ports 0 and 1 RX/TX into each other, and ports 2 and 3 RX/TX
	45	into each other.
	46
	47	This application can be used to benchmark performance using the traffic
	48	generator as shown in the figure below.
	49
	50	.. _figure_dist_perf:
	51
	52	.. figure:: img/dist_perf.*
	53
	54	Performance Benchmarking Setup (Basic Environment)
	55
	56
	57	Compiling the Application
	58	-------------------------
	59
	60	#. Go to the sample application directory:
	61
	62	.. code-block:: console
	63
	64	export RTE_SDK=/path/to/rte_sdk
65	cd ${RTE_SDK}/examples/distributor
66
67	#. Set the target (a default target is used if not specified). For example:
68
69	.. code-block:: console
70
71	export RTE_TARGET=x86_64-native-linuxapp-gcc
72
73	See the DPDK Getting Started Guide for possible RTE_TARGET values.
74
75	#. Build the application:
76
77	.. code-block:: console
78
79	make
80
81	Running the Application
82	-----------------------
83
84	#. The application has a number of command line options:
85
86	.. code-block:: console
87
88	./build/distributor_app [EAL options] -- -p PORTMASK
89
90	where,
91
92	* -p PORTMASK: Hexadecimal bitmask of ports to configure
93
94	#. To run the application in linuxapp environment with 10 lcores, 4 ports,
95	issue the command:
96
97	.. code-block:: console
98
99	$ ./build/distributor_app -c 0x4003fe -n 4 -- -p f
100
101	#. Refer to the DPDK Getting Started Guide for general information on running
102	applications and the Environment Abstraction Layer (EAL) options.
103
104	Explanation
105	-----------
106
107	The distributor application consists of three types of threads: a receive
108	thread (lcore_rx()), a set of worker threads(lcore_worker())
109	and a transmit thread(lcore_tx()). How these threads work together is shown
110	in :numref:`figure_dist_app` below. The main() function launches threads of these three types.
111	Each thread has a while loop which will be doing processing and which is
112	terminated only upon SIGINT or ctrl+C. The receive and transmit threads
113	communicate using a software ring (rte_ring structure).
114
115	The receive thread receives the packets using rte_eth_rx_burst() and gives
116	them to the distributor (using rte_distributor_process() API) which will
117	be called in context of the receive thread itself. The distributor distributes
118	the packets to workers threads based on the tagging of the packet -
119	indicated by the hash field in the mbuf. For IP traffic, this field is
120	automatically filled by the NIC with the "usr" hash value for the packet,
121	which works as a per-flow tag.
122
123	More than one worker thread can exist as part of the application, and these
124	worker threads do simple packet processing by requesting packets from
125	the distributor, doing a simple XOR operation on the input port mbuf field
126	(to indicate the output port which will be used later for packet transmission)
127	and then finally returning the packets back to the distributor in the RX thread.
128
129	Meanwhile, the receive thread will call the distributor api
130	rte_distributor_returned_pkts() to get the packets processed, and will enqueue
131	them to a ring for transfer to the TX thread for transmission on the output port.
132	The transmit thread will dequeue the packets from the ring and transmit them on
133	the output port specified in packet mbuf.
134
135	Users who wish to terminate the running of the application have to press ctrl+C
136	(or send SIGINT to the app). Upon this signal, a signal handler provided
137	in the application will terminate all running threads gracefully and print
138	final statistics to the user.
139
140	.. _figure_dist_app:
141
142	.. figure:: img/dist_app.*
143
144	Distributor Sample Application Layout
145
146
147	Debug Logging Support
148	---------------------
149
150	Debug logging is provided as part of the application; the user needs to uncomment
151	the line "#define DEBUG" defined in start of the application in main.c to enable debug logs.
152
153	Statistics
154	----------
155
156	Upon SIGINT (or) ctrl+C, the print_stats() function displays the count of packets
157	processed at the different stages in the application.
158
159	Application Initialization
160	--------------------------
161
162	Command line parsing is done in the same way as it is done in the L2 Forwarding Sample
163	Application. See :ref:`l2_fwd_app_cmd_arguments`.
164
165	Mbuf pool initialization is done in the same way as it is done in the L2 Forwarding
166	Sample Application. See :ref:`l2_fwd_app_mbuf_init`.
167
168	Driver Initialization is done in same way as it is done in the L2 Forwarding Sample
169	Application. See :ref:`l2_fwd_app_dvr_init`.
170
171	RX queue initialization is done in the same way as it is done in the L2 Forwarding
172	Sample Application. See :ref:`l2_fwd_app_rx_init`.
173
174	TX queue initialization is done in the same way as it is done in the L2 Forwarding
175	Sample Application. See :ref:`l2_fwd_app_tx_init`.