odp-execute: Optimize IP header modification in OVS datapath

odp-execute: Optimize IP header modification in OVS datapath

I measured the packet processing cost of OVS DPDK datapath for different
OpenFlow actions. I configured OVS to use a single pmd thread and
measured the packet throughput in a phy-to-phy setup. I used 10G
interfaces bounded to DPDK driver and overloaded the vSwitch with 64
byte packets through one of the 10G interfaces.

The processing cost of the dec_ttl action seemed to be gratuitously high
compared with other actions.

I looked into the code and saw that dec_ttl is encoded as a masked
nested attribute in OVS_ACTION_ATTR_SET_MASKED(OVS_KEY_ATTR_IPV4).
That way, OVS datapath can modify several IP header fields (TTL, TOS,
source and destination IP addresses) by a single invocation of
packet_set_ipv4() in the odp_set_ipv4() function in the
lib/odp-execute.c file. The packet_set_ipv4() function takes the new
TOS, TTL and IP addresses as arguments, compares them with the actual
ones and updates the fields if needed. This means, that even if only TTL
needs to be updated, each of the four IP header fields is passed to the
callee and is compared to the actual field for each packet.

The odp_set_ipv4() caller function possesses information about the
fields that need to be updated in the 'mask' structure. The idea is to
spare invocation of the packet_set_ipv4() function but use its code
parts directly. So the 'mask' can be used to decide which IP header
fields need to be updated. In addition, a faster packet processing can
be achieved if the values of local variables are
calculated right before their usage.

       | T | T | I | I |
       | T | O | P | P |  Vanilla OVS  ||  + new patch
       | L | S | s | d | (nsec/packet) || (nsec/packet)
-------+---+---+---+---+---------------++---------------
output |   |   |   |   |    67.19      ||    67.19
       | X |   |   |   |    74.48      ||    68.78
       |   | X |   |   |    74.42      ||    70.07
       |   |   | X |   |    84.62      ||    78.03
       |   |   |   | X |    84.25      ||    77.94
       |   |   | X | X |    97.46      ||    91.86
       | X |   | X | X |   100.42      ||    96.00
       | X | X | X | X |   102.80      ||   100.73

The table shows the average processing cost of packets in nanoseconds
for the following actions:
output; output + dec_ttl; output + mod_nw_tos; output + mod_nw_src;
output + mod_nw_dst and some of their combinations.
I ran each test five times. The values are the mean of the readings
obtained.

I added OVS_LIKELY to the 'if' condition for the TTL field, since as far
as I know, this field will typically be decremented when any field of
the IP header is modified.

Signed-off-by: Zoltán Balogh <zoltan.balogh@ericsson.com>
Signed-off-by: Daniele Di Proietto <diproiettod@vmware.com>