]>
Commit | Line | Data |
---|---|---|
79f36875 JS |
1 | .SS "DPIF-NETDEV COMMANDS" |
2 | These commands are used to expose internal information (mostly statistics) | |
3 | about the "dpif-netdev" userspace datapath. If there is only one datapath | |
4 | (as is often the case, unless \fBdpctl/\fR commands are used), the \fIdp\fR | |
5 | argument can be omitted. By default the commands present data for all pmd | |
6 | threads in the datapath. By specifying the "-pmd Core" option one can filter | |
7 | the output for a single pmd in the datapath. | |
8 | . | |
9 | .IP "\fBdpif-netdev/pmd-stats-show\fR [\fB-pmd\fR \fIcore\fR] [\fIdp\fR]" | |
10 | Shows performance statistics for one or all pmd threads of the datapath | |
11 | \fIdp\fR. The special thread "main" sums up the statistics of every non pmd | |
12 | thread. | |
13 | ||
21e9b77b | 14 | The sum of "emc hits", "smc hits", "megaflow hits" and "miss" is the number of |
79f36875 JS |
15 | packet lookups performed by the datapath. Beware that a recirculated packet |
16 | experiences one additional lookup per recirculation, so there may be | |
17 | more lookups than forwarded packets in the datapath. | |
18 | ||
19 | Cycles are counted using the TSC or similar facilities (when available on | |
20 | the platform). The duration of one cycle depends on the processing platform. | |
21 | ||
22 | "idle cycles" refers to cycles spent in PMD iterations not forwarding any | |
23 | any packets. "processing cycles" refers to cycles spent in PMD iterations | |
24 | forwarding at least one packet, including the cost for polling, processing and | |
25 | transmitting said packets. | |
26 | ||
27 | To reset these counters use \fBdpif-netdev/pmd-stats-clear\fR. | |
28 | . | |
29 | .IP "\fBdpif-netdev/pmd-stats-clear\fR [\fIdp\fR]" | |
30 | Resets to zero the per pmd thread performance numbers shown by the | |
31 | \fBdpif-netdev/pmd-stats-show\fR and \fBdpif-netdev/pmd-perf-show\fR commands. | |
32 | It will NOT reset datapath or bridge statistics, only the values shown by | |
33 | the above commands. | |
34 | . | |
35 | .IP "\fBdpif-netdev/pmd-perf-show\fR [\fB-nh\fR] [\fB-it\fR \fIiter_len\fR] \ | |
36 | [\fB-ms\fR \fIms_len\fR] [\fB-pmd\fR \fIcore\fR] [\fIdp\fR]" | |
37 | Shows detailed performance metrics for one or all pmds threads of the | |
38 | user space datapath. | |
39 | ||
40 | The collection of detailed statistics can be controlled by a new | |
41 | configuration parameter "other_config:pmd-perf-metrics". By default it | |
42 | is disabled. The run-time overhead, when enabled, is in the order of 1%. | |
43 | ||
44 | .RS | |
45 | .IP | |
46 | .PD .4v | |
47 | .IP \(em | |
48 | used cycles | |
49 | .IP \(em | |
50 | forwared packets | |
51 | .IP \(em | |
52 | number of rx batches | |
53 | .IP \(em | |
54 | packets/rx batch | |
55 | .IP \(em | |
56 | max. vhostuser queue fill level | |
57 | .IP \(em | |
58 | number of upcalls | |
59 | .IP \(em | |
60 | cycles spent in upcalls | |
61 | .PD | |
62 | .RE | |
63 | .IP | |
64 | This raw recorded data is used threefold: | |
65 | ||
66 | .RS | |
67 | .IP | |
68 | .PD .4v | |
69 | .IP 1. | |
70 | In histograms for each of the following metrics: | |
71 | .RS | |
72 | .IP \(em | |
73 | cycles/iteration (logarithmic) | |
74 | .IP \(em | |
75 | packets/iteration (logarithmic) | |
76 | .IP \(em | |
77 | cycles/packet | |
78 | .IP \(em | |
79 | packets/batch | |
80 | .IP \(em | |
81 | max. vhostuser qlen (logarithmic) | |
82 | .IP \(em | |
83 | upcalls | |
84 | .IP \(em | |
85 | cycles/upcall (logarithmic) | |
86 | The histograms bins are divided linear or logarithmic. | |
87 | .RE | |
88 | .IP 2. | |
89 | A cyclic history of the above metrics for 1024 iterations | |
90 | .IP 3. | |
91 | A cyclic history of the cummulative/average values per millisecond wall | |
92 | clock for the last 1024 milliseconds: | |
93 | .RS | |
94 | .IP \(em | |
95 | number of iterations | |
96 | .IP \(em | |
97 | avg. cycles/iteration | |
98 | .IP \(em | |
99 | packets (Kpps) | |
100 | .IP \(em | |
101 | avg. packets/batch | |
102 | .IP \(em | |
103 | avg. max vhost qlen | |
104 | .IP \(em | |
105 | upcalls | |
106 | .IP \(em | |
107 | avg. cycles/upcall | |
108 | .RE | |
109 | .PD | |
110 | .RE | |
111 | .IP | |
112 | . | |
113 | The command options are: | |
114 | .RS | |
115 | .IP "\fB-nh\fR" | |
116 | Suppress the histograms | |
117 | .IP "\fB-it\fR \fIiter_len\fR" | |
118 | Display the last iter_len iteration stats | |
119 | .IP "\fB-ms\fR \fIms_len\fR" | |
120 | Display the last ms_len millisecond stats | |
121 | .RE | |
122 | .IP | |
123 | The output always contains the following global PMD statistics: | |
124 | .RS | |
125 | .IP | |
6794be1d IM |
126 | .EX |
127 | Time: 15:24:55.270 | |
79f36875 JS |
128 | Measurement duration: 1.008 s |
129 | ||
130 | pmd thread numa_id 0 core_id 1: | |
131 | ||
6794be1d IM |
132 | Iterations: 572817 (1.76 us/it) |
133 | - Used TSC cycles: 2419034712 ( 99.9 % of total cycles) | |
134 | - idle iterations: 486808 ( 15.9 % of used cycles) | |
135 | - busy iterations: 86009 ( 84.1 % of used cycles) | |
136 | Rx packets: 2399607 (2381 Kpps, 848 cycles/pkt) | |
137 | Datapath passes: 3599415 (1.50 passes/pkt) | |
138 | - EMC hits: 336472 ( 9.3 %) | |
139 | - SMC hits: 0 ( 0.0 %) | |
140 | - Megaflow hits: 3262943 ( 90.7 %, 1.00 subtbl lookups/hit) | |
141 | - Upcalls: 0 ( 0.0 %, 0.0 us/upcall) | |
142 | - Lost upcalls: 0 ( 0.0 %) | |
143 | Tx packets: 2399607 (2381 Kpps) | |
144 | Tx batches: 171400 (14.00 pkts/batch) | |
145 | .EE | |
79f36875 JS |
146 | .RE |
147 | .IP | |
148 | Here "Rx packets" actually reflects the number of packets forwarded by the | |
149 | datapath. "Datapath passes" matches the number of packet lookups as | |
150 | reported by the \fBdpif-netdev/pmd-stats-show\fR command. | |
151 | ||
152 | To reset the counters and start a new measurement use | |
153 | \fBdpif-netdev/pmd-stats-clear\fR. | |
154 | . | |
7178fefb JS |
155 | .IP "\fBdpif-netdev/pmd-perf-log-set\fR \fBon\fR|\fBoff\fR \ |
156 | [\fB-b\fR \fIbefore\fR] [\fB-a\fR \fIafter\fR] [\fB-e\fR|\fB-ne\fR] \ | |
157 | [\fB-us\fR \fIusec\fR] [\fB-q\fR \fIqlen\fR]" | |
158 | . | |
159 | The userspace "netdev" datapath is able to supervise the PMD performance | |
160 | metrics and detect iterations with suspicious statistics according to the | |
161 | following criteria: | |
162 | .RS | |
163 | .IP \(em | |
164 | The iteration lasts longer than \fIusec\fR microseconds (default 250). | |
165 | This can be used to capture events where a PMD is blocked or interrupted for | |
166 | such a period of time that there is a risk for dropped packets on any of its Rx | |
167 | queues. | |
168 | .IP \(em | |
169 | The max vhost qlen exceeds a threshold \fIqlen\fR (default 128). This can be | |
170 | used to infer virtio queue overruns and dropped packets inside a VM, which are | |
171 | not visible in OVS otherwise. | |
172 | .RE | |
173 | .IP | |
174 | Such suspicious iterations can be logged together with their iteration | |
175 | statistics in the \fBovs-vswitchd.log\fR to be able to correlate them to | |
176 | packet drop or other events outside OVS. | |
177 | ||
178 | The above command enables (\fBon\fR) or disables (\fBoff\fR) supervision and | |
179 | logging at run-time and can be used to adjust the above thresholds for | |
180 | detecting suspicious iterations. By default supervision and logging is | |
181 | disabled. | |
182 | ||
183 | The command options are: | |
184 | .RS | |
185 | .IP "\fB-b\fR \fIbefore\fR" | |
186 | The number of iterations before the suspicious iteration to be logged | |
187 | (default 5). | |
188 | .IP "\fB-a\fR \fIafter\fR" | |
189 | The number of iterations after the suspicious iteration to be logged | |
190 | (default 5). | |
191 | .IP "\fB-e\fR" | |
192 | Extend logging interval if another suspicious iteration is detected | |
193 | before logging occurs. | |
194 | .IP "\fB-ne\fR" | |
195 | Do not extend logging interval if another suspicious iteration is detected | |
196 | before logging occurs (default). | |
197 | .IP "\fB-q\fR \fIqlen\fR" | |
198 | Suspicious vhost queue fill level threshold. Increase this to 512 if the Qemu | |
199 | supports 1024 virtio queue length (default 128). | |
200 | .IP "\fB-us\fR \fIusec\fR" | |
201 | Change the duration threshold for a suspicious iteration (default 250 us). | |
202 | .RE | |
203 | ||
204 | Note: Logging of suspicious iterations itself consumes a considerable amount | |
205 | of processing cycles of a PMD which may be visible in the iteration history. | |
206 | In the worst case this can lead OVS to detect another suspicious iteration | |
207 | caused by logging. | |
208 | ||
209 | If more than 100 iterations around a suspicious iteration have been logged | |
210 | once, OVS falls back to the safe default values (-b 5 -a 5 -ne) to avoid | |
211 | that logging itself continuously causes logging of further suspicious | |
212 | iterations. | |
213 | . | |
79f36875 JS |
214 | .IP "\fBdpif-netdev/pmd-rxq-show\fR [\fB-pmd\fR \fIcore\fR] [\fIdp\fR]" |
215 | For one or all pmd threads of the datapath \fIdp\fR show the list of queue-ids | |
216 | with port names, which this thread polls. | |
217 | . | |
218 | .IP "\fBdpif-netdev/pmd-rxq-rebalance\fR [\fIdp\fR]" | |
219 | Reassigns rxqs to pmds in the datapath \fIdp\fR based on their current usage. | |
d4a12d87 AM |
220 | . |
221 | .IP "\fBdpif-netdev/bond-show\fR [\fIdp\fR]" | |
222 | When "other_config:lb-output-action" is set to "true", the userspace datapath | |
223 | handles the load balancing of bonds directly instead of depending on flow | |
224 | recirculation (only in balance-tcp mode). | |
225 | ||
226 | When this is the case, the above command prints the load-balancing information | |
227 | of the bonds configured in datapath \fIdp\fR showing the interface associated | |
228 | with each bucket (hash). |