]>
Commit | Line | Data |
---|---|---|
11fdf7f2 TL |
1 | .. SPDX-License-Identifier: BSD-3-Clause |
2 | Copyright(c) 2010-2014 Intel Corporation. | |
7c673cae FG |
3 | |
4 | .. _building_from_source: | |
5 | ||
6 | Compiling the DPDK Target from Source | |
7 | ===================================== | |
8 | ||
9 | System Requirements | |
10 | ------------------- | |
11 | ||
12 | The DPDK and its applications require the GNU make system (gmake) | |
13 | to build on FreeBSD. Optionally, gcc may also be used in place of clang | |
14 | to build the DPDK, in which case it too must be installed prior to | |
15 | compiling the DPDK. The installation of these tools is covered in this | |
16 | section. | |
17 | ||
18 | Compiling the DPDK requires the FreeBSD kernel sources, which should be | |
19 | included during the installation of FreeBSD on the development platform. | |
20 | The DPDK also requires the use of FreeBSD ports to compile and function. | |
21 | ||
22 | To use the FreeBSD ports system, it is required to update and extract the FreeBSD | |
23 | ports tree by issuing the following commands: | |
24 | ||
25 | .. code-block:: console | |
26 | ||
27 | portsnap fetch | |
28 | portsnap extract | |
29 | ||
30 | If the environment requires proxies for external communication, these can be set | |
31 | using: | |
32 | ||
33 | .. code-block:: console | |
34 | ||
35 | setenv http_proxy <my_proxy_host>:<port> | |
36 | setenv ftp_proxy <my_proxy_host>:<port> | |
37 | ||
38 | The FreeBSD ports below need to be installed prior to building the DPDK. | |
39 | In general these can be installed using the following set of commands:: | |
40 | ||
41 | cd /usr/ports/<port_location> | |
42 | ||
43 | make config-recursive | |
44 | ||
45 | make install | |
46 | ||
47 | make clean | |
48 | ||
49 | Each port location can be found using:: | |
50 | ||
51 | whereis <port_name> | |
52 | ||
53 | The ports required and their locations are as follows: | |
54 | ||
55 | * dialog4ports: ``/usr/ports/ports-mgmt/dialog4ports`` | |
56 | ||
57 | * GNU make(gmake): ``/usr/ports/devel/gmake`` | |
58 | ||
59 | * coreutils: ``/usr/ports/sysutils/coreutils`` | |
60 | ||
61 | For compiling and using the DPDK with gcc, the compiler must be installed | |
62 | from the ports collection: | |
63 | ||
64 | * gcc: version 4.9 is recommended ``/usr/ports/lang/gcc49``. | |
65 | Ensure that ``CPU_OPTS`` is selected (default is OFF). | |
66 | ||
67 | When running the make config-recursive command, a dialog may be presented to the | |
68 | user. For the installation of the DPDK, the default options were used. | |
69 | ||
70 | .. note:: | |
71 | ||
72 | To avoid multiple dialogs being presented to the user during make install, | |
73 | it is advisable before running the make install command to re-run the | |
74 | make config-recursive command until no more dialogs are seen. | |
75 | ||
76 | ||
77 | Install the DPDK and Browse Sources | |
78 | ----------------------------------- | |
79 | ||
80 | First, uncompress the archive and move to the DPDK source directory: | |
81 | ||
82 | .. code-block:: console | |
83 | ||
84 | unzip DPDK-<version>.zip | |
85 | cd DPDK-<version> | |
86 | ||
87 | The DPDK is composed of several directories: | |
88 | ||
89 | * lib: Source code of DPDK libraries | |
90 | ||
91 | * app: Source code of DPDK applications (automatic tests) | |
92 | ||
93 | * examples: Source code of DPDK applications | |
94 | ||
11fdf7f2 | 95 | * config, buildtools, mk: Framework-related makefiles, scripts and configuration |
7c673cae FG |
96 | |
97 | Installation of the DPDK Target Environments | |
98 | -------------------------------------------- | |
99 | ||
100 | The format of a DPDK target is:: | |
101 | ||
102 | ARCH-MACHINE-EXECENV-TOOLCHAIN | |
103 | ||
104 | Where: | |
105 | ||
106 | * ``ARCH`` is: ``x86_64`` | |
107 | ||
108 | * ``MACHINE`` is: ``native`` | |
109 | ||
9f95a23c | 110 | * ``EXECENV`` is: ``freebsd`` |
7c673cae FG |
111 | |
112 | * ``TOOLCHAIN`` is: ``gcc`` | ``clang`` | |
113 | ||
114 | The configuration files for the DPDK targets can be found in the DPDK/config | |
115 | directory in the form of:: | |
116 | ||
117 | defconfig_ARCH-MACHINE-EXECENV-TOOLCHAIN | |
118 | ||
119 | .. note:: | |
120 | ||
121 | Configuration files are provided with the ``RTE_MACHINE`` optimization level set. | |
122 | Within the configuration files, the ``RTE_MACHINE`` configuration value is set | |
123 | to native, which means that the compiled software is tuned for the platform | |
124 | on which it is built. For more information on this setting, and its | |
125 | possible values, see the *DPDK Programmers Guide*. | |
126 | ||
127 | To make the target, use ``gmake install T=<target>``. | |
128 | ||
129 | For example to compile for FreeBSD use: | |
130 | ||
131 | .. code-block:: console | |
132 | ||
9f95a23c | 133 | gmake install T=x86_64-native-freebsd-clang |
7c673cae FG |
134 | |
135 | .. note:: | |
136 | ||
137 | If the compiler binary to be used does not correspond to that given in the | |
138 | TOOLCHAIN part of the target, the compiler command may need to be explicitly | |
139 | specified. For example, if compiling for gcc, where the gcc binary is called | |
140 | gcc4.9, the command would need to be ``gmake install T=<target> CC=gcc4.9``. | |
141 | ||
142 | Browsing the Installed DPDK Environment Target | |
143 | ---------------------------------------------- | |
144 | ||
145 | Once a target is created, it contains all the libraries and header files for the | |
146 | DPDK environment that are required to build customer applications. | |
147 | In addition, the test and testpmd applications are built under the build/app | |
148 | directory, which may be used for testing. A kmod directory is also present that | |
149 | contains the kernel modules to install. | |
150 | ||
151 | .. _loading_contigmem: | |
152 | ||
153 | Loading the DPDK contigmem Module | |
154 | --------------------------------- | |
155 | ||
156 | To run a DPDK application, physically contiguous memory is required. | |
157 | In the absence of non-transparent superpages, the included sources for the | |
158 | contigmem kernel module provides the ability to present contiguous blocks of | |
159 | memory for the DPDK to use. The contigmem module must be loaded into the | |
160 | running kernel before any DPDK is run. The module is found in the kmod | |
161 | sub-directory of the DPDK target directory. | |
162 | ||
163 | The amount of physically contiguous memory along with the number of physically | |
164 | contiguous blocks to be reserved by the module can be set at runtime prior to | |
165 | module loading using: | |
166 | ||
167 | .. code-block:: console | |
168 | ||
169 | kenv hw.contigmem.num_buffers=n | |
170 | kenv hw.contigmem.buffer_size=m | |
171 | ||
172 | The kernel environment variables can also be specified during boot by placing the | |
173 | following in ``/boot/loader.conf``:: | |
174 | ||
175 | hw.contigmem.num_buffers=n hw.contigmem.buffer_size=m | |
176 | ||
177 | The variables can be inspected using the following command: | |
178 | ||
179 | .. code-block:: console | |
180 | ||
181 | sysctl -a hw.contigmem | |
182 | ||
183 | Where n is the number of blocks and m is the size in bytes of each area of | |
184 | contiguous memory. A default of two buffers of size 1073741824 bytes (1 Gigabyte) | |
185 | each is set during module load if they are not specified in the environment. | |
186 | ||
187 | The module can then be loaded using kldload (assuming that the current directory | |
188 | is the DPDK target directory): | |
189 | ||
190 | .. code-block:: console | |
191 | ||
192 | kldload ./kmod/contigmem.ko | |
193 | ||
194 | It is advisable to include the loading of the contigmem module during the boot | |
195 | process to avoid issues with potential memory fragmentation during later system | |
196 | up time. This can be achieved by copying the module to the ``/boot/kernel/`` | |
197 | directory and placing the following into ``/boot/loader.conf``:: | |
198 | ||
199 | contigmem_load="YES" | |
200 | ||
201 | .. note:: | |
202 | ||
203 | The contigmem_load directive should be placed after any definitions of | |
204 | ``hw.contigmem.num_buffers`` and ``hw.contigmem.buffer_size`` if the default values | |
205 | are not to be used. | |
206 | ||
207 | An error such as: | |
208 | ||
209 | .. code-block:: console | |
210 | ||
9f95a23c | 211 | kldload: can't load ./x86_64-native-freebsd-gcc/kmod/contigmem.ko: |
7c673cae FG |
212 | Exec format error |
213 | ||
214 | is generally attributed to not having enough contiguous memory | |
215 | available and can be verified via dmesg or ``/var/log/messages``: | |
216 | ||
217 | .. code-block:: console | |
218 | ||
219 | kernel: contigmalloc failed for buffer <n> | |
220 | ||
221 | To avoid this error, reduce the number of buffers or the buffer size. | |
222 | ||
223 | .. _loading_nic_uio: | |
224 | ||
225 | Loading the DPDK nic_uio Module | |
226 | ------------------------------- | |
227 | ||
228 | After loading the contigmem module, the ``nic_uio`` module must also be loaded into the | |
229 | running kernel prior to running any DPDK application. This module must | |
230 | be loaded using the kldload command as shown below (assuming that the current | |
231 | directory is the DPDK target directory). | |
232 | ||
233 | .. code-block:: console | |
234 | ||
235 | kldload ./kmod/nic_uio.ko | |
236 | ||
237 | .. note:: | |
238 | ||
239 | If the ports to be used are currently bound to a existing kernel driver | |
240 | then the ``hw.nic_uio.bdfs sysctl`` value will need to be set before loading the | |
241 | module. Setting this value is described in the next section below. | |
242 | ||
243 | Currently loaded modules can be seen by using the ``kldstat`` command and a module | |
244 | can be removed from the running kernel by using ``kldunload <module_name>``. | |
245 | ||
246 | To load the module during boot, copy the ``nic_uio`` module to ``/boot/kernel`` | |
247 | and place the following into ``/boot/loader.conf``:: | |
248 | ||
249 | nic_uio_load="YES" | |
250 | ||
251 | .. note:: | |
252 | ||
253 | ``nic_uio_load="YES"`` must appear after the contigmem_load directive, if it exists. | |
254 | ||
255 | By default, the ``nic_uio`` module will take ownership of network ports if they are | |
256 | recognized DPDK devices and are not owned by another module. However, since | |
257 | the FreeBSD kernel includes support, either built-in, or via a separate driver | |
258 | module, for most network card devices, it is likely that the ports to be used are | |
259 | already bound to a driver other than ``nic_uio``. The following sub-section describe | |
260 | how to query and modify the device ownership of the ports to be used by | |
261 | DPDK applications. | |
262 | ||
263 | .. _binding_network_ports: | |
264 | ||
265 | Binding Network Ports to the nic_uio Module | |
266 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
267 | ||
268 | Device ownership can be viewed using the pciconf -l command. The example below shows | |
269 | four Intel® 82599 network ports under ``if_ixgbe`` module ownership. | |
270 | ||
271 | .. code-block:: console | |
272 | ||
273 | pciconf -l | |
274 | ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 | |
275 | ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 | |
276 | ix2@pci0:2:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 | |
277 | ix3@pci0:2:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 | |
278 | ||
279 | The first column constitutes three components: | |
280 | ||
281 | #. Device name: ``ixN`` | |
282 | ||
283 | #. Unit name: ``pci0`` | |
284 | ||
285 | #. Selector (Bus:Device:Function): ``1:0:0`` | |
286 | ||
287 | Where no driver is associated with a device, the device name will be ``none``. | |
288 | ||
289 | By default, the FreeBSD kernel will include built-in drivers for the most common | |
290 | devices; a kernel rebuild would normally be required to either remove the drivers | |
291 | or configure them as loadable modules. | |
292 | ||
293 | To avoid building a custom kernel, the ``nic_uio`` module can detach a network port | |
294 | from its current device driver. This is achieved by setting the ``hw.nic_uio.bdfs`` | |
295 | kernel environment variable prior to loading ``nic_uio``, as follows:: | |
296 | ||
297 | hw.nic_uio.bdfs="b:d:f,b:d:f,..." | |
298 | ||
299 | Where a comma separated list of selectors is set, the list must not contain any | |
300 | whitespace. | |
301 | ||
302 | For example to re-bind ``ix2@pci0:2:0:0`` and ``ix3@pci0:2:0:1`` to the ``nic_uio`` module | |
303 | upon loading, use the following command:: | |
304 | ||
305 | kenv hw.nic_uio.bdfs="2:0:0,2:0:1" | |
306 | ||
307 | The variable can also be specified during boot by placing the following into | |
308 | ``/boot/loader.conf``, before the previously-described ``nic_uio_load`` line - as | |
309 | shown:: | |
310 | ||
311 | hw.nic_uio.bdfs="2:0:0,2:0:1" | |
312 | nic_uio_load="YES" | |
313 | ||
314 | Binding Network Ports Back to their Original Kernel Driver | |
315 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
316 | ||
317 | If the original driver for a network port has been compiled into the kernel, | |
318 | it is necessary to reboot FreeBSD to restore the original device binding. Before | |
319 | doing so, update or remove the ``hw.nic_uio.bdfs`` in ``/boot/loader.conf``. | |
320 | ||
321 | If rebinding to a driver that is a loadable module, the network port binding can | |
322 | be reset without rebooting. To do so, unload both the target kernel module and the | |
323 | ``nic_uio`` module, modify or clear the ``hw.nic_uio.bdfs`` kernel environment (kenv) | |
324 | value, and reload the two drivers - first the original kernel driver, and then | |
325 | the ``nic_uio driver``. Note: the latter does not need to be reloaded unless there are | |
326 | ports that are still to be bound to it. | |
327 | ||
328 | Example commands to perform these steps are shown below: | |
329 | ||
330 | .. code-block:: console | |
331 | ||
332 | kldunload nic_uio | |
333 | kldunload <original_driver> | |
334 | ||
335 | # To clear the value completely: | |
336 | kenv -u hw.nic_uio.bdfs | |
337 | ||
338 | # To update the list of ports to bind: | |
339 | kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." | |
340 | ||
341 | kldload <original_driver> | |
342 | ||
343 | kldload nic_uio # optional |