]>
Commit | Line | Data |
---|---|---|
7c673cae FG |
1 | Managing ABI updates |
2 | ==================== | |
3 | ||
4 | Description | |
5 | ----------- | |
6 | ||
7 | This document details some methods for handling ABI management in the DPDK. | |
8 | Note this document is not exhaustive, in that C library versioning is flexible | |
9 | allowing multiple methods to achieve various goals, but it will provide the user | |
10 | with some introductory methods | |
11 | ||
12 | General Guidelines | |
13 | ------------------ | |
14 | ||
15 | #. Whenever possible, ABI should be preserved | |
16 | #. The libraries marked in experimental state may change without constraint. | |
17 | #. The addition of symbols is generally not problematic | |
18 | #. The modification of symbols can generally be managed with versioning | |
19 | #. The removal of symbols generally is an ABI break and requires bumping of the | |
20 | LIBABIVER macro | |
21 | ||
22 | What is an ABI | |
23 | -------------- | |
24 | ||
25 | An ABI (Application Binary Interface) is the set of runtime interfaces exposed | |
26 | by a library. It is similar to an API (Application Programming Interface) but | |
27 | is the result of compilation. It is also effectively cloned when applications | |
28 | link to dynamic libraries. That is to say when an application is compiled to | |
29 | link against dynamic libraries, it is assumed that the ABI remains constant | |
30 | between the time the application is compiled/linked, and the time that it runs. | |
31 | Therefore, in the case of dynamic linking, it is critical that an ABI is | |
32 | preserved, or (when modified), done in such a way that the application is unable | |
33 | to behave improperly or in an unexpected fashion. | |
34 | ||
35 | The DPDK ABI policy | |
36 | ------------------- | |
37 | ||
38 | ABI versions are set at the time of major release labeling, and the ABI may | |
39 | change multiple times, without warning, between the last release label and the | |
40 | HEAD label of the git tree. | |
41 | ||
42 | ABI versions, once released, are available until such time as their | |
43 | deprecation has been noted in the Release Notes for at least one major release | |
44 | cycle. For example consider the case where the ABI for DPDK 2.0 has been | |
45 | shipped and then a decision is made to modify it during the development of | |
46 | DPDK 2.1. The decision will be recorded in the Release Notes for the DPDK 2.1 | |
47 | release and the modification will be made available in the DPDK 2.2 release. | |
48 | ||
49 | ABI versions may be deprecated in whole or in part as needed by a given | |
50 | update. | |
51 | ||
52 | Some ABI changes may be too significant to reasonably maintain multiple | |
53 | versions. In those cases ABI's may be updated without backward compatibility | |
54 | being provided. The requirements for doing so are: | |
55 | ||
56 | #. At least 3 acknowledgments of the need to do so must be made on the | |
57 | dpdk.org mailing list. | |
58 | ||
59 | #. The changes (including an alternative map file) must be gated with | |
60 | the ``RTE_NEXT_ABI`` option, and provided with a deprecation notice at the | |
61 | same time. | |
62 | It will become the default ABI in the next release. | |
63 | ||
64 | #. A full deprecation cycle, as explained above, must be made to offer | |
65 | downstream consumers sufficient warning of the change. | |
66 | ||
67 | #. At the beginning of the next release cycle, every ``RTE_NEXT_ABI`` | |
68 | conditions will be removed, the ``LIBABIVER`` variable in the makefile(s) | |
69 | where the ABI is changed will be incremented, and the map files will | |
70 | be updated. | |
71 | ||
72 | Note that the above process for ABI deprecation should not be undertaken | |
73 | lightly. ABI stability is extremely important for downstream consumers of the | |
74 | DPDK, especially when distributed in shared object form. Every effort should | |
75 | be made to preserve the ABI whenever possible. The ABI should only be changed | |
76 | for significant reasons, such as performance enhancements. ABI breakage due to | |
77 | changes such as reorganizing public structure fields for aesthetic or | |
78 | readability purposes should be avoided. | |
79 | ||
80 | Examples of Deprecation Notices | |
81 | ------------------------------- | |
82 | ||
83 | The following are some examples of ABI deprecation notices which would be | |
84 | added to the Release Notes: | |
85 | ||
86 | * The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0, | |
87 | to be replaced with the inline function ``rte_foo()``. | |
88 | ||
89 | * The function ``rte_mbuf_grok()`` has been updated to include a new parameter | |
90 | in version 2.0. Backwards compatibility will be maintained for this function | |
91 | until the release of version 2.1 | |
92 | ||
93 | * The members of ``struct rte_foo`` have been reorganized in release 2.0 for | |
94 | performance reasons. Existing binary applications will have backwards | |
95 | compatibility in release 2.0, while newly built binaries will need to | |
96 | reference the new structure variant ``struct rte_foo2``. Compatibility will | |
97 | be removed in release 2.2, and all applications will require updating and | |
98 | rebuilding to the new structure at that time, which will be renamed to the | |
99 | original ``struct rte_foo``. | |
100 | ||
101 | * Significant ABI changes are planned for the ``librte_dostuff`` library. The | |
102 | upcoming release 2.0 will not contain these changes, but release 2.1 will, | |
103 | and no backwards compatibility is planned due to the extensive nature of | |
104 | these changes. Binaries using this library built prior to version 2.1 will | |
105 | require updating and recompilation. | |
106 | ||
107 | Versioning Macros | |
108 | ----------------- | |
109 | ||
110 | When a symbol is exported from a library to provide an API, it also provides a | |
111 | calling convention (ABI) that is embodied in its name, return type and | |
112 | arguments. Occasionally that function may need to change to accommodate new | |
113 | functionality or behavior. When that occurs, it is desirable to allow for | |
114 | backward compatibility for a time with older binaries that are dynamically | |
115 | linked to the DPDK. | |
116 | ||
117 | To support backward compatibility the ``lib/librte_compat/rte_compat.h`` | |
118 | header file provides macros to use when updating exported functions. These | |
119 | macros are used in conjunction with the ``rte_<library>_version.map`` file for | |
120 | a given library to allow multiple versions of a symbol to exist in a shared | |
121 | library so that older binaries need not be immediately recompiled. | |
122 | ||
123 | The macros exported are: | |
124 | ||
125 | * ``VERSION_SYMBOL(b, e, n)``: Creates a symbol version table entry binding | |
126 | versioned symbol ``b@DPDK_n`` to the internal function ``b_e``. | |
127 | ||
128 | * ``BIND_DEFAULT_SYMBOL(b, e, n)``: Creates a symbol version entry instructing | |
129 | the linker to bind references to symbol ``b`` to the internal symbol | |
130 | ``b_e``. | |
131 | ||
132 | * ``MAP_STATIC_SYMBOL(f, p)``: Declare the prototype ``f``, and map it to the | |
133 | fully qualified function ``p``, so that if a symbol becomes versioned, it | |
134 | can still be mapped back to the public symbol name. | |
135 | ||
136 | Examples of ABI Macro use | |
137 | ------------------------- | |
138 | ||
139 | Updating a public API | |
140 | ~~~~~~~~~~~~~~~~~~~~~ | |
141 | ||
142 | Assume we have a function as follows | |
143 | ||
144 | .. code-block:: c | |
145 | ||
146 | /* | |
147 | * Create an acl context object for apps to | |
148 | * manipulate | |
149 | */ | |
150 | struct rte_acl_ctx * | |
151 | rte_acl_create(const struct rte_acl_param *param) | |
152 | { | |
153 | ... | |
154 | } | |
155 | ||
156 | ||
157 | Assume that struct rte_acl_ctx is a private structure, and that a developer | |
158 | wishes to enhance the acl api so that a debugging flag can be enabled on a | |
159 | per-context basis. This requires an addition to the structure (which, being | |
160 | private, is safe), but it also requires modifying the code as follows | |
161 | ||
162 | .. code-block:: c | |
163 | ||
164 | /* | |
165 | * Create an acl context object for apps to | |
166 | * manipulate | |
167 | */ | |
168 | struct rte_acl_ctx * | |
169 | rte_acl_create(const struct rte_acl_param *param, int debug) | |
170 | { | |
171 | ... | |
172 | } | |
173 | ||
174 | ||
175 | Note also that, being a public function, the header file prototype must also be | |
176 | changed, as must all the call sites, to reflect the new ABI footprint. We will | |
177 | maintain previous ABI versions that are accessible only to previously compiled | |
178 | binaries | |
179 | ||
180 | The addition of a parameter to the function is ABI breaking as the function is | |
181 | public, and existing application may use it in its current form. However, the | |
182 | compatibility macros in DPDK allow a developer to use symbol versioning so that | |
183 | multiple functions can be mapped to the same public symbol based on when an | |
184 | application was linked to it. To see how this is done, we start with the | |
185 | requisite libraries version map file. Initially the version map file for the | |
186 | acl library looks like this | |
187 | ||
188 | .. code-block:: none | |
189 | ||
190 | DPDK_2.0 { | |
191 | global: | |
192 | ||
193 | rte_acl_add_rules; | |
194 | rte_acl_build; | |
195 | rte_acl_classify; | |
196 | rte_acl_classify_alg; | |
197 | rte_acl_classify_scalar; | |
198 | rte_acl_create; | |
199 | rte_acl_dump; | |
200 | rte_acl_find_existing; | |
201 | rte_acl_free; | |
202 | rte_acl_ipv4vlan_add_rules; | |
203 | rte_acl_ipv4vlan_build; | |
204 | rte_acl_list_dump; | |
205 | rte_acl_reset; | |
206 | rte_acl_reset_rules; | |
207 | rte_acl_set_ctx_classify; | |
208 | ||
209 | local: *; | |
210 | }; | |
211 | ||
212 | This file needs to be modified as follows | |
213 | ||
214 | .. code-block:: none | |
215 | ||
216 | DPDK_2.0 { | |
217 | global: | |
218 | ||
219 | rte_acl_add_rules; | |
220 | rte_acl_build; | |
221 | rte_acl_classify; | |
222 | rte_acl_classify_alg; | |
223 | rte_acl_classify_scalar; | |
224 | rte_acl_create; | |
225 | rte_acl_dump; | |
226 | rte_acl_find_existing; | |
227 | rte_acl_free; | |
228 | rte_acl_ipv4vlan_add_rules; | |
229 | rte_acl_ipv4vlan_build; | |
230 | rte_acl_list_dump; | |
231 | rte_acl_reset; | |
232 | rte_acl_reset_rules; | |
233 | rte_acl_set_ctx_classify; | |
234 | ||
235 | local: *; | |
236 | }; | |
237 | ||
238 | DPDK_2.1 { | |
239 | global: | |
240 | rte_acl_create; | |
241 | ||
242 | } DPDK_2.0; | |
243 | ||
244 | The addition of the new block tells the linker that a new version node is | |
245 | available (DPDK_2.1), which contains the symbol rte_acl_create, and inherits the | |
246 | symbols from the DPDK_2.0 node. This list is directly translated into a list of | |
247 | exported symbols when DPDK is compiled as a shared library | |
248 | ||
249 | Next, we need to specify in the code which function map to the rte_acl_create | |
250 | symbol at which versions. First, at the site of the initial symbol definition, | |
251 | we need to update the function so that it is uniquely named, and not in conflict | |
252 | with the public symbol name | |
253 | ||
254 | .. code-block:: c | |
255 | ||
256 | struct rte_acl_ctx * | |
257 | -rte_acl_create(const struct rte_acl_param *param) | |
258 | +rte_acl_create_v20(const struct rte_acl_param *param) | |
259 | { | |
260 | size_t sz; | |
261 | struct rte_acl_ctx *ctx; | |
262 | ... | |
263 | ||
264 | Note that the base name of the symbol was kept intact, as this is conducive to | |
265 | the macros used for versioning symbols. That is our next step, mapping this new | |
266 | symbol name to the initial symbol name at version node 2.0. Immediately after | |
267 | the function, we add this line of code | |
268 | ||
269 | .. code-block:: c | |
270 | ||
271 | VERSION_SYMBOL(rte_acl_create, _v20, 2.0); | |
272 | ||
273 | Remembering to also add the rte_compat.h header to the requisite c file where | |
274 | these changes are being made. The above macro instructs the linker to create a | |
275 | new symbol ``rte_acl_create@DPDK_2.0``, which matches the symbol created in older | |
276 | builds, but now points to the above newly named function. We have now mapped | |
277 | the original rte_acl_create symbol to the original function (but with a new | |
278 | name) | |
279 | ||
280 | Next, we need to create the 2.1 version of the symbol. We create a new function | |
281 | name, with a different suffix, and implement it appropriately | |
282 | ||
283 | .. code-block:: c | |
284 | ||
285 | struct rte_acl_ctx * | |
286 | rte_acl_create_v21(const struct rte_acl_param *param, int debug); | |
287 | { | |
288 | struct rte_acl_ctx *ctx = rte_acl_create_v20(param); | |
289 | ||
290 | ctx->debug = debug; | |
291 | ||
292 | return ctx; | |
293 | } | |
294 | ||
295 | This code serves as our new API call. Its the same as our old call, but adds | |
296 | the new parameter in place. Next we need to map this function to the symbol | |
297 | ``rte_acl_create@DPDK_2.1``. To do this, we modify the public prototype of the call | |
298 | in the header file, adding the macro there to inform all including applications, | |
299 | that on re-link, the default rte_acl_create symbol should point to this | |
300 | function. Note that we could do this by simply naming the function above | |
301 | rte_acl_create, and the linker would chose the most recent version tag to apply | |
302 | in the version script, but we can also do this in the header file | |
303 | ||
304 | .. code-block:: c | |
305 | ||
306 | struct rte_acl_ctx * | |
307 | -rte_acl_create(const struct rte_acl_param *param); | |
308 | +rte_acl_create(const struct rte_acl_param *param, int debug); | |
309 | +BIND_DEFAULT_SYMBOL(rte_acl_create, _v21, 2.1); | |
310 | ||
311 | The BIND_DEFAULT_SYMBOL macro explicitly tells applications that include this | |
312 | header, to link to the rte_acl_create_v21 function and apply the DPDK_2.1 | |
313 | version node to it. This method is more explicit and flexible than just | |
314 | re-implementing the exact symbol name, and allows for other features (such as | |
315 | linking to the old symbol version by default, when the new ABI is to be opt-in | |
316 | for a period. | |
317 | ||
318 | One last thing we need to do. Note that we've taken what was a public symbol, | |
319 | and duplicated it into two uniquely and differently named symbols. We've then | |
320 | mapped each of those back to the public symbol ``rte_acl_create`` with different | |
321 | version tags. This only applies to dynamic linking, as static linking has no | |
322 | notion of versioning. That leaves this code in a position of no longer having a | |
323 | symbol simply named ``rte_acl_create`` and a static build will fail on that | |
324 | missing symbol. | |
325 | ||
326 | To correct this, we can simply map a function of our choosing back to the public | |
327 | symbol in the static build with the ``MAP_STATIC_SYMBOL`` macro. Generally the | |
328 | assumption is that the most recent version of the symbol is the one you want to | |
329 | map. So, back in the C file where, immediately after ``rte_acl_create_v21`` is | |
330 | defined, we add this | |
331 | ||
332 | .. code-block:: c | |
333 | ||
334 | struct rte_acl_ctx * | |
335 | rte_acl_create_v21(const struct rte_acl_param *param, int debug) | |
336 | { | |
337 | ... | |
338 | } | |
339 | MAP_STATIC_SYMBOL(struct rte_acl_ctx *rte_acl_create(const struct rte_acl_param *param, int debug), rte_acl_create_v21); | |
340 | ||
341 | That tells the compiler that, when building a static library, any calls to the | |
342 | symbol ``rte_acl_create`` should be linked to ``rte_acl_create_v21`` | |
343 | ||
344 | That's it, on the next shared library rebuild, there will be two versions of | |
345 | rte_acl_create, an old DPDK_2.0 version, used by previously built applications, | |
346 | and a new DPDK_2.1 version, used by future built applications. | |
347 | ||
348 | ||
349 | Deprecating part of a public API | |
350 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
351 | ||
352 | Lets assume that you've done the above update, and after a few releases have | |
353 | passed you decide you would like to retire the old version of the function. | |
354 | After having gone through the ABI deprecation announcement process, removal is | |
355 | easy. Start by removing the symbol from the requisite version map file: | |
356 | ||
357 | .. code-block:: none | |
358 | ||
359 | DPDK_2.0 { | |
360 | global: | |
361 | ||
362 | rte_acl_add_rules; | |
363 | rte_acl_build; | |
364 | rte_acl_classify; | |
365 | rte_acl_classify_alg; | |
366 | rte_acl_classify_scalar; | |
367 | rte_acl_dump; | |
368 | - rte_acl_create | |
369 | rte_acl_find_existing; | |
370 | rte_acl_free; | |
371 | rte_acl_ipv4vlan_add_rules; | |
372 | rte_acl_ipv4vlan_build; | |
373 | rte_acl_list_dump; | |
374 | rte_acl_reset; | |
375 | rte_acl_reset_rules; | |
376 | rte_acl_set_ctx_classify; | |
377 | ||
378 | local: *; | |
379 | }; | |
380 | ||
381 | DPDK_2.1 { | |
382 | global: | |
383 | rte_acl_create; | |
384 | } DPDK_2.0; | |
385 | ||
386 | ||
387 | Next remove the corresponding versioned export. | |
388 | ||
389 | .. code-block:: c | |
390 | ||
391 | -VERSION_SYMBOL(rte_acl_create, _v20, 2.0); | |
392 | ||
393 | ||
394 | Note that the internal function definition could also be removed, but its used | |
395 | in our example by the newer version _v21, so we leave it in place. This is a | |
396 | coding style choice. | |
397 | ||
398 | Lastly, we need to bump the LIBABIVER number for this library in the Makefile to | |
399 | indicate to applications doing dynamic linking that this is a later, and | |
400 | possibly incompatible library version: | |
401 | ||
402 | .. code-block:: c | |
403 | ||
404 | -LIBABIVER := 1 | |
405 | +LIBABIVER := 2 | |
406 | ||
407 | Deprecating an entire ABI version | |
408 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
409 | ||
410 | While removing a symbol from and ABI may be useful, it is often more practical | |
411 | to remove an entire version node at once. If a version node completely | |
412 | specifies an API, then removing part of it, typically makes it incomplete. In | |
413 | those cases it is better to remove the entire node | |
414 | ||
415 | To do this, start by modifying the version map file, such that all symbols from | |
416 | the node to be removed are merged into the next node in the map | |
417 | ||
418 | In the case of our map above, it would transform to look as follows | |
419 | ||
420 | .. code-block:: none | |
421 | ||
422 | DPDK_2.1 { | |
423 | global: | |
424 | ||
425 | rte_acl_add_rules; | |
426 | rte_acl_build; | |
427 | rte_acl_classify; | |
428 | rte_acl_classify_alg; | |
429 | rte_acl_classify_scalar; | |
430 | rte_acl_dump; | |
431 | rte_acl_create | |
432 | rte_acl_find_existing; | |
433 | rte_acl_free; | |
434 | rte_acl_ipv4vlan_add_rules; | |
435 | rte_acl_ipv4vlan_build; | |
436 | rte_acl_list_dump; | |
437 | rte_acl_reset; | |
438 | rte_acl_reset_rules; | |
439 | rte_acl_set_ctx_classify; | |
440 | ||
441 | local: *; | |
442 | }; | |
443 | ||
444 | Then any uses of BIND_DEFAULT_SYMBOL that pointed to the old node should be | |
445 | updated to point to the new version node in any header files for all affected | |
446 | symbols. | |
447 | ||
448 | .. code-block:: c | |
449 | ||
450 | -BIND_DEFAULT_SYMBOL(rte_acl_create, _v20, 2.0); | |
451 | +BIND_DEFAULT_SYMBOL(rte_acl_create, _v21, 2.1); | |
452 | ||
453 | Lastly, any VERSION_SYMBOL macros that point to the old version node should be | |
454 | removed, taking care to keep, where need old code in place to support newer | |
455 | versions of the symbol. | |
456 | ||
457 | Running the ABI Validator | |
458 | ------------------------- | |
459 | ||
460 | The ``scripts`` directory in the DPDK source tree contains a utility program, | |
461 | ``validate-abi.sh``, for validating the DPDK ABI based on the Linux `ABI | |
462 | Compliance Checker | |
463 | <http://ispras.linuxbase.org/index.php/ABI_compliance_checker>`_. | |
464 | ||
465 | This has a dependency on the ``abi-compliance-checker`` and ``and abi-dumper`` | |
466 | utilities which can be installed via a package manager. For example:: | |
467 | ||
468 | sudo yum install abi-compliance-checker | |
469 | sudo yum install abi-dumper | |
470 | ||
471 | The syntax of the ``validate-abi.sh`` utility is:: | |
472 | ||
473 | ./scripts/validate-abi.sh <REV1> <REV2> <TARGET> | |
474 | ||
475 | Where ``REV1`` and ``REV2`` are valid gitrevisions(7) | |
476 | https://www.kernel.org/pub/software/scm/git/docs/gitrevisions.html | |
477 | on the local repo and target is the usual DPDK compilation target. | |
478 | ||
479 | For example:: | |
480 | ||
481 | # Check between the previous and latest commit: | |
482 | ./scripts/validate-abi.sh HEAD~1 HEAD x86_64-native-linuxapp-gcc | |
483 | ||
484 | # Check between two tags: | |
485 | ./scripts/validate-abi.sh v2.0.0 v2.1.0 x86_64-native-linuxapp-gcc | |
486 | ||
487 | # Check between git master and local topic-branch "vhost-hacking": | |
488 | ./scripts/validate-abi.sh master vhost-hacking x86_64-native-linuxapp-gcc | |
489 | ||
490 | After the validation script completes (it can take a while since it need to | |
491 | compile both tags) it will create compatibility reports in the | |
492 | ``./compat_report`` directory. Listed incompatibilities can be found as | |
493 | follows:: | |
494 | ||
495 | grep -lr Incompatible compat_reports/ |