[ceph.git] / ceph / src / spdk / dpdk / doc / guides / compressdevs / isal.rst

..  SPDX-License-Identifier: BSD-3-Clause
    Copyright(c) 2018 Intel Corporation.

ISA-L Compression Poll Mode Driver
==================================

The ISA-L PMD (**librte_pmd_isal_comp**) provides poll mode compression &
decompression driver support for utilizing Intel ISA-L library,
which implements the deflate algorithm for both Deflate(compression) and Inflate(decompression).


Features
--------

ISA-L PMD has support for:

Compression/Decompression algorithm:

    * DEFLATE

Huffman code type:

    * FIXED
    * DYNAMIC

Window size support:

    * 32K

Checksum:

    * CRC32
    * ADLER32

To enable a checksum in the driver, the compression and/or decompression xform
structure, rte_comp_xform, must be filled with either of the CompressDev
checksum flags supported. ::

 compress_xform->compress.chksum = RTE_COMP_CHECKSUM_CRC32

 decompress_xform->decompress.chksum = RTE_COMP_CHECKSUM_CRC32

::

 compress_xform->compress.chksum = RTE_COMP_CHECKSUM_ADLER32

 decompress_xform->decompress.chksum = RTE_COMP_CHECKSUM_ADLER32

If you request a checksum for compression or decompression,
the checksum field in the operation structure,  ``op->output_chksum``,
will be filled with the checksum.

.. Note::

 For the compression case above, your output buffer will need to be large enough to hold the compressed data plus a scratchpad for the checksum at the end, the scratchpad is 8 bytes for CRC32 and 4 bytes for Adler32.

Level guide:

The ISA-L levels have been mapped to somewhat correspond to the same ZLIB level,
i.e. ZLIB L1 gives a compression ratio similar to ISA-L L1.
Compressdev level 0 enables "No Compression", which passes the uncompressed
data to the output buffer, plus deflate headers.
The ISA-L library does not support this, therefore compressdev level 0 is not supported.

The compressdev API has 10 levels, 0-9. ISA-L has 4 levels of compression, 0-3.
As a result the level mappings from the API to the PMD are shown below.

.. _table_ISA-L_compression_levels:

.. table:: Level mapping from Compressdev to ISA-L PMD.

   +-------------+----------------------------------------------+-----------------------------------------------+
   | Compressdev | PMD Functionality                            | Internal ISA-L                                |
   | API Level   |                                              | Level                                         |
   +=============+==============================================+===============================================+
   | 0           | No compression, Not Supported                | ---                                           |
   +-------------+----------------------------------------------+-----------------------------------------------+
   | 1           | Dynamic (Fast compression)                   | 1                                             |
   +-------------+----------------------------------------------+-----------------------------------------------+
   | 2           | Dynamic                                      | 2                                             |
   |             | (Higher compression ratio)                   |                                               |
   +-------------+----------------------------------------------+-----------------------------------------------+
   | 3           | Dynamic                                      | 3                                             |
   |             | (Best compression ratio)                     | (Level 2 if                                   |
   |             |                                              | no AVX512/AVX2)                               |
   +-------------+----------------------------------------------+-----------------------------------------------+
   | 4           | Dynamic (Best compression ratio)             | Same as above                                 |
   +-------------+----------------------------------------------+-----------------------------------------------+
   | 5           | Dynamic (Best compression ratio)             | Same as above                                 |
   +-------------+----------------------------------------------+-----------------------------------------------+
   | 6           | Dynamic (Best compression ratio)             | Same as above                                 |
   +-------------+----------------------------------------------+-----------------------------------------------+
   | 7           | Dynamic (Best compression ratio)             | Same as above                                 |
   +-------------+----------------------------------------------+-----------------------------------------------+
   | 8           | Dynamic (Best compression ratio)             | Same as above                                 |
   +-------------+----------------------------------------------+-----------------------------------------------+
   | 9           | Dynamic (Best compression ratio)             | Same as above                                 |
   +-------------+----------------------------------------------+-----------------------------------------------+

.. Note::

 The above table only shows mapping when API calls for dynamic compression.
 For fixed compression, regardless of API level, internally ISA-L level 0 is always used.


Limitations
-----------

* Compressdev level 0, no compression, is not supported.

Installation
------------

* To build DPDK with Intel's ISA-L library, the user is required to download the library from `<https://github.com/01org/isa-l>`_.

* Once downloaded, the user needs to build the library, the ISA-L autotools are usually sufficient::

    ./autogen.sh
    ./configure

* make can  be used to install the library on their system, before building DPDK::

    make
    sudo make install

* To build with meson, the **libisal.pc** file, must be copied into "pkgconfig",
  e.g. /usr/lib/pkgconfig or /usr/lib64/pkgconfig depending on your system,
  for meson to find the ISA-L library. The **libisal.pc** is located in library sources::

    cp isal/libisal.pc /usr/lib/pkgconfig/


Initialization
--------------

In order to enable this virtual compression PMD, user must:

* Set ``CONFIG_RTE_LIBRTE_PMD_ISAL=y`` in config/common_base.

To use the PMD in an application, user must:

* Call ``rte_vdev_init("compress_isal")`` within the application.

* Use ``--vdev="compress_isal"`` in the EAL options, which will call ``rte_vdev_init()`` internally.

The following parameter (optional) can be provided in the previous two calls:

* ``socket_id:`` Specify the socket where the memory for the device is going to be allocated
  (by default, socket_id will be the socket where the core that is creating the PMD is running on).
Commit	Line	Data
11fdf7f2 TL	1	.. SPDX-License-Identifier: BSD-3-Clause
	2	Copyright(c) 2018 Intel Corporation.
	3
	4	ISA-L Compression Poll Mode Driver
	5	==================================
	6
	7	The ISA-L PMD (librte_pmd_isal_comp) provides poll mode compression &
	8	decompression driver support for utilizing Intel ISA-L library,
	9	which implements the deflate algorithm for both Deflate(compression) and Inflate(decompression).
	10
	11
	12	Features
	13	--------
	14
	15	ISA-L PMD has support for:
	16
	17	Compression/Decompression algorithm:
	18
	19	* DEFLATE
	20
	21	Huffman code type:
	22
	23	* FIXED
	24	* DYNAMIC
	25
	26	Window size support:
	27
	28	* 32K
	29
9f95a23c TL	30	Checksum:
	31
	32	* CRC32
	33	* ADLER32
	34
	35	To enable a checksum in the driver, the compression and/or decompression xform
	36	structure, rte_comp_xform, must be filled with either of the CompressDev
	37	checksum flags supported. ::
	38
	39	compress_xform->compress.chksum = RTE_COMP_CHECKSUM_CRC32
	40
	41	decompress_xform->decompress.chksum = RTE_COMP_CHECKSUM_CRC32
	42
	43	::
	44
	45	compress_xform->compress.chksum = RTE_COMP_CHECKSUM_ADLER32
	46
	47	decompress_xform->decompress.chksum = RTE_COMP_CHECKSUM_ADLER32
	48
	49	If you request a checksum for compression or decompression,
	50	the checksum field in the operation structure, ``op->output_chksum``,
	51	will be filled with the checksum.
	52
	53	.. Note::
	54
	55	For the compression case above, your output buffer will need to be large enough to hold the compressed data plus a scratchpad for the checksum at the end, the scratchpad is 8 bytes for CRC32 and 4 bytes for Adler32.
	56
11fdf7f2 TL	57	Level guide:
	58
	59	The ISA-L levels have been mapped to somewhat correspond to the same ZLIB level,
	60	i.e. ZLIB L1 gives a compression ratio similar to ISA-L L1.
	61	Compressdev level 0 enables "No Compression", which passes the uncompressed
	62	data to the output buffer, plus deflate headers.
	63	The ISA-L library does not support this, therefore compressdev level 0 is not supported.
	64
	65	The compressdev API has 10 levels, 0-9. ISA-L has 4 levels of compression, 0-3.
	66	As a result the level mappings from the API to the PMD are shown below.
	67
	68	.. _table_ISA-L_compression_levels:
	69
	70	.. table:: Level mapping from Compressdev to ISA-L PMD.
	71
	72	+-------------+----------------------------------------------+-----------------------------------------------+
	73	\| Compressdev \| PMD Functionality \| Internal ISA-L \|
	74	\| API Level \| \| Level \|
	75	+=============+==============================================+===============================================+
	76	\| 0 \| No compression, Not Supported \| --- \|
	77	+-------------+----------------------------------------------+-----------------------------------------------+
	78	\| 1 \| Dynamic (Fast compression) \| 1 \|
	79	+-------------+----------------------------------------------+-----------------------------------------------+
	80	\| 2 \| Dynamic \| 2 \|
	81	\| \| (Higher compression ratio) \| \|
	82	+-------------+----------------------------------------------+-----------------------------------------------+
	83	\| 3 \| Dynamic \| 3 \|
	84	\| \| (Best compression ratio) \| (Level 2 if \|
	85	\| \| \| no AVX512/AVX2) \|
	86	+-------------+----------------------------------------------+-----------------------------------------------+
	87	\| 4 \| Dynamic (Best compression ratio) \| Same as above \|
	88	+-------------+----------------------------------------------+-----------------------------------------------+
	89	\| 5 \| Dynamic (Best compression ratio) \| Same as above \|
	90	+-------------+----------------------------------------------+-----------------------------------------------+
	91	\| 6 \| Dynamic (Best compression ratio) \| Same as above \|
	92	+-------------+----------------------------------------------+-----------------------------------------------+
	93	\| 7 \| Dynamic (Best compression ratio) \| Same as above \|
	94	+-------------+----------------------------------------------+-----------------------------------------------+
	95	\| 8 \| Dynamic (Best compression ratio) \| Same as above \|
	96	+-------------+----------------------------------------------+-----------------------------------------------+
	97	\| 9 \| Dynamic (Best compression ratio) \| Same as above \|
	98	+-------------+----------------------------------------------+-----------------------------------------------+
	99
	100	.. Note::
	101
	102	The above table only shows mapping when API calls for dynamic compression.
	103	For fixed compression, regardless of API level, internally ISA-L level 0 is always used.
	104
9f95a23c	105
11fdf7f2 TL	106	Limitations
	107	-----------
	108
	109	* Compressdev level 0, no compression, is not supported.
	110
11fdf7f2 TL	111	Installation
	112	------------
	113
	114	* To build DPDK with Intel's ISA-L library, the user is required to download the library from `<https://github.com/01org/isa-l>`_.
	115
	116	* Once downloaded, the user needs to build the library, the ISA-L autotools are usually sufficient::
	117
	118	./autogen.sh
	119	./configure
	120
	121	* make can be used to install the library on their system, before building DPDK::
	122
	123	make
	124	sudo make install
	125
	126	* To build with meson, the libisal.pc file, must be copied into "pkgconfig",
	127	e.g. /usr/lib/pkgconfig or /usr/lib64/pkgconfig depending on your system,
	128	for meson to find the ISA-L library. The libisal.pc is located in library sources::
	129
	130	cp isal/libisal.pc /usr/lib/pkgconfig/
	131
	132
	133	Initialization
	134	--------------
	135
	136	In order to enable this virtual compression PMD, user must:
	137
	138	* Set ``CONFIG_RTE_LIBRTE_PMD_ISAL=y`` in config/common_base.
	139
	140	To use the PMD in an application, user must:
	141
	142	* Call ``rte_vdev_init("compress_isal")`` within the application.
	143
	144	* Use ``--vdev="compress_isal"`` in the EAL options, which will call ``rte_vdev_init()`` internally.
	145
	146	The following parameter (optional) can be provided in the previous two calls:
	147
	148	* ``socket_id:`` Specify the socket where the memory for the device is going to be allocated
	149	(by default, socket_id will be the socket where the core that is creating the PMD is running on).