ceph/src/zstd/lib/README.md

   1 Zstandard library files
   2 ================================
   3
   4 The __lib__ directory is split into several sub-directories,
   5 in order to make it easier to select or exclude features.
   6
   7
   8 #### Building
   9
  10 `Makefile` script is provided, supporting [Makefile conventions](https://www.gnu.org/prep/standards/html_node/Makefile-Conventions.html#Makefile-Conventions),
  11 including commands variables, staged install, directory variables and standard targets.
  12 - `make` : generates both static and dynamic libraries
  13 - `make install` : install libraries and headers in target system directories
  14
  15 `libzstd` default scope is pretty large, including compression, decompression, dictionary builder,
  16 and support for decoding legacy formats >= v0.5.0.
  17 The scope can be reduced on demand (see paragraph _modular build_).
  18
  19
  20 #### Multithreading support
  21
  22 Multithreading is disabled by default when building with `make`.
  23 Enabling multithreading requires 2 conditions :
  24 - set build macro `ZSTD_MULTITHREAD` (`-DZSTD_MULTITHREAD` for `gcc`)
  25 - for POSIX systems : compile with pthread (`-pthread` compilation flag for `gcc`)
  26
  27 Both conditions are automatically applied when invoking `make lib-mt` target.
  28
  29 When linking a POSIX program with a multithreaded version of `libzstd`,
  30 note that it's necessary to request the `-pthread` flag during link stage.
  31
  32 Multithreading capabilities are exposed
  33 via the [advanced API defined in `lib/zstd.h`](https://github.com/facebook/zstd/blob/v1.3.8/lib/zstd.h#L592).
  34
  35
  36 #### API
  37
  38 Zstandard's stable API is exposed within [lib/zstd.h](zstd.h).
  39
  40
  41 #### Advanced API
  42
  43 Optional advanced features are exposed via :
  44
  45 - `lib/common/zstd_errors.h` : translates `size_t` function results
  46                                into a `ZSTD_ErrorCode`, for accurate error handling.
  47
  48 - `ZSTD_STATIC_LINKING_ONLY` : if this macro is defined _before_ including `zstd.h`,
  49                           it unlocks access to the experimental API,
  50                           exposed in the second part of `zstd.h`.
  51                           All definitions in the experimental APIs are unstable,
  52                           they may still change in the future, or even be removed.
  53                           As a consequence, experimental definitions shall ___never be used with dynamic library___ !
  54                           Only static linking is allowed.
  55
  56
  57 #### Modular build
  58
  59 It's possible to compile only a limited set of features within `libzstd`.
  60 The file structure is designed to make this selection manually achievable for any build system :
  61
  62 - Directory `lib/common` is always required, for all variants.
  63
  64 - Compression source code lies in `lib/compress`
  65
  66 - Decompression source code lies in `lib/decompress`
  67
  68 - It's possible to include only `compress` or only `decompress`, they don't depend on each other.
  69
  70 - `lib/dictBuilder` : makes it possible to generate dictionaries from a set of samples.
  71         The API is exposed in `lib/dictBuilder/zdict.h`.
  72         This module depends on both `lib/common` and `lib/compress` .
  73
  74 - `lib/legacy` : makes it possible to decompress legacy zstd formats, starting from `v0.1.0`.
  75         This module depends on `lib/common` and `lib/decompress`.
  76         To enable this feature, define `ZSTD_LEGACY_SUPPORT` during compilation.
  77         Specifying a number limits versions supported to that version onward.
  78         For example, `ZSTD_LEGACY_SUPPORT=2` means : "support legacy formats >= v0.2.0".
  79         Conversely, `ZSTD_LEGACY_SUPPORT=0` means "do __not__ support legacy formats".
  80         By default, this build macro is set as `ZSTD_LEGACY_SUPPORT=5`.
  81         Decoding supported legacy format is a transparent capability triggered within decompression functions.
  82         It's also allowed to invoke legacy API directly, exposed in `lib/legacy/zstd_legacy.h`.
  83         Each version does also provide its own set of advanced API.
  84         For example, advanced API for version `v0.4` is exposed in `lib/legacy/zstd_v04.h` .
  85
  86 - While invoking `make libzstd`, it's possible to define build macros
  87         `ZSTD_LIB_COMPRESSION, ZSTD_LIB_DECOMPRESSION`, `ZSTD_LIB_DICTBUILDER`,
  88         and `ZSTD_LIB_DEPRECATED` as `0` to forgo compilation of the corresponding features.
  89         This will also disable compilation of all dependencies
  90         (eg. `ZSTD_LIB_COMPRESSION=0` will also disable dictBuilder).
  91
  92 - There are some additional build macros that can be used to minify the decoder.
  93
  94   Zstandard often has more than one implementation of a piece of functionality,
  95   where each implementation optimizes for different scenarios. For example, the
  96   Huffman decoder has complementary implementations that decode the stream one
  97   symbol at a time or two symbols at a time. Zstd normally includes both (and
  98   dispatches between them at runtime), but by defining `HUF_FORCE_DECOMPRESS_X1`
  99   or `HUF_FORCE_DECOMPRESS_X2`, you can force the use of one or the other, avoiding
 100   compilation of the other. Similarly, `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT`
 101   and `ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG` force the compilation and use of
 102   only one or the other of two decompression implementations. The smallest
 103   binary is achieved by using `HUF_FORCE_DECOMPRESS_X1` and
 104   `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT`.
 105
 106   For squeezing the last ounce of size out, you can also define
 107   `ZSTD_NO_INLINE`, which disables inlining, and `ZSTD_STRIP_ERROR_STRINGS`,
 108   which removes the error messages that are otherwise returned by
 109   `ZSTD_getErrorName`.
 110
 111 - While invoking `make libzstd`, the build macro `ZSTD_LEGACY_MULTITHREADED_API=1`
 112   will expose the deprecated `ZSTDMT` API exposed by `zstdmt_compress.h` in
 113   the shared library, which is now hidden by default.
 114
 115
 116 #### Windows : using MinGW+MSYS to create DLL
 117
 118 DLL can be created using MinGW+MSYS with the `make libzstd` command.
 119 This command creates `dll\libzstd.dll` and the import library `dll\libzstd.lib`.
 120 The import library is only required with Visual C++.
 121 The header file `zstd.h` and the dynamic library `dll\libzstd.dll` are required to
 122 compile a project using gcc/MinGW.
 123 The dynamic library has to be added to linking options.
 124 It means that if a project that uses ZSTD consists of a single `test-dll.c`
 125 file it should be linked with `dll\libzstd.dll`. For example:
 126 ```
 127     gcc $(CFLAGS) -Iinclude/ test-dll.c -o test-dll dll\libzstd.dll
 128 ```
 129 The compiled executable will require ZSTD DLL which is available at `dll\libzstd.dll`.
 130
 131
 132 #### Deprecated API
 133
 134 Obsolete API on their way out are stored in directory `lib/deprecated`.
 135 At this stage, it contains older streaming prototypes, in `lib/deprecated/zbuff.h`.
 136 These prototypes will be removed in some future version.
 137 Consider migrating code towards supported streaming API exposed in `zstd.h`.
 138
 139
 140 #### Miscellaneous
 141
 142 The other files are not source code. There are :
 143
 144  - `BUCK` : support for `buck` build system (https://buckbuild.com/)
 145  - `Makefile` : `make` script to build and install zstd library (static and dynamic)
 146  - `README.md` : this file
 147  - `dll/` : resources directory for Windows compilation
 148  - `libzstd.pc.in` : script for `pkg-config` (used in `make install`)