]>
Commit | Line | Data |
---|---|---|
7c673cae FG |
1 | Zstandard library files |
2 | ================================ | |
3 | ||
11fdf7f2 | 4 | The __lib__ directory is split into several sub-directories, |
9f95a23c | 5 | in order to make it easier to select or exclude features. |
11fdf7f2 TL |
6 | |
7 | ||
8 | #### Building | |
9 | ||
9f95a23c TL |
10 | `Makefile` script is provided, supporting [Makefile conventions](https://www.gnu.org/prep/standards/html_node/Makefile-Conventions.html#Makefile-Conventions), |
11 | including commands variables, staged install, directory variables and standard targets. | |
11fdf7f2 | 12 | - `make` : generates both static and dynamic libraries |
9f95a23c TL |
13 | - `make install` : install libraries and headers in target system directories |
14 | ||
15 | `libzstd` default scope is pretty large, including compression, decompression, dictionary builder, | |
16 | and support for decoding legacy formats >= v0.5.0. | |
17 | The scope can be reduced on demand (see paragraph _modular build_). | |
18 | ||
19 | ||
20 | #### Multithreading support | |
21 | ||
22 | Multithreading is disabled by default when building with `make`. | |
23 | Enabling multithreading requires 2 conditions : | |
24 | - set build macro `ZSTD_MULTITHREAD` (`-DZSTD_MULTITHREAD` for `gcc`) | |
25 | - for POSIX systems : compile with pthread (`-pthread` compilation flag for `gcc`) | |
26 | ||
27 | Both conditions are automatically applied when invoking `make lib-mt` target. | |
28 | ||
29 | When linking a POSIX program with a multithreaded version of `libzstd`, | |
f67539c2 | 30 | note that it's necessary to invoke the `-pthread` flag during link stage. |
9f95a23c TL |
31 | |
32 | Multithreading capabilities are exposed | |
f67539c2 | 33 | via the [advanced API defined in `lib/zstd.h`](https://github.com/facebook/zstd/blob/v1.4.3/lib/zstd.h#L351). |
7c673cae FG |
34 | |
35 | ||
36 | #### API | |
37 | ||
11fdf7f2 | 38 | Zstandard's stable API is exposed within [lib/zstd.h](zstd.h). |
7c673cae FG |
39 | |
40 | ||
41 | #### Advanced API | |
42 | ||
11fdf7f2 TL |
43 | Optional advanced features are exposed via : |
44 | ||
45 | - `lib/common/zstd_errors.h` : translates `size_t` function results | |
9f95a23c TL |
46 | into a `ZSTD_ErrorCode`, for accurate error handling. |
47 | ||
11fdf7f2 | 48 | - `ZSTD_STATIC_LINKING_ONLY` : if this macro is defined _before_ including `zstd.h`, |
9f95a23c TL |
49 | it unlocks access to the experimental API, |
50 | exposed in the second part of `zstd.h`. | |
51 | All definitions in the experimental APIs are unstable, | |
52 | they may still change in the future, or even be removed. | |
53 | As a consequence, experimental definitions shall ___never be used with dynamic library___ ! | |
7c673cae FG |
54 | Only static linking is allowed. |
55 | ||
56 | ||
57 | #### Modular build | |
58 | ||
9f95a23c TL |
59 | It's possible to compile only a limited set of features within `libzstd`. |
60 | The file structure is designed to make this selection manually achievable for any build system : | |
11fdf7f2 | 61 | |
9f95a23c | 62 | - Directory `lib/common` is always required, for all variants. |
11fdf7f2 | 63 | |
9f95a23c | 64 | - Compression source code lies in `lib/compress` |
11fdf7f2 | 65 | |
9f95a23c | 66 | - Decompression source code lies in `lib/decompress` |
11fdf7f2 | 67 | |
9f95a23c | 68 | - It's possible to include only `compress` or only `decompress`, they don't depend on each other. |
11fdf7f2 | 69 | |
9f95a23c TL |
70 | - `lib/dictBuilder` : makes it possible to generate dictionaries from a set of samples. |
71 | The API is exposed in `lib/dictBuilder/zdict.h`. | |
72 | This module depends on both `lib/common` and `lib/compress` . | |
73 | ||
74 | - `lib/legacy` : makes it possible to decompress legacy zstd formats, starting from `v0.1.0`. | |
75 | This module depends on `lib/common` and `lib/decompress`. | |
76 | To enable this feature, define `ZSTD_LEGACY_SUPPORT` during compilation. | |
77 | Specifying a number limits versions supported to that version onward. | |
78 | For example, `ZSTD_LEGACY_SUPPORT=2` means : "support legacy formats >= v0.2.0". | |
79 | Conversely, `ZSTD_LEGACY_SUPPORT=0` means "do __not__ support legacy formats". | |
80 | By default, this build macro is set as `ZSTD_LEGACY_SUPPORT=5`. | |
81 | Decoding supported legacy format is a transparent capability triggered within decompression functions. | |
82 | It's also allowed to invoke legacy API directly, exposed in `lib/legacy/zstd_legacy.h`. | |
83 | Each version does also provide its own set of advanced API. | |
84 | For example, advanced API for version `v0.4` is exposed in `lib/legacy/zstd_v04.h` . | |
85 | ||
86 | - While invoking `make libzstd`, it's possible to define build macros | |
87 | `ZSTD_LIB_COMPRESSION, ZSTD_LIB_DECOMPRESSION`, `ZSTD_LIB_DICTBUILDER`, | |
f67539c2 TL |
88 | and `ZSTD_LIB_DEPRECATED` as `0` to forgo compilation of the |
89 | corresponding features. This will also disable compilation of all | |
90 | dependencies (eg. `ZSTD_LIB_COMPRESSION=0` will also disable | |
91 | dictBuilder). | |
92 | ||
93 | - There are a number of options that can help minimize the binary size of | |
94 | `libzstd`. | |
95 | ||
96 | The first step is to select the components needed (using the above-described | |
97 | `ZSTD_LIB_COMPRESSION` etc.). | |
98 | ||
99 | The next step is to set `ZSTD_LIB_MINIFY` to `1` when invoking `make`. This | |
100 | disables various optional components and changes the compilation flags to | |
101 | prioritize space-saving. | |
102 | ||
103 | Detailed options: Zstandard's code and build environment is set up by default | |
104 | to optimize above all else for performance. In pursuit of this goal, Zstandard | |
105 | makes significant trade-offs in code size. For example, Zstandard often has | |
106 | more than one implementation of a particular component, with each | |
107 | implementation optimized for different scenarios. For example, the Huffman | |
108 | decoder has complementary implementations that decode the stream one symbol at | |
109 | a time or two symbols at a time. Zstd normally includes both (and dispatches | |
110 | between them at runtime), but by defining `HUF_FORCE_DECOMPRESS_X1` or | |
111 | `HUF_FORCE_DECOMPRESS_X2`, you can force the use of one or the other, avoiding | |
9f95a23c TL |
112 | compilation of the other. Similarly, `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT` |
113 | and `ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG` force the compilation and use of | |
114 | only one or the other of two decompression implementations. The smallest | |
115 | binary is achieved by using `HUF_FORCE_DECOMPRESS_X1` and | |
f67539c2 | 116 | `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT` (implied by `ZSTD_LIB_MINIFY`). |
9f95a23c TL |
117 | |
118 | For squeezing the last ounce of size out, you can also define | |
119 | `ZSTD_NO_INLINE`, which disables inlining, and `ZSTD_STRIP_ERROR_STRINGS`, | |
120 | which removes the error messages that are otherwise returned by | |
f67539c2 TL |
121 | `ZSTD_getErrorName` (implied by `ZSTD_LIB_MINIFY`). |
122 | ||
123 | Finally, when integrating into your application, make sure you're doing link- | |
124 | time optimation and unused symbol garbage collection (via some combination of, | |
125 | e.g., `-flto`, `-ffat-lto-objects`, `-fuse-linker-plugin`, | |
126 | `-ffunction-sections`, `-fdata-sections`, `-fmerge-all-constants`, | |
127 | `-Wl,--gc-sections`, `-Wl,-z,norelro`, and an archiver that understands | |
128 | the compiler's intermediate representation, e.g., `AR=gcc-ar`). Consult your | |
129 | compiler's documentation. | |
9f95a23c TL |
130 | |
131 | - While invoking `make libzstd`, the build macro `ZSTD_LEGACY_MULTITHREADED_API=1` | |
132 | will expose the deprecated `ZSTDMT` API exposed by `zstdmt_compress.h` in | |
133 | the shared library, which is now hidden by default. | |
11fdf7f2 | 134 | |
f67539c2 TL |
135 | - The build macro `DYNAMIC_BMI2` can be set to 1 or 0 in order to generate binaries |
136 | which can detect at runtime the presence of BMI2 instructions, and use them only if present. | |
137 | These instructions contribute to better performance, notably on the decoder side. | |
138 | By default, this feature is automatically enabled on detecting | |
139 | the right instruction set (x64) and compiler (clang or gcc >= 5). | |
140 | It's obviously disabled for different cpus, | |
141 | or when BMI2 instruction set is _required_ by the compiler command line | |
142 | (in this case, only the BMI2 code path is generated). | |
143 | Setting this macro will either force to generate the BMI2 dispatcher (1) | |
144 | or prevent it (0). It overrides automatic detection. | |
145 | ||
11fdf7f2 TL |
146 | |
147 | #### Windows : using MinGW+MSYS to create DLL | |
7c673cae FG |
148 | |
149 | DLL can be created using MinGW+MSYS with the `make libzstd` command. | |
150 | This command creates `dll\libzstd.dll` and the import library `dll\libzstd.lib`. | |
151 | The import library is only required with Visual C++. | |
152 | The header file `zstd.h` and the dynamic library `dll\libzstd.dll` are required to | |
153 | compile a project using gcc/MinGW. | |
154 | The dynamic library has to be added to linking options. | |
155 | It means that if a project that uses ZSTD consists of a single `test-dll.c` | |
156 | file it should be linked with `dll\libzstd.dll`. For example: | |
157 | ``` | |
158 | gcc $(CFLAGS) -Iinclude/ test-dll.c -o test-dll dll\libzstd.dll | |
159 | ``` | |
160 | The compiled executable will require ZSTD DLL which is available at `dll\libzstd.dll`. | |
161 | ||
162 | ||
11fdf7f2 | 163 | #### Deprecated API |
7c673cae | 164 | |
11fdf7f2 TL |
165 | Obsolete API on their way out are stored in directory `lib/deprecated`. |
166 | At this stage, it contains older streaming prototypes, in `lib/deprecated/zbuff.h`. | |
11fdf7f2 TL |
167 | These prototypes will be removed in some future version. |
168 | Consider migrating code towards supported streaming API exposed in `zstd.h`. | |
7c673cae FG |
169 | |
170 | ||
171 | #### Miscellaneous | |
172 | ||
173 | The other files are not source code. There are : | |
174 | ||
11fdf7f2 | 175 | - `BUCK` : support for `buck` build system (https://buckbuild.com/) |
9f95a23c | 176 | - `Makefile` : `make` script to build and install zstd library (static and dynamic) |
11fdf7f2 | 177 | - `README.md` : this file |
9f95a23c TL |
178 | - `dll/` : resources directory for Windows compilation |
179 | - `libzstd.pc.in` : script for `pkg-config` (used in `make install`) |