ceph/src/seastar/doc/rpc-compression.md

   1 # RPC provided compression infrastructure
   2
   3 ## Compression algorithm negotiation
   4
   5 RPC protocol only defines `COMPRESS` feature bit but does not define format of its data.
   6 If application supports multiple compression algorithms it may use the data for algorithm
   7 negotiation. RPC provides convenience class `multi_algo_compressor_factory` to do it
   8 so that each application will not have to re-implement the same logic. The class gets list
   9 of supported compression algorithms and send them as comma separated list in `COMPRESS` feature
  10 payload. On receiving of the list it matches common algorithm between client and server. In case
  11 there is more than one the order of algorithms in client's list is considered to be a tie breaker
  12 (first algorithm wins).
  13
  14 ## Compression algorithms
  15
  16 ### `LZ4` compressor
  17
  18 This compressor uses LZ4 to compress and decompress RPC messages. It requires all memory buffers to be contiguous, which may force it to temporarily linearise fragmented messages. LZ4 is fast enough to often make the cost of those copies not negligible compared to the cost of the whole compression or decompression routine. Therefore, this algorithm is best suited if there is an upper bound of the message size and they are expected to fit in a single fragment of input and output memory buffers.
  19
  20 ### `LZ4_FRAGMENTED` compressor
  21
  22 This compressor uses LZ4 streaming interface to compress and decompress even large messages without linearising them. The LZ4 streaming routines tend to be slower than the basic ones and the general logic for handling buffers is more complex, so this compressor is best suited only when there is no clear upper bound on the message size or if the messages are expected to be fragmented.
  23
  24 Internally, the compressor processes data in a 32 kB chunks and tries to avoid unnecessary copies as much as possible. It is therefore, recommended, that the application uses memory buffer fragment sizes that are an integral multiple of 32 kB.