ceph/doc/dev/encoding.rst

   1
   2 Serialization (encode/decode)
   3 =============================
   4
   5 When a structure is sent over the network or written to disk, it is
   6 encoded into a string of bytes.  Serializable structures have
   7 ``encode`` and ``decode`` methods that write and read from ``bufferlist``
   8 objects representing byte strings.
   9
  10 Adding a field to a structure
  11 -----------------------------
  12
  13 You can see examples of this all over the Ceph code, but here's an
  14 example:
  15
  16 ::
  17
  18     class AcmeClass
  19     {
  20         int member1;
  21         std::string member2;
  22
  23         void encode(bufferlist &bl)
  24         {
  25             ENCODE_START(1, 1, bl);
  26             ::encode(member1, bl);
  27             ::encode(member2, bl);
  28             ENCODE_FINISH(bl);
  29         }
  30
  31         void decode(bufferlist::iterator &bl)
  32         {
  33             DECODE_START(1, bl);
  34             ::decode(member1, bl);
  35             ::decode(member2, bl);
  36             DECODE_FINISH(bl);
  37         }
  38     };
  39
  40 The ``ENCODE_START`` macro writes a header that specifies a *version* and
  41 a *compat_version* (both initially 1).  The message version is incremented
  42 whenever a change is made to the encoding.  The compat_version is incremented
  43 only if the change will break existing decoders -- decoders are tolerant
  44 of trailing bytes, so changes that add fields at the end of the structure
  45 do not require incrementing compat_version.
  46
  47 The ``DECODE_START`` macro takes an argument specifying the most recent
  48 message version that the code can handle.  This is compared with the
  49 compat_version encoded in the message, and if the message is too new then
  50 an exception will be thrown.  Because changes to compat_verison are rare,
  51 this isn't usually something to worry about when adding fields.
  52
  53 In practice, changes to encoding usually involve simply adding the desired fields
  54 at the end of the ``encode`` and ``decode`` functions, and incrementing
  55 the versions in ``ENCODE_START`` and ``DECODE_START``.  For example, here's how
  56 to add a third field to ``AcmeClass``:
  57
  58 ::
  59
  60     class AcmeClass
  61     {
  62         int member1;
  63         std::string member2;
  64         std::vector<std::string> member3;
  65
  66         void encode(bufferlist &bl)
  67         {
  68             ENCODE_START(2, 1, bl);
  69             ::encode(member1, bl);
  70             ::encode(member2, bl);
  71             ::encode(member3, bl);
  72             ENCODE_FINISH(bl);
  73         }
  74
  75         void decode(bufferlist::iterator &bl)
  76         {
  77             DECODE_START(2, bl);
  78             ::decode(member1, bl);
  79             ::decode(member2, bl);
  80             if (struct_v >= 2) {
  81                 ::decode(member3, bl);
  82             }
  83             DECODE_FINISH(bl);
  84         }
  85     };
  86
  87 Note that the compat_version did not change because the encoded message
  88 will still be decodable by versions of the code that only understand
  89 version 1 -- they will just ignore the trailing bytes where we encode ``member3``.
  90
  91 In the ``decode`` function, decoding the new field is conditional: this is
  92 because we might still be passed older-versioned messages that do not
  93 have the field.  The ``struct_v`` variable is a local set by the ``DECODE_START``
  94 macro.
  95