projects / proxmox-backup.git / blame
| Commit | Line | Data |
|---|---|---|
| 998b827a DM |
1 | File Formats |
| 2 | ============ | |
| 3 | ||
| 85e139b7 DM |
4 | .. _pxar-format: |
| 5 | ||
| 998b827a DM |
6 | Proxmox File Archive Format (``.pxar``) |
| 7 | --------------------------------------- | |
| 8 | ||
| 998b827a DM |
9 | .. graphviz:: pxar-format-overview.dot |
| 10 | ||
| ec07a280 DM |
11 | |
| 12 | .. _data-blob-format: | |
| 13 | ||
| e57aa36d DM |
14 | Data Blob Format (``.blob``) |
| 15 | ---------------------------- | |
| ec07a280 DM |
16 | |
| 17 | The data blob format is used to store small binary data. The magic number decides the exact format: | |
| 18 | ||
| 19 | .. list-table:: | |
| 20 | :widths: auto | |
| 21 | ||
| 22 | * - ``[66, 171, 56, 7, 190, 131, 112, 161]`` | |
| 23 | - unencrypted | |
| 24 | - uncompressed | |
| 25 | * - ``[49, 185, 88, 66, 111, 182, 163, 127]`` | |
| 26 | - unencrypted | |
| 27 | - compressed | |
| 28 | * - ``[123, 103, 133, 190, 34, 45, 76, 240]`` | |
| 29 | - encrypted | |
| 30 | - uncompressed | |
| 31 | * - ``[230, 89, 27, 191, 11, 191, 216, 11]`` | |
| 32 | - encrypted | |
| 33 | - compressed | |
| 34 | ||
| 35 | Compression algorithm is ``zstd``. Encryption cipher is ``AES_256_GCM``. | |
| 36 | ||
| 37 | Unencrypted blobs use the following format: | |
| 38 | ||
| 39 | .. list-table:: | |
| 40 | :widths: auto | |
| 41 | ||
| 42 | * - ``MAGIC: [u8; 8]`` | |
| 43 | * - ``CRC32: [u8; 4]`` | |
| 44 | * - ``Data: (max 16MiB)`` | |
| 45 | ||
| 46 | Encrypted blobs additionally contains a 16 byte IV, followed by a 16 | |
| 47 | byte Authenticated Encyryption (AE) tag, followed by the encrypted | |
| 48 | data: | |
| 49 | ||
| 50 | .. list-table:: | |
| 51 | ||
| 52 | * - ``MAGIC: [u8; 8]`` | |
| 53 | * - ``CRC32: [u8; 4]`` | |
| 54 | * - ``ÌV: [u8; 16]`` | |
| 55 | * - ``TAG: [u8; 16]`` | |
| 56 | * - ``Data: (max 16MiB)`` | |
| b488f850 DM |
57 | |
| 58 | ||
| 59 | .. _fixed-index-format: | |
| 60 | ||
| e57aa36d DM |
61 | Fixed Index Format (``.fidx``) |
| 62 | ------------------------------- | |
| b488f850 DM |
63 | |
| 64 | All numbers are stored as little-endian. | |
| 65 | ||
| 66 | .. list-table:: | |
| 67 | ||
| 68 | * - ``MAGIC: [u8; 8]`` | |
| 69 | - ``[47, 127, 65, 237, 145, 253, 15, 205]`` | |
| 70 | * - ``uuid: [u8; 16]``, | |
| 71 | - Unique ID | |
| 72 | * - ``ctime: i64``, | |
| 73 | - Creation Time (epoch) | |
| 74 | * - ``index_csum: [u8; 32]``, | |
| 75 | - Sha256 over the index (without header) ``SHA256(digest1||digest2||...)`` | |
| 76 | * - ``size: u64``, | |
| 77 | - Image size | |
| 78 | * - ``chunk_size: u64``, | |
| 79 | - Chunk size | |
| 80 | * - ``reserved: [u8; 4016]``, | |
| 81 | - overall header size is one page (4096 bytes) | |
| 82 | * - ``digest1: [u8; 32]`` | |
| 83 | - first chunk digest | |
| 84 | * - ``digest2: [u8; 32]`` | |
| 85 | - next chunk | |
| 86 | * - ... | |
| 87 | - next chunk ... | |
| 88 | ||
| 89 | ||
| 90 | .. _dynamic-index-format: | |
| 91 | ||
| e57aa36d DM |
92 | Dynamic Index Format (``.didx``) |
| 93 | -------------------------------- | |
| b488f850 DM |
94 | |
| 95 | All numbers are stored as little-endian. | |
| 96 | ||
| 97 | .. list-table:: | |
| 98 | ||
| 99 | * - ``MAGIC: [u8; 8]`` | |
| 100 | - ``[28, 145, 78, 165, 25, 186, 179, 205]`` | |
| 101 | * - ``uuid: [u8; 16]``, | |
| 102 | - Unique ID | |
| 103 | * - ``ctime: i64``, | |
| 104 | - Creation Time (epoch) | |
| 105 | * - ``index_csum: [u8; 32]``, | |
| 106 | - Sha256 over the index (without header) ``SHA256(offset1||digest1||offset2||digest2||...)`` | |
| 107 | * - ``reserved: [u8; 4032]``, | |
| 108 | - Overall header size is one page (4096 bytes) | |
| 109 | * - ``offset1: u64`` | |
| 110 | - End of first chunk | |
| 111 | * - ``digest1: [u8; 32]`` | |
| 112 | - first chunk digest | |
| 113 | * - ``offset2: u64`` | |
| 114 | - End of second chunk | |
| 115 | * - ``digest2: [u8; 32]`` | |
| 116 | - second chunk digest | |
| 117 | * - ... | |
| 118 | - next chunk offset/digest |