4 Proxmox tape backup provides an easy way to store datastore content
5 onto magnetic tapes. This increases data safety because you get:
7 - an additional copy of the data
8 - to a different media type (tape)
9 - to an additional location (you can move tapes offsite)
11 In most restore jobs, only data from the last backup job is restored.
12 Restore requests further decline the older the data
13 gets. Considering this, tape backup may also help to reduce disk
14 usage, because you can safely remove data from disk once archived on
15 tape. This is especially true if you need to keep data for several
18 Tape backups do not provide random access to the stored data. Instead,
19 you need to restore the data to disk before you can access it
20 again. Also, if you store your tapes offsite (using some kind of tape
21 vaulting service), you need to bring them onsite before you can do any
22 restore. So please consider that restores from tapes can take much
23 longer than restores from disk.
26 Tape Technology Primer
27 ----------------------
29 .. _Linear Tape Open: https://en.wikipedia.org/wiki/Linear_Tape-Open
31 As of 2021, the only broadly available tape technology standard is
32 `Linear Tape Open`_, and different vendors offers LTO Ultrium tape
33 drives, autoloaders and LTO tape cartridges.
35 There are a few vendors offering proprietary drives with
36 slight advantages in performance and capacity, but they have
37 significant disadvantages:
39 - proprietary (single vendor)
40 - a much higher purchase cost
42 So we currently do not test such drives.
44 In general, LTO tapes offer the following advantages:
47 - High Capacity (12 TB)
48 - Relatively low cost per TB
50 - Movable (storable inside vault)
51 - Multiple vendors (for both media and drives)
52 - Build in AES-CGM Encryption engine
54 Please note that `Proxmox Backup Server` already stores compressed
55 data, so we do not need/use the tape compression feature.
61 Proxmox Backup Server supports `Linear Tape Open`_ genertion 4 (LTO4)
62 or later. In general, all SCSI2 tape drives supported by the Linux
63 kernel should work, but feature like hardware encryptions needs LTO4
66 Tape changer support is done using the Linux 'mtx' command line
67 tool. So any changer device supported by that tool should work.
73 Current LTO-8 tapes provide read/write speeds up to 360MB/s. This means,
74 that it still takes a minimum of 9 hours to completely write or
75 read a single tape (even at maximum speed).
77 The only way to speed up that data rate is to use more than one
78 drive. That way you can run several backup jobs in parallel, or run
79 restore jobs while the other dives are used for backups.
81 Also consider that you need to read data first from your datastore
82 (disk). But a single spinning disk is unable to deliver data at this
83 rate. We measured a maximum rate of about 60MB/s to 100MB/s in practice,
84 so it takes 33 hours to read 12TB to fill up an LTO-8 tape. If you want
85 to run your tape at full speed, please make sure that the source
86 datastore is able to deliver that performance (e.g, by using SSDs).
92 :Tape Labels: are used to uniquely indentify a tape. You normally use
93 some sticky paper labels and apply them on the front of the
94 cartridge. We additionally store the label text magnetically on the
95 tape (first file on tape).
97 .. _Code 39: https://en.wikipedia.org/wiki/Code_39
99 .. _LTO Ultrium Cartridge Label Specification: https://www.ibm.com/support/pages/ibm-lto-ultrium-cartridge-label-specification
101 .. _LTO Barcode Generator: lto-barcode/index.html
103 :Barcodes: are a special form of tape labels, which are electronically
104 readable. Most LTO tape robots use an 8 character string encoded as
105 `Code 39`_, as definded in the `LTO Ultrium Cartridge Label
108 You can either buy such barcode labels from your cartridge vendor,
109 or print them yourself. You can use our `LTO Barcode Generator`_ App
112 .. Note:: Physical labels and the associated adhesive shall have an
113 environmental performance to match or exceed the environmental
114 specifications of the cartridge to which it is applied.
116 :Media Pools: A media pool is a logical container for tapes. A backup
117 job targets one media pool, so a job only uses tapes from that
118 pool. The pool additionally defines how long a backup job can
119 append data to tapes (allocation policy) and how long you want to
120 keep the data (retention policy).
122 :Media Set: A group of continuously written tapes (all from the same
125 :Tape drive: The decive used to read and write data to the tape. There
126 are standalone drives, but drives often ship within tape libraries.
128 :Tape changer: A device which can change the tapes inside a tape drive
129 (tape robot). They are usually part of a tape library.
131 .. _Tape Library: https://en.wikipedia.org/wiki/Tape_library
133 :`Tape library`_: A storage device that contains one or more tape drives,
134 a number of slots to hold tape cartridges, a barcode reader to
135 identify tape cartridges and an automated method for loading tapes
138 People als call this 'autoloader', 'tape robot' or 'tape jukebox'.
140 :Inventory: The inventory stores the list of known tapes (with
141 additional status information).
143 :Catalog: A media catalog stores information about the media content.
149 1. Configure your tape hardware (drives and changers)
151 2. Configure one or more media pools
153 3. Label your tape cartridges.
155 4. Start your first tape backup job ...
161 Please note that you can configure anything using the graphical user
162 interface or the command line interface. Both methods results in the
169 Tape changers (robots) are part of a `Tape Library`_. You can skip
170 this step if you are using a standalone drive.
172 Linux is able to auto detect those devices, and you can get a list
173 of available devices using::
175 # proxmox-tape changer scan
176 ┌─────────────────────────────┬─────────┬──────────────┬────────┐
177 │ path │ vendor │ model │ serial │
178 ╞═════════════════════════════╪═════════╪══════════════╪════════╡
179 │ /dev/tape/by-id/scsi-CC2C52 │ Quantum │ Superloader3 │ CC2C52 │
180 └─────────────────────────────┴─────────┴──────────────┴────────┘
182 In order to use that device with Proxmox, you need to create a
183 configuration entry::
185 # proxmox-tape changer create sl3 --path /dev/tape/by-id/scsi-CC2C52
187 Where ``sl3`` is an arbitrary name you can choose.
189 .. Note:: Please use stable device path names from inside
190 ``/dev/tape/by-id/``. Names like ``/dev/sg0`` may point to a
191 different device after reboot, and that is not what you want.
193 You can show the final configuration with::
195 # proxmox-tape changer config sl3
196 ┌──────┬─────────────────────────────┐
198 ╞══════╪═════════════════════════════╡
200 ├──────┼─────────────────────────────┤
201 │ path │ /dev/tape/by-id/scsi-CC2C52 │
202 └──────┴─────────────────────────────┘
204 Or simply list all configured changer devices::
206 # proxmox-tape changer list
207 ┌──────┬─────────────────────────────┬─────────┬──────────────┬────────────┐
208 │ name │ path │ vendor │ model │ serial │
209 ╞══════╪═════════════════════════════╪═════════╪══════════════╪════════════╡
210 │ sl3 │ /dev/tape/by-id/scsi-CC2C52 │ Quantum │ Superloader3 │ CC2C52 │
211 └──────┴─────────────────────────────┴─────────┴──────────────┴────────────┘
213 The Vendor, Model and Serial number are auto detected, but only shown
214 if the device is online.
216 To test your setup, please query the status of the changer device with::
218 # proxmox-tape changer status sl3
219 ┌───────────────┬──────────┬────────────┬─────────────┐
220 │ entry-kind │ entry-id │ changer-id │ loaded-slot │
221 ╞═══════════════╪══════════╪════════════╪═════════════╡
222 │ drive │ 0 │ vtape1 │ 1 │
223 ├───────────────┼──────────┼────────────┼─────────────┤
225 ├───────────────┼──────────┼────────────┼─────────────┤
226 │ slot │ 2 │ vtape2 │ │
227 ├───────────────┼──────────┼────────────┼─────────────┤
229 ├───────────────┼──────────┼────────────┼─────────────┤
231 └───────────────┴──────────┴────────────┴─────────────┘
233 Tape libraries usually provide some special import/export slots (also
234 called "mail slots"). Tapes inside those slots are acessible from
235 outside, making it easy to add/remove tapes to/from the library. Those
236 tapes are considered to be "offline", so backup jobs will not use
237 them. Those special slots are auto-detected and marked as
238 ``import-export`` slot in the status command.
240 It's worth noting that some of the smaller tape libraries don't have
241 such slots. While they have something called "Mail Slot", that slot
242 is just a way to grab the tape from the gripper. But they are unable
243 to hold media while the robot does other things. They also do not
244 expose that "Mail Slot" over the SCSI interface, so you wont see them in
247 As a workaround, you can mark some of the normal slots as export
248 slot. The software treats those slots like real ``import-export``
249 slots, and the media inside those slots is considered to be 'offline'
250 (not available for backup)::
252 # proxmox-tape changer update sl3 --export-slots 15,16
254 After that, you can see those artificial ``import-export`` slots in
257 # proxmox-tape changer status sl3
258 ┌───────────────┬──────────┬────────────┬─────────────┐
259 │ entry-kind │ entry-id │ changer-id │ loaded-slot │
260 ╞═══════════════╪══════════╪════════════╪═════════════╡
261 │ drive │ 0 │ vtape1 │ 1 │
262 ├───────────────┼──────────┼────────────┼─────────────┤
263 │ import-export │ 15 │ │ │
264 ├───────────────┼──────────┼────────────┼─────────────┤
265 │ import-export │ 16 │ │ │
266 ├───────────────┼──────────┼────────────┼─────────────┤
268 ├───────────────┼──────────┼────────────┼─────────────┤
269 │ slot │ 2 │ vtape2 │ │
270 ├───────────────┼──────────┼────────────┼─────────────┤
272 ├───────────────┼──────────┼────────────┼─────────────┤
274 └───────────────┴──────────┴────────────┴─────────────┘
280 Linux is able to auto detect tape drives, and you can get a list
281 of available tape drives using::
283 # proxmox-tape drive scan
284 ┌────────────────────────────────┬────────┬─────────────┬────────┐
285 │ path │ vendor │ model │ serial │
286 ╞════════════════════════════════╪════════╪═════════════╪════════╡
287 │ /dev/tape/by-id/scsi-12345-nst │ IBM │ ULT3580-TD4 │ 12345 │
288 └────────────────────────────────┴────────┴─────────────┴────────┘
290 In order to use that drive with Proxmox, you need to create a
291 configuration entry::
293 # proxmox-tape drive create mydrive --path /dev/tape/by-id/scsi-12345-nst
295 .. Note:: Please use stable device path names from inside
296 ``/dev/tape/by-id/``. Names like ``/dev/nst0`` may point to a
297 different device after reboot, and that is not what you want.
299 If you have a tape library, you also need to set the associated
302 # proxmox-tape drive update mydrive --changer sl3 --changer-drivenum 0
304 The ``--changer-drivenum`` is only necessary if the tape library
305 includes more than one drive (The changer status command lists all
308 You can show the final configuration with::
310 # proxmox-tape drive config mydrive
311 ┌─────────┬────────────────────────────────┐
313 ╞═════════╪════════════════════════════════╡
315 ├─────────┼────────────────────────────────┤
316 │ path │ /dev/tape/by-id/scsi-12345-nst │
317 ├─────────┼────────────────────────────────┤
319 └─────────┴────────────────────────────────┘
321 .. NOTE:: The ``changer-drivenum`` value 0 is not stored in the
322 configuration, because that is the default.
324 To list all configured drives use::
326 # proxmox-tape drive list
327 ┌──────────┬────────────────────────────────┬─────────┬────────┬─────────────┬────────┐
328 │ name │ path │ changer │ vendor │ model │ serial │
329 ╞══════════╪════════════════════════════════╪═════════╪════════╪═════════════╪════════╡
330 │ mydrive │ /dev/tape/by-id/scsi-12345-nst │ sl3 │ IBM │ ULT3580-TD4 │ 12345 │
331 └──────────┴────────────────────────────────┴─────────┴────────┴─────────────┴────────┘
333 The Vendor, Model and Serial number are auto detected, but only shown
334 if the device is online.
336 For testing, you can simply query the drive status with::
338 # proxmox-tape status --drive mydrive
339 ┌───────────┬────────────────────────┐
341 ╞═══════════╪════════════════════════╡
343 ├───────────┼────────────────────────┤
344 │ status │ DRIVE_OPEN | IM_REP_EN │
345 └───────────┴────────────────────────┘
347 .. NOTE:: Blocksize should always be 0 (variable block size
348 mode). This is the default anyways.
354 A media pool is a logical container for tapes. A backup job targets
355 one media pool, so a job only uses tapes from that pool.
359 A media set is a group of continuously written tapes, used to split
360 the larger pool into smaller, restorable units. One or more backup
361 jobs write to a media set, producing an ordered group of
362 tapes. Media sets are identified by an unique ID. That ID and the
363 sequence number is stored on each tape of that set (tape label).
365 Media sets are the basic unit for restore tasks, i.e. you need all
366 tapes in the set to restore the media set content. Data is fully
367 deduplicated inside a media set.
370 .. topic:: Media Set Allocation Policy
372 The pool additionally defines how long backup jobs can append data
373 to a media set. The following settings are possible:
375 - Try to use the current media set.
377 This setting produce one large media set. While this is very
378 space efficient (deduplication, no unused space), it can lead to
379 long restore times, because restore jobs needs to read all tapes in the
382 .. NOTE:: Data is fully deduplicated inside a media set. That
383 also means that data is randomly distributed over the tapes in
384 the set. So even if you restore a single VM, this may have to
385 read data from all tapes inside the media set.
387 Larger media sets are also more error prone, because a single
388 damaged media makes the restore fail.
390 Usage scenario: Mostly used with tape libraries, and you manually
391 trigger new set creation by running a backup job with the
394 .. NOTE:: Retention period starts with the existence of a newer
397 - Always create a new media set.
399 With this setting each backup job creates a new media set. This
400 is less space efficient, because the last media from the last set
401 may not be fully written, leaving the remaining space unused.
403 The advantage is that this procudes media sets of minimal
404 size. Small set are easier to handle, you can move sets to an
405 off-site vault, and restore is much faster.
407 .. NOTE:: Retention period starts with the creation time of the
410 - Create a new set when the specified Calendar Event triggers.
412 .. _systemd.time manpage: https://manpages.debian.org/buster/systemd/systemd.time.7.en.html
414 This allows you to specify points in time by using systemd like
415 Calendar Event specifications (see `systemd.time manpage`_).
417 For example, the value ``weekly`` (or ``Mon *-*-* 00:00:00``)
418 will create a new set each week.
420 This balances between space efficency and media count.
422 .. NOTE:: Retention period starts when the calendar event
425 Additionally, the following events may allocate a new media set:
427 - Required tape is offline (and you use a tape library).
429 - Current set contains damaged of retired tapes.
431 - Media pool encryption changed
433 - Database consistency errors, e.g. if the inventory does not
434 contain required media info, or contain conflicting infos
437 .. topic:: Retention Policy
439 Defines how long we want to keep the data.
441 - Always overwrite media.
443 - Protect data for the duration specified.
445 We use systemd like time spans to specify durations, e.g. ``2
446 weeks`` (see `systemd.time manpage`_).
448 - Never overwrite data.
450 .. topic:: Hardware Encryption
452 LTO4 (or later) tape drives support hardware encryption. If you
453 configure the media pool to use encryption, all data written to the
454 tapes is encrypted using the configured key.
456 That way, unauthorized users cannot read data from the media,
457 e.g. if you loose a media while shipping to an offsite location.
459 .. Note:: If the backup client also encrypts data, data on tape
460 will be double encrypted.
462 The password protected key is stored on each media, so it is
463 possbible to `restore the key <restore_encryption_key_>`_ using the password. Please make sure
464 you remember the password in case you need to restore the key.
467 .. NOTE:: FIXME: Add note about global content namespace. (We do not store
468 the source datastore, so it is impossible to distinguish
469 store1:/vm/100 from store2:/vm/100. Please use different media
470 pools if the source is from a different name space)
473 The following command creates a new media pool::
475 // proxmox-tape pool create <name> --drive <string> [OPTIONS]
477 # proxmox-tape pool create daily --drive mydrive
480 Additional option can be set later using the update command::
482 # proxmox-tape pool update daily --allocation daily --retention 7days
485 To list all configured pools use::
487 # proxmox-tape pool list
488 ┌───────┬──────────┬────────────┬───────────┬──────────┐
489 │ name │ drive │ allocation │ retention │ template │
490 ╞═══════╪══════════╪════════════╪═══════════╪══════════╡
491 │ daily │ mydrive │ daily │ 7days │ │
492 └───────┴──────────┴────────────┴───────────┴──────────┘
502 Many sub-command of the ``proxmox-tape`` command line tools take a
503 parameter called ``--drive``, which specifies the tape drive you want
504 to work on. For convenience, you can set that in an environment
507 # export PROXMOX_TAPE_DRIVE=mydrive
509 You can then omit the ``--drive`` parameter from the command. If the
510 drive has an associated changer device, you may also omit the changer
511 parameter from commands that needs a changer device, for example::
513 # proxmox-tape changer status
515 Should displays the changer status of the changer device associated with
522 By default, tape cartidges all looks the same, so you need to put a
523 label on them for unique identification. So first, put a sticky paper
524 label with some human readable text on the cartridge.
526 If you use a `Tape Library`_, you should use an 8 character string
527 encoded as `Code 39`_, as definded in the `LTO Ultrium Cartridge Label
528 Specification`_. You can either bye such barcode labels from your
529 cartidge vendor, or print them yourself. You can use our `LTO Barcode
530 Generator`_ App for that.
532 Next, you need to write that same label text to the tape, so that the
533 software can uniquely identify the tape too.
535 For a standalone drive, manually insert the new tape cartidge into the
538 # proxmox-tape label --changer-id <label-text> [--pool <pool-name>]
540 You may omit the ``--pool`` argument to allow the tape to be used by any pool.
542 .. Note:: For safety reasons, this command fails if the tape contain
543 any data. If you want to overwrite it anways, erase the tape first.
545 You can verify success by reading back the label::
547 # proxmox-tape read-label
548 ┌─────────────────┬──────────────────────────────────────┐
550 ╞═════════════════╪══════════════════════════════════════╡
551 │ changer-id │ vtape1 │
552 ├─────────────────┼──────────────────────────────────────┤
553 │ uuid │ 7f42c4dd-9626-4d89-9f2b-c7bc6da7d533 │
554 ├─────────────────┼──────────────────────────────────────┤
555 │ ctime │ Wed Jan 6 09:07:51 2021 │
556 ├─────────────────┼──────────────────────────────────────┤
558 ├─────────────────┼──────────────────────────────────────┤
559 │ media-set-uuid │ 00000000-0000-0000-0000-000000000000 │
560 ├─────────────────┼──────────────────────────────────────┤
561 │ media-set-ctime │ Wed Jan 6 09:07:51 2021 │
562 └─────────────────┴──────────────────────────────────────┘
564 .. NOTE:: The ``media-set-uuid`` using all zeros indicates an empty
565 tape (not used by any media set).
567 If you have a tape library, apply the sticky barcode label to the tape
568 cartridges first. Then load those empty tapes into the library. You
569 can then label all unlabeled tapes with a single command::
571 # proxmox-tape barcode-label [--pool <pool-name>]
577 To manually run a backup job use::
579 # proxmox-tape backup <store> <pool> [OPTIONS]
581 The following options are available:
583 --eject-media Eject media upon job completion.
585 It is normally good practice to eject the tape after use. This unmounts the
586 tape from the drive and prevents the tape from getting dirty with dust.
588 --export-media-set Export media set upon job completion.
590 After a sucessful backup job, this moves all tapes from the used
591 media set into import-export slots. The operator can then pick up
592 those tapes and move them to a media vault.
598 Restore is done at media-set granularity, so you first need to find
599 out which media set contains the data you want to restore. This
600 information is stored in the media catalog. If you do not have media
601 catalogs, you need to restore them first. Please note that you need
602 the catalog to find your data, but restoring a complete media-set does
603 not need media catalogs.
605 The following command shows the media content (from catalog)::
607 # proxmox-tape media content
608 ┌────────────┬──────┬──────────────────────────┬────────┬────────────────────────────────┬──────────────────────────────────────┐
609 │ label-text │ pool │ media-set-name │ seq-nr │ snapshot │ media-set-uuid │
610 ╞════════════╪══════╪══════════════════════════╪════════╪════════════════════════════════╪══════════════════════════════════════╡
611 │ TEST01L8 │ p2 │ Wed Jan 13 13:55:55 2021 │ 0 │ vm/201/2021-01-11T10:43:48Z │ 9da37a55-aac7-4deb-91c6-482b3b675f30 │
612 ├────────────┼──────┼──────────────────────────┼────────┼────────────────────────────────┼──────────────────────────────────────┤
613 │ ... │ ... │ ... │ ... │ ... │ ... │
614 └────────────┴──────┴──────────────────────────┴────────┴────────────────────────────────┴──────────────────────────────────────┘
617 A restore job reads the data from the media set and moves it back to
618 data disk (datastore)::
620 // proxmox-tape restore <media-set-uuid> <datastore>
622 # proxmox-tape restore 9da37a55-aac7-4deb-91c6-482b3b675f30 mystore
633 Encryption Key Management
634 ~~~~~~~~~~~~~~~~~~~~~~~~~
636 Creating a new encryption key::
638 # proxmox-tape key create --hint "tape pw 2020"
639 Tape Encryption Key Password: **********
640 Verify Password: **********
641 "14:f8:79:b9:f5:13:e5:dc:bf:b6:f9:88:48:51:81:dc:79:bf:a0:22:68:47:d1:73:35:2d:b6:20:e1:7f:f5:0f"
643 List existing encryption keys::
645 # proxmox-tape key list
646 ┌───────────────────────────────────────────────────┬───────────────┐
647 │ fingerprint │ hint │
648 ╞═══════════════════════════════════════════════════╪═══════════════╡
649 │ 14:f8:79:b9:f5:13:e5:dc: ... :b6:20:e1:7f:f5:0f │ tape pw 2020 │
650 └───────────────────────────────────────────────────┴───────────────┘
652 To show encryption key details::
654 # proxmox-tape key show 14:f8:79:b9:f5:13:e5:dc:...:b6:20:e1:7f:f5:0f
655 ┌─────────────┬───────────────────────────────────────────────┐
657 ╞═════════════╪═══════════════════════════════════════════════╡
659 ├─────────────┼───────────────────────────────────────────────┤
660 │ created │ Sat Jan 23 14:47:21 2021 │
661 ├─────────────┼───────────────────────────────────────────────┤
662 │ modified │ Sat Jan 23 14:47:21 2021 │
663 ├─────────────┼───────────────────────────────────────────────┤
664 │ fingerprint │ 14:f8:79:b9:f5:13:e5:dc:...:b6:20:e1:7f:f5:0f │
665 ├─────────────┼───────────────────────────────────────────────┤
666 │ hint │ tape pw 2020 │
667 └─────────────┴───────────────────────────────────────────────┘
669 The ``paperkey`` subcommand can be used to create a QR encoded
670 version of a tape encryption key. The following command sends the output of the
671 ``paperkey`` command to a text file, for easy printing::
673 proxmox-tape key paperkey <fingerprint> --output-format text > qrkey.txt
676 .. _restore_encryption_key:
678 Restoring Encryption Keys
679 ^^^^^^^^^^^^^^^^^^^^^^^^^
681 You can restore the encryption key from the tape, using the password
682 used to generate the key. First, load the tape you want to restore
683 into the drive. Then run::
685 # proxmox-tape key restore
686 Tepe Encryption Key Password: ***********
688 If the password is correct, the key will get imported to the
689 database. Further restore jobs automatically use any availbale key.
695 LTO tape drives requires regular cleaning. This is done by loading a
696 cleaning cartridge into the drive, which is a manual task for
699 For tape libraries, cleaning cartridges are identified using special
700 labels starting with letters "CLN". For example, our tape library has a
701 cleaning cartridge inside slot 3::
703 # proxmox-tape changer status sl3
704 ┌───────────────┬──────────┬────────────┬─────────────┐
705 │ entry-kind │ entry-id │ changer-id │ loaded-slot │
706 ╞═══════════════╪══════════╪════════════╪═════════════╡
707 │ drive │ 0 │ vtape1 │ 1 │
708 ├───────────────┼──────────┼────────────┼─────────────┤
710 ├───────────────┼──────────┼────────────┼─────────────┤
711 │ slot │ 2 │ vtape2 │ │
712 ├───────────────┼──────────┼────────────┼─────────────┤
713 │ slot │ 3 │ CLN001CU │ │
714 ├───────────────┼──────────┼────────────┼─────────────┤
716 └───────────────┴──────────┴────────────┴─────────────┘
718 To initiate a cleaning operation simply run::
722 This command does the following:
724 - find the cleaning tape (in slot 3)
726 - unload the current media from the drive (back to slot1)
728 - load the cleaning tape into the drive
730 - run drive cleaning operation
732 - unload the cleaning tape (to slot 3)
745 .. include:: config/media-pool/format.rst
751 .. include:: config/media-pool/config.rst
760 .. include:: config/tape/format.rst
766 .. include:: config/tape/config.rst
775 .. include:: proxmox-tape/synopsis.rst
781 .. include:: pmt/options.rst
785 .. include:: pmt/synopsis.rst
791 .. include:: pmtx/synopsis.rst