4 .\" The contents of this file are subject to the terms of the
5 .\" Common Development and Distribution License (the "License").
6 .\" You may not use this file except in compliance with the License.
8 .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9 .\" or https://opensource.org/licenses/CDDL-1.0.
10 .\" See the License for the specific language governing permissions
11 .\" and limitations under the License.
13 .\" When distributing Covered Code, include this CDDL HEADER in each
14 .\" file and include the License file at usr/src/OPENSOLARIS.LICENSE.
15 .\" If applicable, add the following below this CDDL HEADER, with the
16 .\" fields enclosed by brackets "[]" replaced with your own identifying
17 .\" information: Portions Copyright [yyyy] [name of copyright owner]
21 .\" Copyright (c) 2009 Sun Microsystems, Inc. All Rights Reserved.
22 .\" Copyright 2011 Joshua M. Clulow <josh@sysmgr.org>
23 .\" Copyright (c) 2011, 2019 by Delphix. All rights reserved.
24 .\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
25 .\" Copyright (c) 2014, Joyent, Inc. All rights reserved.
26 .\" Copyright (c) 2014 by Adam Stevko. All rights reserved.
27 .\" Copyright (c) 2014 Integros [integros.com]
28 .\" Copyright 2019 Richard Laager. All rights reserved.
29 .\" Copyright 2018 Nexenta Systems, Inc.
30 .\" Copyright 2019 Joyent, Inc.
38 .Nd generate backup stream of ZFS dataset
43 .Op Fl R Op Fl X Ar dataset Ns Oo , Ns Ar dataset Oc Ns …
44 .Op Oo Fl I Ns | Ns Fl i Oc Ar snapshot
49 .Op Fl i Ar snapshot Ns | Ns Ar bookmark
50 .Ar filesystem Ns | Ns Ar volume Ns | Ns Ar snapshot
53 .Fl -redact Ar redaction_bookmark
55 .Op Fl i Ar snapshot Ns | Ns Ar bookmark
61 .Ar receive_resume_token
68 .Ar snapshot redaction_bookmark
69 .Ar redaction_snapshot Ns …
77 .Op Fl R Op Fl X Ar dataset Ns Oo , Ns Ar dataset Oc Ns …
78 .Op Oo Fl I Ns | Ns Fl i Oc Ar snapshot
81 Creates a stream representation of the second
83 which is written to standard output.
84 The output can be redirected to a file or to a different system
85 .Po for example, using
88 By default, a full stream is generated.
91 Deduplicated send is no longer supported.
92 This flag is accepted for backwards compatibility, but a regular,
93 non-deduplicated stream will be generated.
95 Generate a stream package that sends all intermediary snapshots from the first
96 snapshot to the second snapshot.
100 .Fl i Em @a Em fs@b Ns \&; Fl i Em @b Em fs@c Ns \&; Fl i Em @c Em fs@d .
101 The incremental source may be specified as with the
104 .It Fl L , -large-block
105 Generate a stream which may contain blocks larger than 128 KiB.
106 This flag has no effect if the
108 pool feature is disabled, or if the
110 property of this filesystem has never been set above 128 KiB.
111 The receiving system must have the
113 pool feature enabled as well.
116 for details on ZFS feature flags and the
120 Print machine-parsable verbose information about the stream package generated.
121 .It Fl R , -replicate
122 Generate a replication stream package, which will replicate the specified
123 file system, and all descendent file systems, up to the named snapshot.
124 When received, all properties, snapshots, descendent file systems, and clones
131 flags are used in conjunction with the
133 flag, an incremental replication stream is generated.
134 The current values of properties, and current snapshot and file system names are
135 set when the stream is received.
138 flag is specified when this stream is received, snapshots and file systems that
139 do not exist on the sending side are destroyed.
142 flag is used to send encrypted datasets, then
144 must also be specified.
145 .It Fl V , -proctitle
146 Set the process title to a per-second report of how much data has been sent.
147 .It Fl X , -exclude Ar dataset Ns Oo , Ns Ar dataset Oc Ns …
151 specifies a set of datasets (and, hence, their descendants),
152 to be excluded from the send stream.
153 The root dataset may not be excluded.
156 .Fl X Ar a , Ns Ar b .
158 Generate a more compact stream by using
160 records for blocks which are stored more compactly on disk by the
163 This flag has no effect if the
166 The receiving system must have the
171 feature is active on the sending system, then the receiving system must have
172 that feature enabled as well.
173 Datasets that are sent with this flag may not be
174 received as an encrypted dataset, since encrypted datasets cannot use the
179 for details on ZFS feature flags and the
183 Sends only received property values whether or not they are overridden by local
184 settings, but only if the dataset has ever been received.
185 Use this option when you want
187 to restore received properties backed up on the sent dataset and to avoid
188 sending local settings that may have nothing to do with the source dataset,
189 but only with how the data is backed up.
190 .It Fl c , -compressed
191 Generate a more compact stream by using compressed WRITE records for blocks
192 which are compressed on disk and in memory
199 feature is active on the sending system, then the receiving system must have
200 that feature enabled as well.
203 feature is enabled on the sending system but the
205 option is not supplied in conjunction with
207 then the data will be decompressed before sending so it can be split into
211 will not have their data recompressed on the receiver side using
212 .Fl o Sy compress Ns = Ar value .
213 The data will stay compressed as it was from the sender.
214 The new compression property will be set for future data.
215 Note that uncompressed data from the sender will still attempt to
216 compress on the receiver, unless you specify
217 .Fl o Sy compress Ns = Em off .
219 For encrypted datasets, send data exactly as it exists on disk.
220 This allows backups to be taken even if encryption keys are not currently
222 The backup may then be received on an untrusted machine since that machine will
223 not have the encryption keys to read the protected data or alter it without
225 Upon being received, the dataset will have the same encryption
226 keys as it did on the send side, although the
228 property will be defaulted to
230 if not otherwise provided.
231 For unencrypted datasets, this flag will be equivalent to
233 Note that if you do not use this flag for sending encrypted datasets, data will
234 be sent unencrypted and may be re-encrypted with a different encryption key on
235 the receiving system, which will disable the ability to do a raw send to that
236 system for incrementals.
238 Generate a stream package that includes any snapshot holds (created with the
240 command), and indicating to
242 that the holds be applied to the dataset on the receiving system.
244 Generate an incremental stream from the first
246 .Pq the incremental source
249 .Pq the incremental target .
250 The incremental source can be specified as the last component of the snapshot
254 character and following
256 and it is assumed to be from the same file system as the incremental target.
258 If the destination is a clone, the source may be the origin snapshot, which must
269 Do not generate any actual send data.
270 This is useful in conjunction with the
274 flags to determine what data will be sent.
275 In this case, the verbose output will be written to standard output
276 .Po contrast with a non-dry-run, where the stream is written to standard output
277 and the verbose output goes to standard error
280 Include the dataset's properties in the stream.
281 This flag is implicit when
284 The receiving system must also support this feature.
285 Sends of encrypted datasets must use
287 when using this flag.
288 .It Fl s , -skip-missing
289 Allows sending a replication stream even when there are snapshots missing in the
291 When a snapshot is missing, instead of throwing an error and aborting the send,
292 a warning is printed to the standard error stream and the dataset to which it
294 and its descendents are skipped.
295 This flag can only be used in conjunction with
298 Print verbose information about the stream package generated.
299 This information includes a per-second report of how much data has been sent.
300 The same report can be requested by sending
307 The format of the stream is committed.
308 You will be able to receive your streams on future versions of ZFS.
314 .Op Fl i Ar snapshot Ns | Ns Ar bookmark
315 .Ar filesystem Ns | Ns Ar volume Ns | Ns Ar snapshot
317 Generate a send stream, which may be of a filesystem, and may be incremental
319 If the destination is a filesystem or volume, the pool must be read-only, or the
320 filesystem must not be mounted.
321 When the stream generated from a filesystem or volume is received, the default
322 snapshot name will be
326 Deduplicated send is no longer supported.
327 This flag is accepted for backwards compatibility, but a regular,
328 non-deduplicated stream will be generated.
329 .It Fl L , -large-block
330 Generate a stream which may contain blocks larger than 128 KiB.
331 This flag has no effect if the
333 pool feature is disabled, or if the
335 property of this filesystem has never been set above 128 KiB.
336 The receiving system must have the
338 pool feature enabled as well.
341 for details on ZFS feature flags and the
345 Print machine-parsable verbose information about the stream package generated.
346 .It Fl c , -compressed
347 Generate a more compact stream by using compressed WRITE records for blocks
348 which are compressed on disk and in memory
355 feature is active on the sending system, then the receiving system must have
356 that feature enabled as well.
359 feature is enabled on the sending system but the
361 option is not supplied in conjunction with
363 then the data will be decompressed before sending so it can be split into
366 For encrypted datasets, send data exactly as it exists on disk.
367 This allows backups to be taken even if encryption keys are not currently
369 The backup may then be received on an untrusted machine since that machine will
370 not have the encryption keys to read the protected data or alter it without
372 Upon being received, the dataset will have the same encryption
373 keys as it did on the send side, although the
375 property will be defaulted to
377 if not otherwise provided.
378 For unencrypted datasets, this flag will be equivalent to
380 Note that if you do not use this flag for sending encrypted datasets, data will
381 be sent unencrypted and may be re-encrypted with a different encryption key on
382 the receiving system, which will disable the ability to do a raw send to that
383 system for incrementals.
385 Generate a more compact stream by using
387 records for blocks which are stored more compactly on disk by the
390 This flag has no effect if the
393 The receiving system must have the
398 feature is active on the sending system, then the receiving system must have
399 that feature enabled as well.
400 Datasets that are sent with this flag may not be received as an encrypted
402 since encrypted datasets cannot use the
407 for details on ZFS feature flags and the
410 .It Fl i Ar snapshot Ns | Ns Ar bookmark
411 Generate an incremental send stream.
412 The incremental source must be an earlier snapshot in the destination's history.
413 It will commonly be an earlier snapshot in the destination's file system, in
414 which case it can be specified as the last component of the name
419 character and following
422 If the incremental target is a clone, the incremental source can be the origin
423 snapshot, or an earlier snapshot in the origin's filesystem, or the origin's
429 Do not generate any actual send data.
430 This is useful in conjunction with the
434 flags to determine what data will be sent.
435 In this case, the verbose output will be written to standard output
436 .Po contrast with a non-dry-run, where the stream is written to standard output
437 and the verbose output goes to standard error
440 Print verbose information about the stream package generated.
441 This information includes a per-second report of how much data has been sent.
442 The same report can be requested by sending
452 .Fl -redact Ar redaction_bookmark
454 .Op Fl i Ar snapshot Ns | Ns Ar bookmark
457 Generate a redacted send stream.
458 This send stream contains all blocks from the snapshot being sent that aren't
459 included in the redaction list contained in the bookmark specified by the
464 The resulting send stream is said to be redacted with respect to the snapshots
465 the bookmark specified by the
466 .Fl -redact No flag was created with .
467 The bookmark must have been created by running
469 on the snapshot being sent.
471 This feature can be used to allow clones of a filesystem to be made available on
472 a remote system, in the case where their parent need not (or needs to not) be
474 For example, if a filesystem contains sensitive data, and it has clones where
475 that sensitive data has been secured or replaced with dummy data, redacted sends
476 can be used to replicate the secured data without replicating the original
477 sensitive data, while still sharing all possible blocks.
478 A snapshot that has been redacted with respect to a set of snapshots will
479 contain all blocks referenced by at least one snapshot in the set, but will
480 contain none of the blocks referenced by none of the snapshots in the set.
481 In other words, if all snapshots in the set have modified a given block in the
482 parent, that block will not be sent; but if one or more snapshots have not
483 modified a block in the parent, they will still reference the parent's block, so
484 that block will be sent.
485 Note that only user data will be redacted.
487 When the redacted send stream is received, we will generate a redacted
489 Due to the nature of redaction, a redacted dataset can only be used in the
491 .Bl -enum -width "a."
493 To receive, as a clone, an incremental send from the original snapshot to one
494 of the snapshots it was redacted with respect to.
495 In this case, the stream will produce a valid dataset when received because all
496 blocks that were redacted in the parent are guaranteed to be present in the
498 This use case will produce a normal snapshot, which can be used just like other
502 To receive an incremental send from the original snapshot to something
503 redacted with respect to a subset of the set of snapshots the initial snapshot
504 was redacted with respect to.
505 In this case, each block that was redacted in the original is still redacted
506 (redacting with respect to additional snapshots causes less data to be redacted
507 (because the snapshots define what is permitted, and everything else is
509 This use case will produce a new redacted snapshot.
511 To receive an incremental send from a redaction bookmark of the original
512 snapshot that was created when redacting with respect to a subset of the set of
513 snapshots the initial snapshot was created with respect to
515 A send stream from such a redaction bookmark will contain all of the blocks
516 necessary to fill in any redacted data, should it be needed, because the sending
517 system is aware of what blocks were originally redacted.
518 This will either produce a normal snapshot or a redacted one, depending on
519 whether the new send stream is redacted.
521 To receive an incremental send from a redacted version of the initial
522 snapshot that is redacted with respect to a subject of the set of snapshots the
523 initial snapshot was created with respect to.
524 A send stream from a compatible redacted dataset will contain all of the blocks
525 necessary to fill in any redacted data.
526 This will either produce a normal snapshot or a redacted one, depending on
527 whether the new send stream is redacted.
529 To receive a full send as a clone of the redacted snapshot.
530 Since the stream is a full send, it definitionally contains all the data needed
531 to create a new dataset.
532 This use case will either produce a normal snapshot or a redacted one, depending
533 on whether the full send stream was redacted.
536 These restrictions are detected and enforced by
538 a redacted send stream will contain the list of snapshots that the stream is
539 redacted with respect to.
540 These are stored with the redacted snapshot, and are used to detect and
541 correctly handle the cases above.
542 Note that for technical reasons,
543 raw sends and redacted sends cannot be combined at this time.
549 .Ar receive_resume_token
551 Creates a send stream which resumes an interrupted receive.
553 .Ar receive_resume_token
554 is the value of this property on the filesystem or volume that was being
556 See the documentation for
557 .Nm zfs Cm receive Fl s
563 .Op Fl i Ar snapshot Ns | Ns Ar bookmark
567 Generate a send stream from a dataset that has been partially received.
570 This flag requires that the specified filesystem previously received a resumable
571 send that did not finish and was interrupted.
572 In such scenarios this flag
573 enables the user to send this partially received state.
574 Using this flag will always use the last fully received snapshot
575 as the incremental source if it exists.
580 .Ar snapshot redaction_bookmark
581 .Ar redaction_snapshot Ns …
583 Generate a new redaction bookmark.
584 In addition to the typical bookmark information, a redaction bookmark contains
585 the list of redacted blocks and the list of redaction snapshots specified.
586 The redacted blocks are blocks in the snapshot which are not referenced by any
587 of the redaction snapshots.
588 These blocks are found by iterating over the metadata in each redaction snapshot
589 to determine what has been changed since the target snapshot.
590 Redaction is designed to support redacted zfs sends; see the entry for
592 for more information on the purpose of this operation.
593 If a redact operation fails partway through (due to an error or a system
594 failure), the redaction can be resumed by rerunning the same command.
597 ZFS has support for a limited version of data subsetting, in the form of
602 .Sy redaction bookmark
603 can be created that stores a list of blocks containing sensitive information.
609 Redacted sends omit the blocks containing sensitive information,
610 replacing them with REDACT records.
611 When these send streams are received, a
614 A redacted dataset cannot be mounted by default, since it is incomplete.
615 It can be used to receive other send streams.
616 In this way datasets can be used for data backup and replication,
617 with all the benefits that zfs send and receive have to offer,
618 while protecting sensitive information from being
619 stored on less-trusted machines or services.
621 For the purposes of redaction, there are two steps to the process.
622 A redact step, and a send/receive step.
623 First, a redaction bookmark is created.
624 This is done by providing the
626 command with a parent snapshot, a bookmark to be created, and a number of
628 These redaction snapshots must be descendants of the parent snapshot,
629 and they should modify data that is considered sensitive in some way.
630 Any blocks of data modified by all of the redaction snapshots will
631 be listed in the redaction bookmark, because it represents the truly sensitive
633 When it comes to the send step, the send process will not send
634 the blocks listed in the redaction bookmark, instead replacing them with
636 When received on the target system, this will create a
637 redacted dataset, missing the data that corresponds to the blocks in the
638 redaction bookmark on the sending system.
639 The incremental send streams from
640 the original parent to the redaction snapshots can then also be received on
641 the target system, and this will produce a complete snapshot that can be used
643 Incrementals from one snapshot on the parent filesystem and another
644 can also be done by sending from the redaction bookmark, rather than the
645 snapshots themselves.
647 In order to make the purpose of the feature more clear, an example is provided.
648 Consider a zfs filesystem containing four files.
649 These files represent information for an online shopping service.
650 One file contains a list of usernames and passwords, another contains purchase
652 a third contains click tracking data, and a fourth contains user preferences.
653 The owner of this data wants to make it available for their development teams to
654 test against, and their market research teams to do analysis on.
655 The development teams need information about user preferences and the click
656 tracking data, while the market research teams need information about purchase
657 histories and user preferences.
658 Neither needs access to the usernames and passwords.
659 However, because all of this data is stored in one ZFS filesystem,
660 it must all be sent and received together.
661 In addition, the owner of the data
662 wants to take advantage of features like compression, checksumming, and
663 snapshots, so they do want to continue to use ZFS to store and transmit their
665 Redaction can help them do so.
666 First, they would make two clones of a snapshot of the data on the source.
667 In one clone, they create the setup they want their market research team to see;
668 they delete the usernames and passwords file,
669 and overwrite the click tracking data with dummy information.
670 In another, they create the setup they want the development teams
671 to see, by replacing the passwords with fake information and replacing the
672 purchase histories with randomly generated ones.
673 They would then create a redaction bookmark on the parent snapshot,
674 using snapshots on the two clones as redaction snapshots.
675 The parent can then be sent, redacted, to the target
676 server where the research and development teams have access.
677 Finally, incremental sends from the parent snapshot to each of the clones can be
679 to and received on the target server; these snapshots are identical to the
680 ones on the source, and are ready to be used, while the parent snapshot on the
681 target contains none of the username and password data present on the source,
682 because it was removed by the redacted send operation.
689 .\" These are, respectively, examples 12, 13 from zfs.8
690 .\" Make sure to update them bidirectionally
691 .Ss Example 1 : No Remotely Replicating ZFS Data
692 The following commands send a full stream and then an incremental stream to a
693 remote machine, restoring them into
694 .Em poolB/received/fs@a
696 .Em poolB/received/fs@b ,
699 must contain the file system
701 and must not initially contain
702 .Em poolB/received/fs .
703 .Bd -literal -compact -offset Ds
704 .No # Nm zfs Cm send Ar pool/fs@a |
705 .No " " Nm ssh Ar host Nm zfs Cm receive Ar poolB/received/fs Ns @ Ns Ar a
706 .No # Nm zfs Cm send Fl i Ar a pool/fs@b |
707 .No " " Nm ssh Ar host Nm zfs Cm receive Ar poolB/received/fs
710 .Ss Example 2 : No Using the Nm zfs Cm receive Fl d No Option
711 The following command sends a full stream of
712 .Ar poolA/fsA/fsB@snap
713 to a remote machine, receiving it into
714 .Ar poolB/received/fsA/fsB@snap .
717 portion of the received snapshot's name is determined from the name of the sent
720 must contain the file system
723 .Ar poolB/received/fsA
724 does not exist, it is created as an empty file system.
725 .Bd -literal -compact -offset Ds
726 .No # Nm zfs Cm send Ar poolA/fsA/fsB@snap |
727 .No " " Nm ssh Ar host Nm zfs Cm receive Fl d Ar poolB/received
projects / mirror_zfs.git / blob