.\"
.\"
.\" Copyright (c) 2007, Sun Microsystems, Inc. All Rights Reserved.
-.\" Copyright (c) 2013 by Delphix. All rights reserved.
+.\" Copyright (c) 2012, 2018 by Delphix. All rights reserved.
.\" Copyright (c) 2012 Cyril Plisko. All Rights Reserved.
.\" Copyright (c) 2017 Datto Inc.
-.\" Copyright (c) 2017 George Melikov. All Rights Reserved.
+.\" Copyright (c) 2018 George Melikov. All Rights Reserved.
.\" Copyright 2017 Nexenta Systems, Inc.
.\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved.
.\"
-.Dd March 9, 2018
+.Dd November 29, 2018
.Dt ZPOOL 8 SMM
.Os Linux
.Sh NAME
.Nd configure ZFS storage pools
.Sh SYNOPSIS
.Nm
-.Fl ?
+.Fl ?V
.Nm
.Cm add
.Op Fl fgLnP
.Oo Fl o Ar property Ns = Ns Ar value Oc
.Ar pool device new_device
.Nm
+.Cm checkpoint
+.Op Fl d, -discard
+.Ar pool
+.Nm
.Cm clear
.Ar pool
.Op Ar device
.Op Fl Hp
.Op Fl o Ar field Ns Oo , Ns Ar field Oc Ns ...
.Sy all Ns | Ns Ar property Ns Oo , Ns Ar property Oc Ns ...
-.Ar pool Ns ...
+.Oo Ar pool Oc Ns ...
.Nm
.Cm history
.Op Fl il
.Fl a
.Op Fl DflmN
.Op Fl F Oo Fl n Oc Oo Fl T Oc Oo Fl X Oc
+.Op Fl -rewind-to-checkpoint
.Op Fl c Ar cachefile Ns | Ns Fl d Ar dir Ns | Ns device
.Op Fl o Ar mntopts
.Oo Fl o Ar property Ns = Ns Ar value Oc Ns ...
.Cm import
.Op Fl Dflm
.Op Fl F Oo Fl n Oc Oo Fl T Oc Oo Fl X Oc
+.Op Fl -rewind-to-checkpoint
.Op Fl c Ar cachefile Ns | Ns Fl d Ar dir Ns | Ns device
.Op Fl o Ar mntopts
.Oo Fl o Ar property Ns = Ns Ar value Oc Ns ...
.Ar pool Ns | Ns Ar id
.Op Ar newpool Oo Fl t Oc
.Nm
+.Cm initialize
+.Op Fl c | Fl s
+.Ar pool
+.Op Ar device Ns ...
+.Nm
.Cm iostat
.Op Oo Oo Fl c Ar SCRIPT Oc Oo Fl lq Oc Oc Ns | Ns Fl rw
.Op Fl T Sy u Ns | Ns Sy d
-.Op Fl ghHLpPvy
+.Op Fl ghHLnpPvy
.Oo Oo Ar pool Ns ... Oc Ns | Ns Oo Ar pool vdev Ns ... Oc Ns | Ns Oo Ar vdev Ns ... Oc Oc
.Op Ar interval Op Ar count
.Nm
.Ar pool
.Nm
.Cm remove
+.Op Fl np
.Ar pool Ar device Ns ...
.Nm
+.Cm remove
+.Fl s
+.Ar pool
+.Nm
.Cm replace
.Op Fl f
.Oo Fl o Ar property Ns = Ns Ar value Oc
.Ar pool Ar device Op Ar new_device
.Nm
+.Cm resilver
+.Ar pool Ns ...
+.Nm
.Cm scrub
.Op Fl s | Fl p
.Ar pool Ns ...
.Nm
+.Cm trim
+.Op Fl d
+.Op Fl r Ar rate
+.Op Fl c | Fl s
+.Ar pool
+.Op Ar device Ns ...
+.Nm
.Cm set
.Ar property Ns = Ns Ar value
.Ar pool
.Nm
.Cm status
.Oo Fl c Ar SCRIPT Oc
-.Op Fl gLPvxD
+.Op Fl DigLpPstvx
.Op Fl T Sy u Ns | Ns Sy d
.Oo Ar pool Oc Ns ...
.Op Ar interval Op Ar count
.Cm upgrade
.Op Fl V Ar version
.Fl a Ns | Ns Ar pool Ns ...
+.Nm
+.Cm version
.Sh DESCRIPTION
The
.Nm
parity disks.
The recommended number is between 3 and 9 to help increase performance.
.It Sy spare
-A special pseudo-vdev which keeps track of available hot spares for a pool.
+A pseudo-vdev which keeps track of available hot spares for a pool.
For more information, see the
.Sx Hot Spares
section.
For more information, see the
.Sx Intent Log
section.
+.It Sy dedup
+A device dedicated solely for dedup data.
+The redundancy of this device should match the redundancy of the other normal
+devices in the pool. If more than one dedup device is specified, then
+allocations are load-balanced between those devices.
+.It Sy special
+A device dedicated solely for allocating various kinds of internal metadata,
+and optionally small file data.
+The redundancy of this device should match the redundancy of the other normal
+devices in the pool. If more than one special device is specified, then
+allocations are load-balanced between those devices.
+.Pp
+For more information on special allocations, see the
+.Sx Special Allocation Class
+section.
.It Sy cache
A device used to cache storage pool data.
A cache device cannot be configured as a mirror or raidz group.
exported since other pools may use this shared spare, which may lead to
potential data corruption.
.Pp
+Shared spares add some risk. If the pools are imported on different hosts, and
+both pools suffer a device failure at the same time, both could attempt to use
+the spare at the same time. This may not be detected, resulting in data
+corruption.
+.Pp
An in-progress spare replacement can be cancelled by detaching the hot spare.
If the original faulted device is detached, then the hot spare assumes its
place in the configuration, and is removed from the spare list of all active
Log devices can be added, replaced, attached, detached and removed. In
addition, log devices are imported and exported as part of the pool
that contains them.
-Mirrored log devices can be removed by specifying the top-level mirror for the
-log.
+Mirrored devices can be removed by specifying the top-level mirror vdev.
.Ss Cache Devices
Devices can be added to a storage pool as
.Qq cache devices .
.Pp
The content of the cache devices is considered volatile, as is the case with
other system caches.
+.Ss Pool checkpoint
+Before starting critical procedures that include destructive actions (e.g
+.Nm zfs Cm destroy
+), an administrator can checkpoint the pool's state and in the case of a
+mistake or failure, rewind the entire pool back to the checkpoint.
+Otherwise, the checkpoint can be discarded when the procedure has completed
+successfully.
+.Pp
+A pool checkpoint can be thought of as a pool-wide snapshot and should be used
+with care as it contains every part of the pool's state, from properties to vdev
+configuration.
+Thus, while a pool has a checkpoint certain operations are not allowed.
+Specifically, vdev removal/attach/detach, mirror splitting, and
+changing the pool's guid.
+Adding a new vdev is supported but in the case of a rewind it will have to be
+added again.
+Finally, users of this feature should keep in mind that scrubs in a pool that
+has a checkpoint do not repair checkpointed data.
+.Pp
+To create a checkpoint for a pool:
+.Bd -literal
+# zpool checkpoint pool
+.Ed
+.Pp
+To later rewind to its checkpointed state, you need to first export it and
+then rewind it during import:
+.Bd -literal
+# zpool export pool
+# zpool import --rewind-to-checkpoint pool
+.Ed
+.Pp
+To discard the checkpoint from a pool:
+.Bd -literal
+# zpool checkpoint -d pool
+.Ed
+.Pp
+Dataset reservations (controlled by the
+.Nm reservation
+or
+.Nm refreservation
+zfs properties) may be unenforceable while a checkpoint exists, because the
+checkpoint is allowed to consume the dataset's reservation.
+Finally, data that is part of the checkpoint but has been freed in the
+current state of the pool won't be scanned during a scrub.
+.Ss Special Allocation Class
+The allocations in the special class are dedicated to specific block types.
+By default this includes all metadata, the indirect blocks of user data, and
+any dedup data. The class can also be provisioned to accept a limited
+percentage of small file data blocks.
+.Pp
+A pool must always have at least one general (non-specified) vdev before
+other devices can be assigned to the special class. If the special class
+becomes full, then allocations intended for it will spill back into the
+normal class.
+.Pp
+Dedup data can be excluded from the special class by setting the
+.Sy zfs_ddt_data_is_special
+zfs module parameter to false (0).
+.Pp
+Inclusion of small file blocks in the special class is opt-in. Each dataset
+can control the size of small file blocks allowed in the special class by
+setting the
+.Sy special_small_blocks
+dataset property. It defaults to zero, so you must opt-in by setting it to a
+non-zero value. See
+.Xr zfs 8
+for more info on setting this property.
.Ss Properties
Each pool has several properties associated with it.
Some properties are read-only statistics while others are configurable and
.Bl -tag -width Ds
.It Cm allocated
Amount of storage used within the pool.
+See
+.Sy fragmentation
+and
+.Sy free
+for more information.
.It Sy capacity
Percentage of pool space used.
This property can also be referred to by its shortened column name,
.Pc .
This space occurs when a LUN is dynamically expanded.
.It Sy fragmentation
-The amount of fragmentation in the pool.
+The amount of fragmentation in the pool. As the amount of space
+.Sy allocated
+increases, it becomes more difficult to locate
+.Sy free
+space. This may result in lower write performance compared to pools with more
+unfragmented free space.
.It Sy free
The amount of free space available in the pool.
+By contrast, the
+.Xr zfs 8
+.Sy available
+property describes how much new data can be written to ZFS filesystems/volumes.
+The zpool
+.Sy free
+property is not generally useful for this purpose, and can be substantially more than the zfs
+.Sy available
+space. This discrepancy is due to several factors, including raidz party; zfs
+reservation, quota, refreservation, and refquota properties; and space set aside by
+.Sy spa_slop_shift
+(see
+.Xr zfs-module-parameters 5
+for more information).
.It Sy freeing
After a file system or snapshot is destroyed, the space it was using is
returned to the pool asynchronously.
.Sy ONLINE , DEGRADED , FAULTED , OFFLINE, REMOVED , UNAVAIL .
.It Sy guid
A unique identifier for the pool.
+.It Sy load_guid
+A unique identifier for the pool.
+Unlike the
+.Sy guid
+property, this identifier is generated every time we load the pool (e.g. does
+not persist across imports/exports) and never changes while the pool is loaded
+(even if a
+.Sy reguid
+operation takes place).
.It Sy size
Total size of the storage pool.
.It Sy unsupported@ Ns Em feature_guid
.Sy 2
(internally referred to as
.Sy ashift
-). Values from 9 to 16, inclusive, are valid; also, the special
+). Values from 9 to 16, inclusive, are valid; also, the
value 0 (the default) means to auto-detect using the kernel's block
layer and a ZFS internal exception list. I/O operations will be aligned
to the specified size boundaries. Additionally, the minimum (disk)
Setting this property caches the pool configuration in a different location that
can later be imported with
.Nm zpool Cm import Fl c .
-Setting it to the special value
+Setting it to the value
.Sy none
-creates a temporary pool that is never cached, and the special value
+creates a temporary pool that is never cached, and the
.Qq
.Pq empty string
uses the default location.
An administrator can provide additional information about a pool using this
property.
.It Sy dedupditto Ns = Ns Ar number
+This property is deprecated. In a future release, it will no longer have any
+effect.
+.Pp
Threshold for the number of block ditto copies.
If the reference count for a deduplicated block increases above this number, a
new ditto copy of this block is automatically stored.
.It Sy panic
Prints out a message to the console and generates a system crash dump.
.El
+.It Sy autotrim Ns = Ns Sy on Ns | Ns Sy off
+When set to
+.Sy on
+space which has been recently freed, and is no longer allocated by the pool,
+will be periodically trimmed. This allows block device vdevs which support
+BLKDISCARD, such as SSDs, or file vdevs on which the underlying file system
+supports hole-punching, to reclaim unused blocks. The default setting for
+this property is
+.Sy off .
+.Pp
+Automatic TRIM does not immediately reclaim blocks after a free. Instead,
+it will optimistically delay allowing smaller ranges to be aggregated in to
+a few larger ones. These can then be issued more efficiently to the storage.
+.Pp
+Be aware that automatic trimming of recently freed data blocks can put
+significant stress on the underlying storage devices. This will vary
+depending of how well the specific device handles these commands. For
+lower end devices it is often possible to achieve most of the benefits
+of automatic trimming by running an on-demand (manual) TRIM periodically
+using the
+.Nm zpool Cm trim
+command.
.It Sy feature@ Ns Ar feature_name Ns = Ns Sy enabled
The value of this property is the current state of
.Ar feature_name .
When a pool is determined to be active it cannot be imported, even with the
.Fl f
option. This property is intended to be used in failover configurations
-where multiple hosts have access to a pool on shared storage. When this
-property is on, periodic writes to storage occur to show the pool is in use.
-See
+where multiple hosts have access to a pool on shared storage.
+.Pp
+Multihost provides protection on import only. It does not protect against an
+individual device being used in multiple pools, regardless of the type of vdev.
+See the discussion under
+.Sy zpool create.
+.Pp
+When this property is on, periodic writes to storage occur to show the pool is
+in use. See
.Sy zfs_multihost_interval
in the
.Xr zfs-module-parameters 5
Displays a help message.
.It Xo
.Nm
+.Fl V, -version
+.Xc
+An alias for the
+.Nm zpool Cm version
+subcommand.
+.It Xo
+.Nm
.Cm add
.Op Fl fgLnP
.Oo Fl o Ar property Ns = Ns Ar value Oc
Display real paths for
.Ar vdev Ns s
instead of only the last component of the path. This can be used in
-conjunction with the -L flag.
+conjunction with the
+.Fl L
+flag.
.It Fl o Ar property Ns = Ns Ar value
Sets the given pool properties. See the
.Sx Properties
.It Fl f
Forces use of
.Ar new_device ,
-even if its appears to be in use.
+even if it appears to be in use.
Not all devices can be overridden in this manner.
.It Fl o Ar property Ns = Ns Ar value
Sets the given pool properties. See the
.El
.It Xo
.Nm
+.Cm checkpoint
+.Op Fl d, -discard
+.Ar pool
+.Xc
+Checkpoints the current state of
+.Ar pool
+, which can be later restored by
+.Nm zpool Cm import --rewind-to-checkpoint .
+The existence of a checkpoint in a pool prohibits the following
+.Nm zpool
+commands:
+.Cm remove ,
+.Cm attach ,
+.Cm detach ,
+.Cm split ,
+and
+.Cm reguid .
+In addition, it may break reservation boundaries if the pool lacks free
+space.
+The
+.Nm zpool Cm status
+command indicates the existence of a checkpoint or the progress of discarding a
+checkpoint from a pool.
+The
+.Nm zpool Cm list
+command reports how much space the checkpoint takes from the pool.
+.Bl -tag -width Ds
+.It Fl d, -discard
+Discards an existing checkpoint from
+.Ar pool .
+.El
+.It Xo
+.Nm
.Cm clear
.Ar pool
.Op Ar device
If no arguments are specified, all device errors within the pool are cleared.
If one or more devices is specified, only those errors associated with the
specified device or devices are cleared.
+If multihost is enabled, and the pool has been suspended, this will not
+resume I/O. While the pool was suspended, it may have been imported on
+another host, and resuming I/O could result in pool damage.
.It Xo
.Nm
.Cm create
.Sx Virtual Devices
section.
.Pp
-The command verifies that each device specified is accessible and not currently
-in use by another subsystem.
+The command attempts to verify that each device specified is accessible and not
+currently in use by another subsystem. However this check is not robust enough
+to detect simultaneous attempts to use a new device in different pools, even if
+.Sy multihost
+is
+.Sy enabled.
+The
+administrator must ensure that simultaneous invocations of any combination of
+.Sy zpool replace ,
+.Sy zpool create ,
+.Sy zpool add ,
+or
+.Sy zpool labelclear ,
+do not refer to the same device. Using the same device in two pools will
+result in pool corruption.
+.Pp
There are some uses, such as being currently mounted, or specified as the
dedicated dump device, that prevents a device from ever being used by ZFS.
Other uses, such as having a preexisting UFS file system, can be overridden with
.Op Fl Hp
.Op Fl o Ar field Ns Oo , Ns Ar field Oc Ns ...
.Sy all Ns | Ns Ar property Ns Oo , Ns Ar property Oc Ns ...
-.Ar pool Ns ...
+.Oo Ar pool Oc Ns ...
.Xc
Retrieves the given list of properties
.Po
.Sy altroot
property to
.Ar root .
+.It Fl -rewind-to-checkpoint
+Rewinds pool to the checkpointed state.
+Once the pool is imported with this flag there is no way to undo the rewind.
+All changes and data that were written after the checkpoint are lost!
+The only exception is when the
+.Sy readonly
+mounting option is enabled.
+In this case, the checkpointed state of the pool is opened and an
+administrator can see how the pool would look like if they were
+to fully rewind.
.It Fl s
Scan using the default search path, the libblkid cache will not be
consulted. A custom search path may be specified by setting the
.El
.It Xo
.Nm
+.Cm initialize
+.Op Fl c | Fl s
+.Ar pool
+.Op Ar device Ns ...
+.Xc
+Begins initializing by writing to all unallocated regions on the specified
+devices, or all eligible devices in the pool if no individual devices are
+specified.
+Only leaf data or log devices may be initialized.
+.Bl -tag -width Ds
+.It Fl c, -cancel
+Cancel initializing on the specified devices, or all eligible devices if none
+are specified.
+If one or more target devices are invalid or are not currently being
+initialized, the command will fail and no cancellation will occur on any device.
+.It Fl s -suspend
+Suspend initializing on the specified devices, or all eligible devices if none
+are specified.
+If one or more target devices are invalid or are not currently being
+initialized, the command will fail and no suspension will occur on any device.
+Initializing can then be resumed by running
+.Nm zpool Cm initialize
+with no flags on the relevant target devices.
+.El
+.It Xo
+.Nm
.Cm iostat
.Op Oo Oo Fl c Ar SCRIPT Oc Oo Fl lq Oc Oc Ns | Ns Fl rw
.Op Fl T Sy u Ns | Ns Sy d
-.Op Fl ghHLpPvy
+.Op Fl ghHLnpPvy
.Oo Oo Ar pool Ns ... Oc Ns | Ns Oo Ar pool vdev Ns ... Oc Ns | Ns Oo Ar vdev Ns ... Oc Oc
.Op Ar interval Op Ar count
.Xc
-Displays I/O statistics for the given pools/vdevs. You can pass in a
-list of pools, a pool and list of vdevs in that pool, or a list of any
-vdevs from any pool. If no items are specified, statistics for every
-pool in the system are shown.
+Displays logical I/O statistics for the given pools/vdevs. Physical I/Os may
+be observed via
+.Xr iostat 1 .
+If writes are located nearby, they may be merged into a single
+larger operation. Additional I/O may be generated depending on the level of
+vdev redundancy.
+To filter output, you may pass in a list of pools, a pool and list of vdevs
+in that pool, or a list of any vdevs from any pool. If no items are specified,
+statistics for every pool in the system are shown.
When given an
.Ar interval ,
the statistics are printed every
.Ar interval
-seconds until ^C is pressed. If count is specified, the command exits
+seconds until ^C is pressed. If
+.Fl n
+flag is specified the headers are displayed only once, otherwise they are
+displayed periodically. If count is specified, the command exits
after count reports are printed. The first report printed is always
the statistics since boot regardless of whether
.Ar interval
be used to look up the current block device name regardless of the
.Pa /dev/disk/
path used to open it.
+.It Fl n
+Print headers only once when passed
.It Fl p
Display numbers in parsable (exact) values. Time values are in
nanoseconds.
.Fl L
flag.
.It Fl r
-Print request size histograms for the leaf ZIOs. This includes
-histograms of individual ZIOs (
-.Ar ind )
-and aggregate ZIOs (
-.Ar agg ).
-These stats can be useful for seeing how well the ZFS IO aggregator is
-working. Do not confuse these request size stats with the block layer
-requests; it's possible ZIOs can be broken up before being sent to the
-block device.
+Print request size histograms for the leaf vdev's IO. This includes
+histograms of individual IOs (ind) and aggregate IOs (agg). These stats
+can be useful for observing how well IO aggregation is working. Note
+that TRIM IOs may exceed 16M, but will be counted as 16M.
.It Fl v
Verbose statistics Reports usage statistics for individual vdevs within the
pool, in addition to the pool-wide statistics.
.It Fl y
+Omit statistics since boot.
+Normally the first line of output reports the statistics since boot.
+This option suppresses that first line of output.
+.Ar interval
.It Fl w
+Display latency histograms:
+.Pp
+.Ar total_wait :
+Total IO time (queuing + disk IO time).
+.Ar disk_wait :
+Disk IO time (time reading/writing the disk).
+.Ar syncq_wait :
+Amount of time IO spent in synchronous priority queues. Does not include
+disk time.
+.Ar asyncq_wait :
+Amount of time IO spent in asynchronous priority queues. Does not include
+disk time.
+.Ar scrub :
+Amount of time IO spent in scrub queue. Does not include disk time.
.It Fl l
Include average latency statistics:
.Pp
Does not include disk time.
.Ar scrub :
Average queuing time in scrub queue. Does not include disk time.
+.Ar trim :
+Average queuing time in trim queue. Does not include disk time.
.It Fl q
Include active queue statistics. Each priority queue has both
pending (
Current number of entries in asynchronous priority queues.
.Ar scrubq_read :
Current number of entries in scrub queue.
+.Ar trimq_write :
+Current number of entries in trim queue.
.Pp
All queue statistics are instantaneous measurements of the number of
entries in the queues. If you specify an interval, the measurements
.Sx Properties
section for a list of valid properties.
The default list is
-.Cm name , size , allocated , free , expandsize , fragmentation , capacity ,
-.Cm dedupratio , health , altroot .
+.Cm name , size , allocated , free , checkpoint, expandsize , fragmentation ,
+.Cm capacity , dedupratio , health , altroot .
.It Fl L
Display real paths for vdevs resolving all symbolic links. This can
be used to look up the current block device name regardless of the
.It Fl P
Display full paths for vdevs instead of only the last component of
the path. This can be used in conjunction with the
-.Fl L flag.
+.Fl L
+flag.
.It Fl T Sy u Ns | Ns Sy d
Display a time stamp.
Specify
-.Fl u
+.Sy u
for a printed representation of the internal representation of time.
See
.Xr time 2 .
Specify
-.Fl d
+.Sy d
for standard date format.
See
.Xr date 1 .
.It Xo
.Nm
.Cm remove
+.Op Fl np
.Ar pool Ar device Ns ...
.Xc
Removes the specified device from the pool.
-This command currently only supports removing hot spares, cache, and log
-devices.
-A mirrored log device can be removed by specifying the top-level mirror for the
-log.
-Non-log devices that are part of a mirrored configuration can be removed using
+This command supports removing hot spare, cache, log, and both mirrored and
+non-redundant primary top-level vdevs, including dedup and special vdevs.
+When the primary pool storage includes a top-level raidz vdev only hot spare,
+cache, and log devices can be removed.
+.sp
+Removing a top-level vdev reduces the total amount of space in the storage pool.
+The specified device will be evacuated by copying all allocated space from it to
+the other devices in the pool.
+In this case, the
+.Nm zpool Cm remove
+command initiates the removal and returns, while the evacuation continues in
+the background.
+The removal progress can be monitored with
+.Nm zpool Cm status .
+If an IO error is encountered during the removal process it will be
+cancelled. The
+.Sy device_removal
+feature flag must be enabled to remove a top-level vdev, see
+.Xr zpool-features 5 .
+.Pp
+A mirrored top-level device (log or data) can be removed by specifying the top-level mirror for the
+same.
+Non-log devices or data devices that are part of a mirrored configuration can be removed using
the
.Nm zpool Cm detach
command.
-Non-redundant and raidz devices cannot be removed from a pool.
+.Bl -tag -width Ds
+.It Fl n
+Do not actually perform the removal ("no-op").
+Instead, print the estimated amount of memory that will be used by the
+mapping table after the removal completes.
+This is nonzero only for top-level vdevs.
+.El
+.Bl -tag -width Ds
+.It Fl p
+Used in conjunction with the
+.Fl n
+flag, displays numbers as parsable (exact) values.
+.El
+.It Xo
+.Nm
+.Cm remove
+.Fl s
+.Ar pool
+.Xc
+Stops and cancels an in-progress removal of a top-level vdev.
.It Xo
.Nm
.Cm replace
.It Fl f
Forces use of
.Ar new_device ,
-even if its appears to be in use.
+even if it appears to be in use.
Not all devices can be overridden in this manner.
.It Fl o Ar property Ns = Ns Ar value
Sets the given pool properties. See the
resumes it.
If a resilver is in progress, ZFS does not allow a scrub to be started until the
resilver completes.
+.Pp
+Note that, due to changes in pool data on a live system, it is possible for
+scrubs to progress slightly beyond 100% completion. During this period, no
+completion time estimate will be provided.
.Bl -tag -width Ds
.It Fl s
Stop scrubbing.
.El
.It Xo
.Nm
+.Cm resilver
+.Ar pool Ns ...
+.Xc
+Starts a resilver. If an existing resilver is already running it will be
+restarted from the beginning. Any drives that were scheduled for a deferred
+resilver will be added to the new one.
+.It Xo
+.Nm
+.Cm trim
+.Op Fl d
+.Op Fl c | Fl s
+.Ar pool
+.Op Ar device Ns ...
+.Xc
+Initiates an immediate on-demand TRIM operation for all of the free space in
+a pool. This operation informs the underlying storage devices of all blocks
+in the pool which are no longer allocated and allows thinly provisioned
+devices to reclaim the space.
+.Pp
+A manual on-demand TRIM operation can be initiated irrespective of the
+.Sy autotrim
+pool property setting. See the documentation for the
+.Sy autotrim
+property above for the types of vdev devices which can be trimmed.
+.Bl -tag -width Ds
+.It Fl d -secure
+Causes a secure TRIM to be initiated. When performing a secure TRIM, the
+device guarantees that data stored on the trimmed blocks has been erased.
+This requires support from the device and is not supported by all SSDs.
+.It Fl r -rate Ar rate
+Controls the rate at which the TRIM operation progresses. Without this
+option TRIM is executed as quickly as possible. The rate, expressed in bytes
+per second, is applied on a per-vdev basis and may be set differently for
+each leaf vdev.
+.It Fl c, -cancel
+Cancel trimming on the specified devices, or all eligible devices if none
+are specified.
+If one or more target devices are invalid or are not currently being
+trimmed, the command will fail and no cancellation will occur on any device.
+.It Fl s -suspend
+Suspend trimming on the specified devices, or all eligible devices if none
+are specified.
+If one or more target devices are invalid or are not currently being
+trimmed, the command will fail and no suspension will occur on any device.
+Trimming can then be resumed by running
+.Nm zpool Cm trim
+with no flags on the relevant target devices.
+.El
+.It Xo
+.Nm
.Cm set
.Ar property Ns = Ns Ar value
.Ar pool
.It Fl P
Display full paths for vdevs instead of only the last component of
the path. This can be used in conjunction with the
-.Fl L flag.
+.Fl L
+flag.
.It Fl o Ar property Ns = Ns Ar value
Sets the specified property for
.Ar newpool .
.Nm
.Cm status
.Op Fl c Op Ar SCRIPT1 Ns Oo , Ns Ar SCRIPT2 Oc Ns ...
-.Op Fl gLPvxD
+.Op Fl DigLpPstvx
.Op Fl T Sy u Ns | Ns Sy d
.Oo Ar pool Oc Ns ...
.Op Ar interval Op Ar count
option of
.Nm zpool Cm iostat
for complete details.
+.It Fl i
+Display vdev initialization status.
.It Fl g
Display vdev GUIDs instead of the normal device names. These GUIDs
can be used in place of device names for the zpool
be used to look up the current block device name regardless of the
.Pa /dev/disk/
path used to open it.
+.It Fl p
+Display numbers in parsable (exact) values.
.It Fl P
Display full paths for vdevs instead of only the last component of
the path. This can be used in conjunction with the
-.Fl L flag.
+.Fl L
+flag.
.It Fl D
Display a histogram of deduplication statistics, showing the allocated
.Pq physically present on disk
and referenced
.Pq logically referenced in the pool
block counts and sizes by reference count.
+.It Fl s
+Display the number of leaf VDEV slow IOs. This is the number of IOs that
+didn't complete in \fBzio_slow_io_ms\fR milliseconds (default 30 seconds).
+This does not necessarily mean the IOs failed to complete, just took an
+unreasonably long amount of time. This may indicate a problem with the
+underlying storage.
+.It Fl t
+Display vdev TRIM status.
.It Fl T Sy u Ns | Ns Sy d
Display a time stamp.
Specify
-.Fl u
+.Sy u
for a printed representation of the internal representation of time.
See
.Xr time 2 .
Specify
-.Fl d
+.Sy d
for standard date format.
See
.Xr date 1 .
Once this is done, the pool will no longer be accessible on systems that do not
support feature flags.
See
-.Xr zfs-features 5
+.Xr zpool-features 5
for details on compatibility with systems that support feature flags, but do not
support all features enabled on the pool.
.Bl -tag -width Ds
This option can only be used to increase the version number up to the last
supported legacy version number.
.El
+.It Xo
+.Nm
+.Cm version
+.Xc
+Displays the software version of the
+.Nm
+userland utility and the zfs kernel module.
.El
.Sh EXIT STATUS
The following exit values are returned:
.Bd -literal
# zpool iostat -v pool 5
.Ed
-.It Sy Example 14 No Removing a Mirrored Log Device
-The following command removes the mirrored log device
-.Sy mirror-2 .
+.It Sy Example 14 No Removing a Mirrored top-level (Log or Data) Device
+The following commands remove the mirrored log device
+.Sy mirror-2
+and mirrored top-level data device
+.Sy mirror-1 .
+.Pp
Given this configuration:
.Bd -literal
pool: tank
.Bd -literal
# zpool remove tank mirror-2
.Ed
+.Pp
+The command to remove the mirrored data
+.Sy mirror-1
+is:
+.Bd -literal
+# zpool remove tank mirror-1
+.Ed
.It Sy Example 15 No Displaying expanded space on a device
The following command displays the detailed information for the pool
.Em data .
.Bl -tag -width "ZPOOL_VDEV_NAME_GUID"
.It Ev ZPOOL_VDEV_NAME_GUID
Cause
-.Nm zpool subcommands to output vdev guids by default. This behavior
-is identical to the
+.Nm zpool
+subcommands to output vdev guids by default. This behavior is identical to the
.Nm zpool status -g
command line option.
.El
projects / mirror_zfs.git / blobdiff