.\" Copyright 2017 Nexenta Systems, Inc.
.\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved.
.\"
-.Dd April 27, 2018
+.Dd November 29, 2018
.Dt ZPOOL 8 SMM
.Os Linux
.Sh NAME
.Nd configure ZFS storage pools
.Sh SYNOPSIS
.Nm
-.Fl ?
+.Fl ?V
.Nm
.Cm add
.Op Fl fgLnP
.Ar pool Ns | Ns Ar id
.Op Ar newpool Oo Fl t Oc
.Nm
+.Cm initialize
+.Op Fl c | Fl s
+.Ar pool
+.Op Ar device Ns ...
+.Nm
.Cm iostat
.Op Oo Oo Fl c Ar SCRIPT Oc Oo Fl lq Oc Oc Ns | Ns Fl rw
.Op Fl T Sy u Ns | Ns Sy d
-.Op Fl ghHLpPvy
+.Op Fl ghHLnpPvy
.Oo Oo Ar pool Ns ... Oc Ns | Ns Oo Ar pool vdev Ns ... Oc Ns | Ns Oo Ar vdev Ns ... Oc Oc
.Op Ar interval Op Ar count
.Nm
.Op Fl s | Fl p
.Ar pool Ns ...
.Nm
+.Cm trim
+.Op Fl d
+.Op Fl r Ar rate
+.Op Fl c | Fl s
+.Ar pool
+.Op Ar device Ns ...
+.Nm
.Cm set
.Ar property Ns = Ns Ar value
.Ar pool
.Nm
.Cm status
.Oo Fl c Ar SCRIPT Oc
-.Op Fl gLPvxD
+.Op Fl DigLpPstvx
.Op Fl T Sy u Ns | Ns Sy d
.Oo Ar pool Oc Ns ...
.Op Ar interval Op Ar count
.Cm upgrade
.Op Fl V Ar version
.Fl a Ns | Ns Ar pool Ns ...
+.Nm
+.Cm version
.Sh DESCRIPTION
The
.Nm
parity disks.
The recommended number is between 3 and 9 to help increase performance.
.It Sy spare
-A special pseudo-vdev which keeps track of available hot spares for a pool.
+A pseudo-vdev which keeps track of available hot spares for a pool.
For more information, see the
.Sx Hot Spares
section.
.Sx Intent Log
section.
.It Sy dedup
-A device dedicated solely for allocating dedup data.
+A device dedicated solely for dedup data.
The redundancy of this device should match the redundancy of the other normal
devices in the pool. If more than one dedup device is specified, then
-allocations are load-balanced between devices.
+allocations are load-balanced between those devices.
.It Sy special
A device dedicated solely for allocating various kinds of internal metadata,
and optionally small file data.
The redundancy of this device should match the redundancy of the other normal
devices in the pool. If more than one special device is specified, then
-allocations are load-balanced between devices.
+allocations are load-balanced between those devices.
.Pp
For more information on special allocations, see the
.Sx Special Allocation Class
exported since other pools may use this shared spare, which may lead to
potential data corruption.
.Pp
+Shared spares add some risk. If the pools are imported on different hosts, and
+both pools suffer a device failure at the same time, both could attempt to use
+the spare at the same time. This may not be detected, resulting in data
+corruption.
+.Pp
An in-progress spare replacement can be cancelled by detaching the hot spare.
If the original faulted device is detached, then the hot spare assumes its
place in the configuration, and is removed from the spare list of all active
can control the size of small file blocks allowed in the special class by
setting the
.Sy special_small_blocks
-dataset property. It defaults to zero so you must opt-in by setting it to a
+dataset property. It defaults to zero, so you must opt-in by setting it to a
non-zero value. See
.Xr zfs 8
for more info on setting this property.
.Bl -tag -width Ds
.It Cm allocated
Amount of storage used within the pool.
+See
+.Sy fragmentation
+and
+.Sy free
+for more information.
.It Sy capacity
Percentage of pool space used.
This property can also be referred to by its shortened column name,
.Pc .
This space occurs when a LUN is dynamically expanded.
.It Sy fragmentation
-The amount of fragmentation in the pool.
+The amount of fragmentation in the pool. As the amount of space
+.Sy allocated
+increases, it becomes more difficult to locate
+.Sy free
+space. This may result in lower write performance compared to pools with more
+unfragmented free space.
.It Sy free
The amount of free space available in the pool.
+By contrast, the
+.Xr zfs 8
+.Sy available
+property describes how much new data can be written to ZFS filesystems/volumes.
+The zpool
+.Sy free
+property is not generally useful for this purpose, and can be substantially more than the zfs
+.Sy available
+space. This discrepancy is due to several factors, including raidz party; zfs
+reservation, quota, refreservation, and refquota properties; and space set aside by
+.Sy spa_slop_shift
+(see
+.Xr zfs-module-parameters 5
+for more information).
.It Sy freeing
After a file system or snapshot is destroyed, the space it was using is
returned to the pool asynchronously.
.Sy 2
(internally referred to as
.Sy ashift
-). Values from 9 to 16, inclusive, are valid; also, the special
+). Values from 9 to 16, inclusive, are valid; also, the
value 0 (the default) means to auto-detect using the kernel's block
layer and a ZFS internal exception list. I/O operations will be aligned
to the specified size boundaries. Additionally, the minimum (disk)
Setting this property caches the pool configuration in a different location that
can later be imported with
.Nm zpool Cm import Fl c .
-Setting it to the special value
+Setting it to the value
.Sy none
-creates a temporary pool that is never cached, and the special value
+creates a temporary pool that is never cached, and the
.Qq
.Pq empty string
uses the default location.
An administrator can provide additional information about a pool using this
property.
.It Sy dedupditto Ns = Ns Ar number
+This property is deprecated. In a future release, it will no longer have any
+effect.
+.Pp
Threshold for the number of block ditto copies.
If the reference count for a deduplicated block increases above this number, a
new ditto copy of this block is automatically stored.
.It Sy panic
Prints out a message to the console and generates a system crash dump.
.El
+.It Sy autotrim Ns = Ns Sy on Ns | Ns Sy off
+When set to
+.Sy on
+space which has been recently freed, and is no longer allocated by the pool,
+will be periodically trimmed. This allows block device vdevs which support
+BLKDISCARD, such as SSDs, or file vdevs on which the underlying file system
+supports hole-punching, to reclaim unused blocks. The default setting for
+this property is
+.Sy off .
+.Pp
+Automatic TRIM does not immediately reclaim blocks after a free. Instead,
+it will optimistically delay allowing smaller ranges to be aggregated in to
+a few larger ones. These can then be issued more efficiently to the storage.
+.Pp
+Be aware that automatic trimming of recently freed data blocks can put
+significant stress on the underlying storage devices. This will vary
+depending of how well the specific device handles these commands. For
+lower end devices it is often possible to achieve most of the benefits
+of automatic trimming by running an on-demand (manual) TRIM periodically
+using the
+.Nm zpool Cm trim
+command.
.It Sy feature@ Ns Ar feature_name Ns = Ns Sy enabled
The value of this property is the current state of
.Ar feature_name .
When a pool is determined to be active it cannot be imported, even with the
.Fl f
option. This property is intended to be used in failover configurations
-where multiple hosts have access to a pool on shared storage. When this
-property is on, periodic writes to storage occur to show the pool is in use.
-See
+where multiple hosts have access to a pool on shared storage.
+.Pp
+Multihost provides protection on import only. It does not protect against an
+individual device being used in multiple pools, regardless of the type of vdev.
+See the discussion under
+.Sy zpool create.
+.Pp
+When this property is on, periodic writes to storage occur to show the pool is
+in use. See
.Sy zfs_multihost_interval
in the
.Xr zfs-module-parameters 5
Displays a help message.
.It Xo
.Nm
+.Fl V, -version
+.Xc
+An alias for the
+.Nm zpool Cm version
+subcommand.
+.It Xo
+.Nm
.Cm add
.Op Fl fgLnP
.Oo Fl o Ar property Ns = Ns Ar value Oc
.It Fl f
Forces use of
.Ar new_device ,
-even if its appears to be in use.
+even if it appears to be in use.
Not all devices can be overridden in this manner.
.It Fl o Ar property Ns = Ns Ar value
Sets the given pool properties. See the
If no arguments are specified, all device errors within the pool are cleared.
If one or more devices is specified, only those errors associated with the
specified device or devices are cleared.
+If multihost is enabled, and the pool has been suspended, this will not
+resume I/O. While the pool was suspended, it may have been imported on
+another host, and resuming I/O could result in pool damage.
.It Xo
.Nm
.Cm create
.Sx Virtual Devices
section.
.Pp
-The command verifies that each device specified is accessible and not currently
-in use by another subsystem.
+The command attempts to verify that each device specified is accessible and not
+currently in use by another subsystem. However this check is not robust enough
+to detect simultaneous attempts to use a new device in different pools, even if
+.Sy multihost
+is
+.Sy enabled.
+The
+administrator must ensure that simultaneous invocations of any combination of
+.Sy zpool replace ,
+.Sy zpool create ,
+.Sy zpool add ,
+or
+.Sy zpool labelclear ,
+do not refer to the same device. Using the same device in two pools will
+result in pool corruption.
+.Pp
There are some uses, such as being currently mounted, or specified as the
dedicated dump device, that prevents a device from ever being used by ZFS.
Other uses, such as having a preexisting UFS file system, can be overridden with
.El
.It Xo
.Nm
+.Cm initialize
+.Op Fl c | Fl s
+.Ar pool
+.Op Ar device Ns ...
+.Xc
+Begins initializing by writing to all unallocated regions on the specified
+devices, or all eligible devices in the pool if no individual devices are
+specified.
+Only leaf data or log devices may be initialized.
+.Bl -tag -width Ds
+.It Fl c, -cancel
+Cancel initializing on the specified devices, or all eligible devices if none
+are specified.
+If one or more target devices are invalid or are not currently being
+initialized, the command will fail and no cancellation will occur on any device.
+.It Fl s -suspend
+Suspend initializing on the specified devices, or all eligible devices if none
+are specified.
+If one or more target devices are invalid or are not currently being
+initialized, the command will fail and no suspension will occur on any device.
+Initializing can then be resumed by running
+.Nm zpool Cm initialize
+with no flags on the relevant target devices.
+.El
+.It Xo
+.Nm
.Cm iostat
.Op Oo Oo Fl c Ar SCRIPT Oc Oo Fl lq Oc Oc Ns | Ns Fl rw
.Op Fl T Sy u Ns | Ns Sy d
-.Op Fl ghHLpPvy
+.Op Fl ghHLnpPvy
.Oo Oo Ar pool Ns ... Oc Ns | Ns Oo Ar pool vdev Ns ... Oc Ns | Ns Oo Ar vdev Ns ... Oc Oc
.Op Ar interval Op Ar count
.Xc
-Displays I/O statistics for the given pools/vdevs. You can pass in a
-list of pools, a pool and list of vdevs in that pool, or a list of any
-vdevs from any pool. If no items are specified, statistics for every
-pool in the system are shown.
+Displays logical I/O statistics for the given pools/vdevs. Physical I/Os may
+be observed via
+.Xr iostat 1 .
+If writes are located nearby, they may be merged into a single
+larger operation. Additional I/O may be generated depending on the level of
+vdev redundancy.
+To filter output, you may pass in a list of pools, a pool and list of vdevs
+in that pool, or a list of any vdevs from any pool. If no items are specified,
+statistics for every pool in the system are shown.
When given an
.Ar interval ,
the statistics are printed every
.Ar interval
-seconds until ^C is pressed. If count is specified, the command exits
+seconds until ^C is pressed. If
+.Fl n
+flag is specified the headers are displayed only once, otherwise they are
+displayed periodically. If count is specified, the command exits
after count reports are printed. The first report printed is always
the statistics since boot regardless of whether
.Ar interval
be used to look up the current block device name regardless of the
.Pa /dev/disk/
path used to open it.
+.It Fl n
+Print headers only once when passed
.It Fl p
Display numbers in parsable (exact) values. Time values are in
nanoseconds.
.Fl L
flag.
.It Fl r
-Print request size histograms for the leaf ZIOs. This includes
-histograms of individual ZIOs (
-.Ar ind )
-and aggregate ZIOs (
-.Ar agg ).
-These stats can be useful for seeing how well the ZFS IO aggregator is
-working. Do not confuse these request size stats with the block layer
-requests; it's possible ZIOs can be broken up before being sent to the
-block device.
+Print request size histograms for the leaf vdev's IO. This includes
+histograms of individual IOs (ind) and aggregate IOs (agg). These stats
+can be useful for observing how well IO aggregation is working. Note
+that TRIM IOs may exceed 16M, but will be counted as 16M.
.It Fl v
Verbose statistics Reports usage statistics for individual vdevs within the
pool, in addition to the pool-wide statistics.
Omit statistics since boot.
Normally the first line of output reports the statistics since boot.
This option suppresses that first line of output.
+.Ar interval
.It Fl w
Display latency histograms:
.Pp
Does not include disk time.
.Ar scrub :
Average queuing time in scrub queue. Does not include disk time.
+.Ar trim :
+Average queuing time in trim queue. Does not include disk time.
.It Fl q
Include active queue statistics. Each priority queue has both
pending (
Current number of entries in asynchronous priority queues.
.Ar scrubq_read :
Current number of entries in scrub queue.
+.Ar trimq_write :
+Current number of entries in trim queue.
.Pp
All queue statistics are instantaneous measurements of the number of
entries in the queues. If you specify an interval, the measurements
.It Fl T Sy u Ns | Ns Sy d
Display a time stamp.
Specify
-.Fl u
+.Sy u
for a printed representation of the internal representation of time.
See
.Xr time 2 .
Specify
-.Fl d
+.Sy d
for standard date format.
See
.Xr date 1 .
command initiates the removal and returns, while the evacuation continues in
the background.
The removal progress can be monitored with
-.Nm zpool Cm status.
-The
+.Nm zpool Cm status .
+If an IO error is encountered during the removal process it will be
+cancelled. The
.Sy device_removal
feature flag must be enabled to remove a top-level vdev, see
.Xr zpool-features 5 .
.It Fl f
Forces use of
.Ar new_device ,
-even if its appears to be in use.
+even if it appears to be in use.
Not all devices can be overridden in this manner.
.It Fl o Ar property Ns = Ns Ar value
Sets the given pool properties. See the
resumes it.
If a resilver is in progress, ZFS does not allow a scrub to be started until the
resilver completes.
+.Pp
+Note that, due to changes in pool data on a live system, it is possible for
+scrubs to progress slightly beyond 100% completion. During this period, no
+completion time estimate will be provided.
.Bl -tag -width Ds
.It Fl s
Stop scrubbing.
resilver will be added to the new one.
.It Xo
.Nm
+.Cm trim
+.Op Fl d
+.Op Fl c | Fl s
+.Ar pool
+.Op Ar device Ns ...
+.Xc
+Initiates an immediate on-demand TRIM operation for all of the free space in
+a pool. This operation informs the underlying storage devices of all blocks
+in the pool which are no longer allocated and allows thinly provisioned
+devices to reclaim the space.
+.Pp
+A manual on-demand TRIM operation can be initiated irrespective of the
+.Sy autotrim
+pool property setting. See the documentation for the
+.Sy autotrim
+property above for the types of vdev devices which can be trimmed.
+.Bl -tag -width Ds
+.It Fl d -secure
+Causes a secure TRIM to be initiated. When performing a secure TRIM, the
+device guarantees that data stored on the trimmed blocks has been erased.
+This requires support from the device and is not supported by all SSDs.
+.It Fl r -rate Ar rate
+Controls the rate at which the TRIM operation progresses. Without this
+option TRIM is executed as quickly as possible. The rate, expressed in bytes
+per second, is applied on a per-vdev basis and may be set differently for
+each leaf vdev.
+.It Fl c, -cancel
+Cancel trimming on the specified devices, or all eligible devices if none
+are specified.
+If one or more target devices are invalid or are not currently being
+trimmed, the command will fail and no cancellation will occur on any device.
+.It Fl s -suspend
+Suspend trimming on the specified devices, or all eligible devices if none
+are specified.
+If one or more target devices are invalid or are not currently being
+trimmed, the command will fail and no suspension will occur on any device.
+Trimming can then be resumed by running
+.Nm zpool Cm trim
+with no flags on the relevant target devices.
+.El
+.It Xo
+.Nm
.Cm set
.Ar property Ns = Ns Ar value
.Ar pool
.Nm
.Cm status
.Op Fl c Op Ar SCRIPT1 Ns Oo , Ns Ar SCRIPT2 Oc Ns ...
-.Op Fl gLPvxD
+.Op Fl DigLpPstvx
.Op Fl T Sy u Ns | Ns Sy d
.Oo Ar pool Oc Ns ...
.Op Ar interval Op Ar count
option of
.Nm zpool Cm iostat
for complete details.
+.It Fl i
+Display vdev initialization status.
.It Fl g
Display vdev GUIDs instead of the normal device names. These GUIDs
can be used in place of device names for the zpool
be used to look up the current block device name regardless of the
.Pa /dev/disk/
path used to open it.
+.It Fl p
+Display numbers in parsable (exact) values.
.It Fl P
Display full paths for vdevs instead of only the last component of
the path. This can be used in conjunction with the
and referenced
.Pq logically referenced in the pool
block counts and sizes by reference count.
+.It Fl s
+Display the number of leaf VDEV slow IOs. This is the number of IOs that
+didn't complete in \fBzio_slow_io_ms\fR milliseconds (default 30 seconds).
+This does not necessarily mean the IOs failed to complete, just took an
+unreasonably long amount of time. This may indicate a problem with the
+underlying storage.
+.It Fl t
+Display vdev TRIM status.
.It Fl T Sy u Ns | Ns Sy d
Display a time stamp.
Specify
-.Fl u
+.Sy u
for a printed representation of the internal representation of time.
See
.Xr time 2 .
Specify
-.Fl d
+.Sy d
for standard date format.
See
.Xr date 1 .
This option can only be used to increase the version number up to the last
supported legacy version number.
.El
+.It Xo
+.Nm
+.Cm version
+.Xc
+Displays the software version of the
+.Nm
+userland utility and the zfs kernel module.
.El
.Sh EXIT STATUS
The following exit values are returned:
.Bl -tag -width "ZPOOL_VDEV_NAME_GUID"
.It Ev ZPOOL_VDEV_NAME_GUID
Cause
-.Nm zpool subcommands to output vdev guids by default. This behavior
-is identical to the
+.Nm zpool
+subcommands to output vdev guids by default. This behavior is identical to the
.Nm zpool status -g
command line option.
.El
projects / mirror_zfs.git / blobdiff