2 .\" Copyright (c) 2013 by Turbo Fredriksson <turbo@bayour.com>. All rights reserved.
3 .\" The contents of this file are subject to the terms of the Common Development
4 .\" and Distribution License (the "License"). You may not use this file except
5 .\" in compliance with the License. You can obtain a copy of the license at
6 .\" usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing.
8 .\" See the License for the specific language governing permissions and
9 .\" limitations under the License. When distributing Covered Code, include this
10 .\" CDDL HEADER in each file and include the License file at
11 .\" usr/src/OPENSOLARIS.LICENSE. If applicable, add the following below this
12 .\" CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your
13 .\" own identifying information:
14 .\" Portions Copyright [yyyy] [name of copyright owner]
15 .TH ZFS-MODULE-PARAMETERS 5 "Nov 16, 2013"
17 zfs\-module\-parameters \- ZFS module parameters
21 Description of the different parameters to the ZFS module.
23 .SS "Module parameters"
30 \fBl2arc_feed_again\fR (int)
35 Use \fB1\fR for yes (default) and \fB0\fR to disable.
41 \fBl2arc_feed_min_ms\fR (ulong)
44 Min feed interval in milliseconds
46 Default value: \fB200\fR.
52 \fBl2arc_feed_secs\fR (ulong)
55 Seconds between L2ARC writing
57 Default value: \fB1\fR.
63 \fBl2arc_headroom\fR (ulong)
66 Number of max device writes to precache
68 Default value: \fB2\fR.
74 \fBl2arc_headroom_boost\fR (ulong)
77 Compressed l2arc_headroom multiplier
79 Default value: \fB200\fR.
85 \fBl2arc_nocompress\fR (int)
88 Skip compressing L2ARC buffers
90 Use \fB1\fR for yes and \fB0\fR for no (default).
96 \fBl2arc_noprefetch\fR (int)
99 Skip caching prefetched buffers
101 Use \fB1\fR for yes (default) and \fB0\fR to disable.
107 \fBl2arc_norw\fR (int)
110 No reads during writes
112 Use \fB1\fR for yes and \fB0\fR for no (default).
118 \fBl2arc_write_boost\fR (ulong)
121 Extra write bytes during device warmup
123 Default value: \fB8,388,608\fR.
129 \fBl2arc_write_max\fR (ulong)
132 Max write bytes per interval
134 Default value: \fB8,388,608\fR.
140 \fBmetaslab_bias_enabled\fR (int)
143 Enable metaslab group biasing based on its vdev's over- or under-utilization
144 relative to the pool.
146 Use \fB1\fR for yes (default) and \fB0\fR for no.
152 \fBmetaslab_debug_load\fR (int)
155 Load all metaslabs during pool import.
157 Use \fB1\fR for yes and \fB0\fR for no (default).
163 \fBmetaslab_debug_unload\fR (int)
166 Prevent metaslabs from being unloaded.
168 Use \fB1\fR for yes and \fB0\fR for no (default).
174 \fBmetaslab_fragmentation_factor_enabled\fR (int)
177 Enable use of the fragmentation metric in computing metaslab weights.
179 Use \fB1\fR for yes (default) and \fB0\fR for no.
185 \fBmetaslabs_per_vdev\fR (int)
188 When a vdev is added, it will be divided into approximately (but no more than) this number of metaslabs.
190 Default value: \fB200\fR.
196 \fBmetaslab_preload_enabled\fR (int)
199 Enable metaslab group preloading.
201 Use \fB1\fR for yes (default) and \fB0\fR for no.
207 \fBmetaslab_lba_weighting_enabled\fR (int)
210 Give more weight to metaslabs with lower LBAs, assuming they have
211 greater bandwidth as is typically the case on a modern constant
212 angular velocity disk drive.
214 Use \fB1\fR for yes (default) and \fB0\fR for no.
220 \fBspa_config_path\fR (charp)
225 Default value: \fB/etc/zfs/zpool.cache\fR.
231 \fBspa_asize_inflation\fR (int)
234 Multiplication factor used to estimate actual disk consumption from the
235 size of data being written. The default value is a worst case estimate,
236 but lower values may be valid for a given pool depending on its
237 configuration. Pool administrators who understand the factors involved
238 may wish to specify a more realistic inflation factor, particularly if
239 they operate close to quota or capacity limits.
247 \fBspa_load_verify_data\fR (int)
250 Whether to traverse data blocks during an "extreme rewind" (\fB-X\fR)
251 import. Use 0 to disable and 1 to enable.
253 An extreme rewind import normally performs a full traversal of all
254 blocks in the pool for verification. If this parameter is set to 0,
255 the traversal skips non-metadata blocks. It can be toggled once the
256 import has started to stop or start the traversal of non-metadata blocks.
264 \fBspa_load_verify_metadata\fR (int)
267 Whether to traverse blocks during an "extreme rewind" (\fB-X\fR)
268 pool import. Use 0 to disable and 1 to enable.
270 An extreme rewind import normally performs a full traversal of all
271 blocks in the pool for verification. If this parameter is set to 1,
272 the traversal is not performed. It can be toggled once the import has
273 started to stop or start the traversal.
281 \fBspa_load_verify_maxinflight\fR (int)
284 Maximum concurrent I/Os during the traversal performed during an "extreme
285 rewind" (\fB-X\fR) pool import.
293 \fBzfetch_array_rd_sz\fR (ulong)
296 If prefetching is enabled, disable prefetching for reads larger than this size.
298 Default value: \fB1,048,576\fR.
304 \fBzfetch_block_cap\fR (uint)
307 Max number of blocks to prefetch at a time
309 Default value: \fB256\fR.
315 \fBzfetch_max_streams\fR (uint)
318 Max number of streams per zfetch (prefetch streams per file).
320 Default value: \fB8\fR.
326 \fBzfetch_min_sec_reap\fR (uint)
329 Min time before an active prefetch stream can be reclaimed
331 Default value: \fB2\fR.
337 \fBzfs_arc_average_blocksize\fR (int)
340 The ARC's buffer hash table is sized based on the assumption of an average
341 block size of \fBzfs_arc_average_blocksize\fR (default 8K). This works out
342 to roughly 1MB of hash table per 1GB of physical memory with 8-byte pointers.
343 For configurations with a known larger average block size this value can be
344 increased to reduce the memory footprint.
347 Default value: \fB8192\fR.
353 \fBzfs_arc_grow_retry\fR (int)
356 Seconds before growing arc size
358 Default value: \fB5\fR.
364 \fBzfs_arc_max\fR (ulong)
369 Default value: \fB0\fR.
375 \fBzfs_arc_memory_throttle_disable\fR (int)
378 Disable memory throttle
380 Use \fB1\fR for yes (default) and \fB0\fR to disable.
386 \fBzfs_arc_meta_limit\fR (ulong)
389 Meta limit for arc size
391 Default value: \fB0\fR.
397 \fBzfs_arc_meta_prune\fR (int)
400 Bytes of meta data to prune
402 Default value: \fB1,048,576\fR.
408 \fBzfs_arc_min\fR (ulong)
413 Default value: \fB100\fR.
419 \fBzfs_arc_min_prefetch_lifespan\fR (int)
422 Min life of prefetch block
424 Default value: \fB100\fR.
430 \fBzfs_arc_p_aggressive_disable\fR (int)
433 Disable aggressive arc_p growth
435 Use \fB1\fR for yes (default) and \fB0\fR to disable.
441 \fBzfs_arc_p_dampener_disable\fR (int)
444 Disable arc_p adapt dampener
446 Use \fB1\fR for yes (default) and \fB0\fR to disable.
452 \fBzfs_arc_shrink_shift\fR (int)
455 log2(fraction of arc to reclaim)
457 Default value: \fB5\fR.
463 \fBzfs_autoimport_disable\fR (int)
466 Disable pool import at module load by ignoring the cache file (typically \fB/etc/zfs/zpool.cache\fR).
468 Use \fB1\fR for yes and \fB0\fR for no (default).
474 \fBzfs_dbuf_state_index\fR (int)
477 Calculate arc header index
479 Default value: \fB0\fR.
485 \fBzfs_deadman_enabled\fR (int)
490 Use \fB1\fR for yes (default) and \fB0\fR to disable.
496 \fBzfs_deadman_synctime_ms\fR (ulong)
499 Expiration time in milliseconds. This value has two meanings. First it is
500 used to determine when the spa_deadman() logic should fire. By default the
501 spa_deadman() will fire if spa_sync() has not completed in 1000 seconds.
502 Secondly, the value determines if an I/O is considered "hung". Any I/O that
503 has not completed in zfs_deadman_synctime_ms is considered "hung" resulting
504 in a zevent being logged.
506 Default value: \fB1,000,000\fR.
512 \fBzfs_dedup_prefetch\fR (int)
515 Enable prefetching dedup-ed blks
517 Use \fB1\fR for yes and \fB0\fR to disable (default).
523 \fBzfs_delay_min_dirty_percent\fR (int)
526 Start to delay each transaction once there is this amount of dirty data,
527 expressed as a percentage of \fBzfs_dirty_data_max\fR.
528 This value should be >= zfs_vdev_async_write_active_max_dirty_percent.
529 See the section "ZFS TRANSACTION DELAY".
531 Default value: \fB60\fR.
537 \fBzfs_delay_scale\fR (int)
540 This controls how quickly the transaction delay approaches infinity.
541 Larger values cause longer delays for a given amount of dirty data.
543 For the smoothest delay, this value should be about 1 billion divided
544 by the maximum number of operations per second. This will smoothly
545 handle between 10x and 1/10th this number.
547 See the section "ZFS TRANSACTION DELAY".
549 Note: \fBzfs_delay_scale\fR * \fBzfs_dirty_data_max\fR must be < 2^64.
551 Default value: \fB500,000\fR.
557 \fBzfs_dirty_data_max\fR (int)
560 Determines the dirty space limit in bytes. Once this limit is exceeded, new
561 writes are halted until space frees up. This parameter takes precedence
562 over \fBzfs_dirty_data_max_percent\fR.
563 See the section "ZFS TRANSACTION DELAY".
565 Default value: 10 percent of all memory, capped at \fBzfs_dirty_data_max_max\fR.
571 \fBzfs_dirty_data_max_max\fR (int)
574 Maximum allowable value of \fBzfs_dirty_data_max\fR, expressed in bytes.
575 This limit is only enforced at module load time, and will be ignored if
576 \fBzfs_dirty_data_max\fR is later changed. This parameter takes
577 precedence over \fBzfs_dirty_data_max_max_percent\fR. See the section
578 "ZFS TRANSACTION DELAY".
580 Default value: 25% of physical RAM.
586 \fBzfs_dirty_data_max_max_percent\fR (int)
589 Maximum allowable value of \fBzfs_dirty_data_max\fR, expressed as a
590 percentage of physical RAM. This limit is only enforced at module load
591 time, and will be ignored if \fBzfs_dirty_data_max\fR is later changed.
592 The parameter \fBzfs_dirty_data_max_max\fR takes precedence over this
593 one. See the section "ZFS TRANSACTION DELAY".
601 \fBzfs_dirty_data_max_percent\fR (int)
604 Determines the dirty space limit, expressed as a percentage of all
605 memory. Once this limit is exceeded, new writes are halted until space frees
606 up. The parameter \fBzfs_dirty_data_max\fR takes precedence over this
607 one. See the section "ZFS TRANSACTION DELAY".
609 Default value: 10%, subject to \fBzfs_dirty_data_max_max\fR.
615 \fBzfs_dirty_data_sync\fR (int)
618 Start syncing out a transaction group if there is at least this much dirty data.
620 Default value: \fB67,108,864\fR.
626 \fBzfs_vdev_async_read_max_active\fR (int)
629 Maxium asynchronous read I/Os active to each device.
630 See the section "ZFS I/O SCHEDULER".
632 Default value: \fB3\fR.
638 \fBzfs_vdev_async_read_min_active\fR (int)
641 Minimum asynchronous read I/Os active to each device.
642 See the section "ZFS I/O SCHEDULER".
644 Default value: \fB1\fR.
650 \fBzfs_vdev_async_write_active_max_dirty_percent\fR (int)
653 When the pool has more than
654 \fBzfs_vdev_async_write_active_max_dirty_percent\fR dirty data, use
655 \fBzfs_vdev_async_write_max_active\fR to limit active async writes. If
656 the dirty data is between min and max, the active I/O limit is linearly
657 interpolated. See the section "ZFS I/O SCHEDULER".
659 Default value: \fB60\fR.
665 \fBzfs_vdev_async_write_active_min_dirty_percent\fR (int)
668 When the pool has less than
669 \fBzfs_vdev_async_write_active_min_dirty_percent\fR dirty data, use
670 \fBzfs_vdev_async_write_min_active\fR to limit active async writes. If
671 the dirty data is between min and max, the active I/O limit is linearly
672 interpolated. See the section "ZFS I/O SCHEDULER".
674 Default value: \fB30\fR.
680 \fBzfs_vdev_async_write_max_active\fR (int)
683 Maxium asynchronous write I/Os active to each device.
684 See the section "ZFS I/O SCHEDULER".
686 Default value: \fB10\fR.
692 \fBzfs_vdev_async_write_min_active\fR (int)
695 Minimum asynchronous write I/Os active to each device.
696 See the section "ZFS I/O SCHEDULER".
698 Default value: \fB1\fR.
704 \fBzfs_vdev_max_active\fR (int)
707 The maximum number of I/Os active to each device. Ideally, this will be >=
708 the sum of each queue's max_active. It must be at least the sum of each
709 queue's min_active. See the section "ZFS I/O SCHEDULER".
711 Default value: \fB1,000\fR.
717 \fBzfs_vdev_scrub_max_active\fR (int)
720 Maxium scrub I/Os active to each device.
721 See the section "ZFS I/O SCHEDULER".
723 Default value: \fB2\fR.
729 \fBzfs_vdev_scrub_min_active\fR (int)
732 Minimum scrub I/Os active to each device.
733 See the section "ZFS I/O SCHEDULER".
735 Default value: \fB1\fR.
741 \fBzfs_vdev_sync_read_max_active\fR (int)
744 Maxium synchronous read I/Os active to each device.
745 See the section "ZFS I/O SCHEDULER".
747 Default value: \fB10\fR.
753 \fBzfs_vdev_sync_read_min_active\fR (int)
756 Minimum synchronous read I/Os active to each device.
757 See the section "ZFS I/O SCHEDULER".
759 Default value: \fB10\fR.
765 \fBzfs_vdev_sync_write_max_active\fR (int)
768 Maxium synchronous write I/Os active to each device.
769 See the section "ZFS I/O SCHEDULER".
771 Default value: \fB10\fR.
777 \fBzfs_vdev_sync_write_min_active\fR (int)
780 Minimum synchronous write I/Os active to each device.
781 See the section "ZFS I/O SCHEDULER".
783 Default value: \fB10\fR.
789 \fBzfs_disable_dup_eviction\fR (int)
792 Disable duplicate buffer eviction
794 Use \fB1\fR for yes and \fB0\fR for no (default).
800 \fBzfs_expire_snapshot\fR (int)
803 Seconds to expire .zfs/snapshot
805 Default value: \fB300\fR.
811 \fBzfs_flags\fR (int)
814 Set additional debugging flags
816 Default value: \fB1\fR.
822 \fBzfs_free_leak_on_eio\fR (int)
825 If destroy encounters an EIO while reading metadata (e.g. indirect
826 blocks), space referenced by the missing metadata can not be freed.
827 Normally this causes the background destroy to become "stalled", as
828 it is unable to make forward progress. While in this stalled state,
829 all remaining space to free from the error-encountering filesystem is
830 "temporarily leaked". Set this flag to cause it to ignore the EIO,
831 permanently leak the space from indirect blocks that can not be read,
832 and continue to free everything else that it can.
834 The default, "stalling" behavior is useful if the storage partially
835 fails (i.e. some but not all i/os fail), and then later recovers. In
836 this case, we will be able to continue pool operations while it is
837 partially failed, and when it recovers, we can continue to free the
838 space, with no leaks. However, note that this case is actually
841 Typically pools either (a) fail completely (but perhaps temporarily,
842 e.g. a top-level vdev going offline), or (b) have localized,
843 permanent errors (e.g. disk returns the wrong data due to bit flip or
844 firmware bug). In case (a), this setting does not matter because the
845 pool will be suspended and the sync thread will not be able to make
846 forward progress regardless. In case (b), because the error is
847 permanent, the best we can do is leak the minimum amount of space,
848 which is what setting this flag will do. Therefore, it is reasonable
849 for this flag to normally be set, but we chose the more conservative
850 approach of not setting it, so that there is no possibility of
851 leaking space in the "partial temporary" failure case.
853 Default value: \fB0\fR.
859 \fBzfs_free_min_time_ms\fR (int)
862 Min millisecs to free per txg
864 Default value: \fB1,000\fR.
870 \fBzfs_immediate_write_sz\fR (long)
873 Largest data block to write to zil
875 Default value: \fB32,768\fR.
881 \fBzfs_mdcomp_disable\fR (int)
884 Disable meta data compression
886 Use \fB1\fR for yes and \fB0\fR for no (default).
892 \fBzfs_metaslab_fragmentation_threshold\fR (int)
895 Allow metaslabs to keep their active state as long as their fragmentation
896 percentage is less than or equal to this value. An active metaslab that
897 exceeds this threshold will no longer keep its active status allowing
898 better metaslabs to be selected.
900 Default value: \fB70\fR.
906 \fBzfs_mg_fragmentation_threshold\fR (int)
909 Metaslab groups are considered eligible for allocations if their
910 fragmenation metric (measured as a percentage) is less than or equal to
911 this value. If a metaslab group exceeds this threshold then it will be
912 skipped unless all metaslab groups within the metaslab class have also
913 crossed this threshold.
915 Default value: \fB85\fR.
921 \fBzfs_mg_noalloc_threshold\fR (int)
924 Defines a threshold at which metaslab groups should be eligible for
925 allocations. The value is expressed as a percentage of free space
926 beyond which a metaslab group is always eligible for allocations.
927 If a metaslab group's free space is less than or equal to the
928 the threshold, the allocator will avoid allocating to that group
929 unless all groups in the pool have reached the threshold. Once all
930 groups have reached the threshold, all groups are allowed to accept
931 allocations. The default value of 0 disables the feature and causes
932 all metaslab groups to be eligible for allocations.
934 This parameter allows to deal with pools having heavily imbalanced
935 vdevs such as would be the case when a new vdev has been added.
936 Setting the threshold to a non-zero percentage will stop allocations
937 from being made to vdevs that aren't filled to the specified percentage
938 and allow lesser filled vdevs to acquire more allocations than they
939 otherwise would under the old \fBzfs_mg_alloc_failures\fR facility.
941 Default value: \fB0\fR.
947 \fBzfs_no_scrub_io\fR (int)
952 Use \fB1\fR for yes and \fB0\fR for no (default).
958 \fBzfs_no_scrub_prefetch\fR (int)
961 Set for no scrub prefetching
963 Use \fB1\fR for yes and \fB0\fR for no (default).
969 \fBzfs_nocacheflush\fR (int)
972 Disable cache flushes
974 Use \fB1\fR for yes and \fB0\fR for no (default).
980 \fBzfs_nopwrite_enabled\fR (int)
985 Use \fB1\fR for yes (default) and \fB0\fR to disable.
991 \fBzfs_pd_blks_max\fR (int)
994 Max number of blocks to prefetch
996 Default value: \fB100\fR.
1002 \fBzfs_prefetch_disable\fR (int)
1005 Disable all ZFS prefetching
1007 Use \fB1\fR for yes and \fB0\fR for no (default).
1013 \fBzfs_read_chunk_size\fR (long)
1016 Bytes to read per chunk
1018 Default value: \fB1,048,576\fR.
1024 \fBzfs_read_history\fR (int)
1027 Historic statistics for the last N reads
1029 Default value: \fB0\fR.
1035 \fBzfs_read_history_hits\fR (int)
1038 Include cache hits in read history
1040 Use \fB1\fR for yes and \fB0\fR for no (default).
1046 \fBzfs_recover\fR (int)
1049 Set to attempt to recover from fatal errors. This should only be used as a
1050 last resort, as it typically results in leaked space, or worse.
1052 Use \fB1\fR for yes and \fB0\fR for no (default).
1058 \fBzfs_resilver_delay\fR (int)
1061 Number of ticks to delay prior to issuing a resilver I/O operation when
1062 a non-resilver or non-scrub I/O operation has occurred within the past
1063 \fBzfs_scan_idle\fR ticks.
1065 Default value: \fB2\fR.
1071 \fBzfs_resilver_min_time_ms\fR (int)
1074 Min millisecs to resilver per txg
1076 Default value: \fB3,000\fR.
1082 \fBzfs_scan_idle\fR (int)
1085 Idle window in clock ticks. During a scrub or a resilver, if
1086 a non-scrub or non-resilver I/O operation has occurred during this
1087 window, the next scrub or resilver operation is delayed by, respectively
1088 \fBzfs_scrub_delay\fR or \fBzfs_resilver_delay\fR ticks.
1090 Default value: \fB50\fR.
1096 \fBzfs_scan_min_time_ms\fR (int)
1099 Min millisecs to scrub per txg
1101 Default value: \fB1,000\fR.
1107 \fBzfs_scrub_delay\fR (int)
1110 Number of ticks to delay prior to issuing a scrub I/O operation when
1111 a non-scrub or non-resilver I/O operation has occurred within the past
1112 \fBzfs_scan_idle\fR ticks.
1114 Default value: \fB4\fR.
1120 \fBzfs_send_corrupt_data\fR (int)
1123 Allow to send corrupt data (ignore read/checksum errors when sending data)
1125 Use \fB1\fR for yes and \fB0\fR for no (default).
1131 \fBzfs_sync_pass_deferred_free\fR (int)
1134 Defer frees starting in this pass
1136 Default value: \fB2\fR.
1142 \fBzfs_sync_pass_dont_compress\fR (int)
1145 Don't compress starting in this pass
1147 Default value: \fB5\fR.
1153 \fBzfs_sync_pass_rewrite\fR (int)
1156 Rewrite new bps starting in this pass
1158 Default value: \fB2\fR.
1164 \fBzfs_top_maxinflight\fR (int)
1167 Max I/Os per top-level vdev during scrub or resilver operations.
1169 Default value: \fB32\fR.
1175 \fBzfs_txg_history\fR (int)
1178 Historic statistics for the last N txgs
1180 Default value: \fB0\fR.
1186 \fBzfs_txg_timeout\fR (int)
1189 Max seconds worth of delta per txg
1191 Default value: \fB5\fR.
1197 \fBzfs_vdev_aggregation_limit\fR (int)
1200 Max vdev I/O aggregation size
1202 Default value: \fB131,072\fR.
1208 \fBzfs_vdev_cache_bshift\fR (int)
1211 Shift size to inflate reads too
1213 Default value: \fB16\fR.
1219 \fBzfs_vdev_cache_max\fR (int)
1222 Inflate reads small than max
1228 \fBzfs_vdev_cache_size\fR (int)
1231 Total size of the per-disk cache
1233 Default value: \fB0\fR.
1239 \fBzfs_vdev_mirror_switch_us\fR (int)
1242 Switch mirrors every N usecs
1244 Default value: \fB10,000\fR.
1250 \fBzfs_vdev_read_gap_limit\fR (int)
1253 Aggregate read I/O over gap
1255 Default value: \fB32,768\fR.
1261 \fBzfs_vdev_scheduler\fR (charp)
1266 Default value: \fBnoop\fR.
1272 \fBzfs_vdev_write_gap_limit\fR (int)
1275 Aggregate write I/O over gap
1277 Default value: \fB4,096\fR.
1283 \fBzfs_zevent_cols\fR (int)
1286 Max event column width
1288 Default value: \fB80\fR.
1294 \fBzfs_zevent_console\fR (int)
1297 Log events to the console
1299 Use \fB1\fR for yes and \fB0\fR for no (default).
1305 \fBzfs_zevent_len_max\fR (int)
1308 Max event queue length
1310 Default value: \fB0\fR.
1316 \fBzil_replay_disable\fR (int)
1319 Disable intent logging replay
1321 Use \fB1\fR for yes and \fB0\fR for no (default).
1327 \fBzil_slog_limit\fR (ulong)
1330 Max commit bytes to separate log device
1332 Default value: \fB1,048,576\fR.
1338 \fBzio_bulk_flags\fR (int)
1341 Additional flags to pass to bulk buffers
1343 Default value: \fB0\fR.
1349 \fBzio_delay_max\fR (int)
1352 Max zio millisec delay before posting event
1354 Default value: \fB30,000\fR.
1360 \fBzio_injection_enabled\fR (int)
1363 Enable fault injection
1365 Use \fB1\fR for yes and \fB0\fR for no (default).
1371 \fBzio_requeue_io_start_cut_in_line\fR (int)
1374 Prioritize requeued I/O
1376 Default value: \fB0\fR.
1382 \fBzvol_inhibit_dev\fR (uint)
1385 Do not create zvol device nodes
1387 Use \fB1\fR for yes and \fB0\fR for no (default).
1393 \fBzvol_major\fR (uint)
1396 Major number for zvol device
1398 Default value: \fB230\fR.
1404 \fBzvol_max_discard_blocks\fR (ulong)
1407 Max number of blocks to discard at once
1409 Default value: \fB16,384\fR.
1415 \fBzvol_threads\fR (uint)
1418 Number of threads for zvol device
1420 Default value: \fB32\fR.
1423 .SH ZFS I/O SCHEDULER
1424 ZFS issues I/O operations to leaf vdevs to satisfy and complete I/Os.
1425 The I/O scheduler determines when and in what order those operations are
1426 issued. The I/O scheduler divides operations into five I/O classes
1427 prioritized in the following order: sync read, sync write, async read,
1428 async write, and scrub/resilver. Each queue defines the minimum and
1429 maximum number of concurrent operations that may be issued to the
1430 device. In addition, the device has an aggregate maximum,
1431 \fBzfs_vdev_max_active\fR. Note that the sum of the per-queue minimums
1432 must not exceed the aggregate maximum. If the sum of the per-queue
1433 maximums exceeds the aggregate maximum, then the number of active I/Os
1434 may reach \fBzfs_vdev_max_active\fR, in which case no further I/Os will
1435 be issued regardless of whether all per-queue minimums have been met.
1437 For many physical devices, throughput increases with the number of
1438 concurrent operations, but latency typically suffers. Further, physical
1439 devices typically have a limit at which more concurrent operations have no
1440 effect on throughput or can actually cause it to decrease.
1442 The scheduler selects the next operation to issue by first looking for an
1443 I/O class whose minimum has not been satisfied. Once all are satisfied and
1444 the aggregate maximum has not been hit, the scheduler looks for classes
1445 whose maximum has not been satisfied. Iteration through the I/O classes is
1446 done in the order specified above. No further operations are issued if the
1447 aggregate maximum number of concurrent operations has been hit or if there
1448 are no operations queued for an I/O class that has not hit its maximum.
1449 Every time an I/O is queued or an operation completes, the I/O scheduler
1450 looks for new operations to issue.
1452 In general, smaller max_active's will lead to lower latency of synchronous
1453 operations. Larger max_active's may lead to higher overall throughput,
1454 depending on underlying storage.
1456 The ratio of the queues' max_actives determines the balance of performance
1457 between reads, writes, and scrubs. E.g., increasing
1458 \fBzfs_vdev_scrub_max_active\fR will cause the scrub or resilver to complete
1459 more quickly, but reads and writes to have higher latency and lower throughput.
1461 All I/O classes have a fixed maximum number of outstanding operations
1462 except for the async write class. Asynchronous writes represent the data
1463 that is committed to stable storage during the syncing stage for
1464 transaction groups. Transaction groups enter the syncing state
1465 periodically so the number of queued async writes will quickly burst up
1466 and then bleed down to zero. Rather than servicing them as quickly as
1467 possible, the I/O scheduler changes the maximum number of active async
1468 write I/Os according to the amount of dirty data in the pool. Since
1469 both throughput and latency typically increase with the number of
1470 concurrent operations issued to physical devices, reducing the
1471 burstiness in the number of concurrent operations also stabilizes the
1472 response time of operations from other -- and in particular synchronous
1473 -- queues. In broad strokes, the I/O scheduler will issue more
1474 concurrent operations from the async write queue as there's more dirty
1479 The number of concurrent operations issued for the async write I/O class
1480 follows a piece-wise linear function defined by a few adjustable points.
1483 | o---------| <-- zfs_vdev_async_write_max_active
1490 |-------o | | <-- zfs_vdev_async_write_min_active
1491 0|_______^______|_________|
1492 0% | | 100% of zfs_dirty_data_max
1494 | `-- zfs_vdev_async_write_active_max_dirty_percent
1495 `--------- zfs_vdev_async_write_active_min_dirty_percent
1498 Until the amount of dirty data exceeds a minimum percentage of the dirty
1499 data allowed in the pool, the I/O scheduler will limit the number of
1500 concurrent operations to the minimum. As that threshold is crossed, the
1501 number of concurrent operations issued increases linearly to the maximum at
1502 the specified maximum percentage of the dirty data allowed in the pool.
1504 Ideally, the amount of dirty data on a busy pool will stay in the sloped
1505 part of the function between \fBzfs_vdev_async_write_active_min_dirty_percent\fR
1506 and \fBzfs_vdev_async_write_active_max_dirty_percent\fR. If it exceeds the
1507 maximum percentage, this indicates that the rate of incoming data is
1508 greater than the rate that the backend storage can handle. In this case, we
1509 must further throttle incoming writes, as described in the next section.
1511 .SH ZFS TRANSACTION DELAY
1512 We delay transactions when we've determined that the backend storage
1513 isn't able to accommodate the rate of incoming writes.
1515 If there is already a transaction waiting, we delay relative to when
1516 that transaction will finish waiting. This way the calculated delay time
1517 is independent of the number of threads concurrently executing
1520 If we are the only waiter, wait relative to when the transaction
1521 started, rather than the current time. This credits the transaction for
1522 "time already served", e.g. reading indirect blocks.
1524 The minimum time for a transaction to take is calculated as:
1526 min_time = zfs_delay_scale * (dirty - min) / (max - dirty)
1527 min_time is then capped at 100 milliseconds.
1530 The delay has two degrees of freedom that can be adjusted via tunables. The
1531 percentage of dirty data at which we start to delay is defined by
1532 \fBzfs_delay_min_dirty_percent\fR. This should typically be at or above
1533 \fBzfs_vdev_async_write_active_max_dirty_percent\fR so that we only start to
1534 delay after writing at full speed has failed to keep up with the incoming write
1535 rate. The scale of the curve is defined by \fBzfs_delay_scale\fR. Roughly speaking,
1536 this variable determines the amount of delay at the midpoint of the curve.
1540 10ms +-------------------------------------------------------------*+
1556 2ms + (midpoint) * +
1559 | zfs_delay_scale ----------> ******** |
1560 0 +-------------------------------------*********----------------+
1561 0% <- zfs_dirty_data_max -> 100%
1564 Note that since the delay is added to the outstanding time remaining on the
1565 most recent transaction, the delay is effectively the inverse of IOPS.
1566 Here the midpoint of 500us translates to 2000 IOPS. The shape of the curve
1567 was chosen such that small changes in the amount of accumulated dirty data
1568 in the first 3/4 of the curve yield relatively small differences in the
1571 The effects can be easier to understand when the amount of delay is
1572 represented on a log scale:
1576 100ms +-------------------------------------------------------------++
1585 + zfs_delay_scale ----------> ***** +
1596 +--------------------------------------------------------------+
1597 0% <- zfs_dirty_data_max -> 100%
1600 Note here that only as the amount of dirty data approaches its limit does
1601 the delay start to increase rapidly. The goal of a properly tuned system
1602 should be to keep the amount of dirty data out of that range by first
1603 ensuring that the appropriate limits are set for the I/O scheduler to reach
1604 optimal throughput on the backend storage, and then by changing the value
1605 of \fBzfs_delay_scale\fR to increase the steepness of the curve.
projects / mirror_zfs.git / blob