]> git.proxmox.com Git - mirror_zfs.git/log
mirror_zfs.git
9 years agoIllumos 5314 - Remove "dbuf phys" db->db_data pointer aliases in ZFS
Justin T. Gibbs [Wed, 1 Apr 2015 15:14:34 +0000 (02:14 +1100)]
Illumos 5314 - Remove "dbuf phys" db->db_data pointer aliases in ZFS

5314 Remove "dbuf phys" db->db_data pointer aliases in ZFS
Author: Justin T. Gibbs <justing@spectralogic.com>
Reviewed by: Andriy Gapon <avg@freebsd.org>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Will Andrews <willa@spectralogic.com>
Approved by: Dan McDonald <danmcd@omniti.com>

References:
  https://www.illumos.org/issues/5314
  https://github.com/illumos/illumos-gate/commit/c137962

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoIllumos 5310 - Remove always true tests for non-NULL ds->ds_phys
Justin T. Gibbs [Wed, 1 Apr 2015 15:13:28 +0000 (02:13 +1100)]
Illumos 5310 - Remove always true tests for non-NULL ds->ds_phys

5310 Remove always true tests for non-NULL ds->ds_phys
Author: Justin T. Gibbs <justing@spectralogic.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Will Andrews <willa@spectralogic.com>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Approved by: Dan McDonald <danmcd@omniti.com>

References:
  https://www.illumos.org/issues/5310
  https://github.com/illumos/illumos-gate/commit/d808a4f

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoIllumos 5095 - panic when adding a duplicate dbuf to dn_dbufs
Alex Reece [Wed, 1 Apr 2015 15:10:58 +0000 (02:10 +1100)]
Illumos 5095 - panic when adding a duplicate dbuf to dn_dbufs

5095 panic when adding a duplicate dbuf to dn_dbufs
Author: Alex Reece <alex@delphix.com>
Reviewed by: Adam Leventhal <adam.leventhal@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Mattew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Reviewed by: Josef Sipek <jeffpc@josefsipek.net>
Approved by: Robert Mustacchi <rm@joyent.com>

References:
  https://www.illumos.org/issues/5095
  https://github.com/illumos/illumos-gate/commit/86bb58a

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoIllumos 5038 - Remove "old-style" flexible array usage in ZFS.
Justin T. Gibbs [Wed, 1 Apr 2015 15:09:20 +0000 (02:09 +1100)]
Illumos 5038 - Remove "old-style" flexible array usage in ZFS.

5038 Remove "old-style" flexible array usage in ZFS.
Author: Justin T. Gibbs <justing@spectralogic.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/5038
  https://github.com/illumos/illumos-gate/commit/7f18da4

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoIllumos 4873 - zvol unmap calls can take a very long time for larger datasets
Alex Reece [Fri, 3 Apr 2015 03:14:28 +0000 (14:14 +1100)]
Illumos 4873 - zvol unmap calls can take a very long time for larger datasets

4873 zvol unmap calls can take a very long time for larger datasets
Author: Alex Reece <alex@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <paul.dagnelie@delphix.com>
Reviewed by: Basil Crow <basil.crow@delphix.com>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Approved by: Robert Mustacchi <rm@joyent.com>

References:
  https://www.illumos.org/issues/4873
  https://github.com/illumos/illumos-gate/commit/0f6d88a

Porting Notes:

dbuf_free_range():
  - reduce stack usage using kmem_alloc()
  - the sorted AVL tree will handle the spill block case correctly
    without all the special handling in the for() loop

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoIllumos 4975 - missing mutex_destroy() calls in zfs
Jorgen Lundman [Wed, 1 Apr 2015 13:49:14 +0000 (00:49 +1100)]
Illumos 4975 - missing mutex_destroy() calls in zfs

4975 missing mutex_destroy() calls in zfs
Author: Jorgen Lundman <lundman@lundman.net>
Reviewed by: Matthew Ahrens <matthew.ahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Rich Lowe <richlowe@richlowe.net>
Reviewed by: Seth Nimbosa <darth.Serious@gmail.com>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Reviewed by: Don Brady <dev.fs.zfs@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>

References:
  https://www.illumos.org/issues/4975
  https://github.com/illumos/illumos-gate/commit/d2b3cbb

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoIllumos 3897 - zfs filesystem and snapshot limits (fix leak)
Alex Reece [Wed, 1 Apr 2015 13:10:21 +0000 (00:10 +1100)]
Illumos 3897 - zfs filesystem and snapshot limits (fix leak)

3897 zfs filesystem and snapshot limits (fix leak)
Author: Alex Reece <alex.reece@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3897
  https://github.com/illumos/illumos-gate/commit/fb7001f

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoIllumos 3897 - zfs filesystem and snapshot limits
Jerry Jelinek [Wed, 1 Apr 2015 13:07:48 +0000 (00:07 +1100)]
Illumos 3897 - zfs filesystem and snapshot limits

3897 zfs filesystem and snapshot limits
Author: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3897
  https://github.com/illumos/illumos-gate/commit/a2afb61

Porting Notes:

dsl_dataset_snapshot_check(): reduce stack usage using kmem_alloc().

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoIllumos 5134 - if ZFS_DEBUG or debug= is set, libzpool should enable debug prints
Matthew Ahrens [Fri, 5 Sep 2014 22:03:09 +0000 (00:03 +0200)]
Illumos 5134 - if ZFS_DEBUG or debug= is set, libzpool should enable debug prints

5134 if ZFS_DEBUG or debug= is set, libzpool should enable debug prints
Reviewed by: Adam Leventhal <adam.leventhal@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Saso Kiselkov <skiselkov.ml@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>

References:
  https://www.illumos.org/projects/illumos-gate/issues/5134
  https://github.com/illumos/illumos-gate/commit/7fa49ea

Porting notes:
  Added dprintf_setup() to main in zfs_main.c and zpool_main.c.

Ported by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2669

9 years agoFix misuse of input argument in traverse_visitbp
tuxoko [Sun, 26 Apr 2015 08:09:41 +0000 (16:09 +0800)]
Fix misuse of input argument in traverse_visitbp

In traverse_visitbp(), the input argument dnp is modified in the middle to
point to a temporary buffer. Originally this doesn't matter, because no user
of TRAVERSE_POST dereferences it. However, in fbeddd6 a piece of code is added
dereferencing dnp after the modification, creating a possible bug.

We fix this by creating a new local variable cdnp for the DMU_OT_DNODE case,
so we don't modify the input argument. Also we introduce different local
variables in the DMU_OT_OBJSET case to prevent confusion between the input
argument.

Signed-off-by: Chunwei Chen <tuxoko@gmail.com>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2060

9 years agoMerge branch 'zed-pushbullet'
Brian Behlendorf [Mon, 27 Apr 2015 19:15:42 +0000 (12:15 -0700)]
Merge branch 'zed-pushbullet'

This patch stack begins with cleaning up the existing ZEDLETs,
refactoring common code blocks into zed-functions.sh, adopting a
more consistent coding style, updating exit codes, etc. All scripts
now run cleanly through ShellCheck.

The old "email" ZEDLETs are replaced with new "notify" ZEDLETs. A
notification can now be sent via email and/or Pushbullet. Additional
notification methods will likely be added in the future.

Pushbullet notifications are enabled by setting the
ZED_PUSHBULLET_ACCESS_TOKEN and (optionally) ZED_PUSHBULLET_CHANNEL_TAG
in zed.rc. The Pushbullet implementation requires awk, curl, and sed
executables to be installed in the standard PATH.

Signed-off-by: Chris Dunlap <cdunlap@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3327

9 years agoCombine data-notify.sh with io-notify.sh
Chris Dunlap [Wed, 25 Mar 2015 20:10:32 +0000 (13:10 -0700)]
Combine data-notify.sh with io-notify.sh

The data-notify.sh ZEDLET serves a very similar purpose to
io-notify.sh, namely, to generate a notification in response to a
particular error event.  Initially, data-notify.sh was separated from
io-notify.sh since the "data" zevent does not (as I understand it)
pertain to a specific vdev device.  This stands in contrast to the
"checksum" and "io" zevents (both handled by io-notify.sh) that can
be attributed to a specific vdev.  At the time, it seemed simpler to
handle these two cases in separate scripts.

This commit adds support for the "data" zevent to io-notify.sh, and
symlinks io-notify.sh to data-notify.sh.  It also adds the counts
for vdev_read_errors, vdev_write_errors, and vdev_cksum_errors to
the notification message.

Signed-off-by: Chris Dunlap <cdunlap@llnl.gov>
9 years agoAdd notification to io-spare.sh
Chris Dunlap [Fri, 20 Mar 2015 00:40:54 +0000 (17:40 -0700)]
Add notification to io-spare.sh

The io-spare.sh ZEDLET does not generate a notification when a failing
device is replaced with a hot spare.  Maybe it should tell someone.

This commit adds a notification message to the io-spare.sh ZEDLET.
This notification is triggered when a failing device is successfully
replaced with a hot spare after encountering a checksum or io error.

Signed-off-by: Chris Dunlap <cdunlap@llnl.gov>
9 years agoAdd support for Pushbullet notifications
Chris Dunlap [Fri, 27 Feb 2015 00:26:48 +0000 (16:26 -0800)]
Add support for Pushbullet notifications

This commit adds the zed_notify_pushbullet() function and hooks
it into zed_notify(), thereby integrating it with the existing
"notify" ZEDLETs.  This enables ZED to push notifications to your
desktop computer and/or mobile device(s).  It is configured with the
ZED_PUSHBULLET_ACCESS_TOKEN and ZED_PUSHBULLET_CHANNEL_TAG variables
in zed.rc.

  https://www.pushbullet.com/

The Makefile install-data-local target has been replaced with
install-data-hook.  With the "-local" target, there is no particular
guarantee of execution order.  But with the zed.rc now potentially
containing sensitive information (i.e., the Pushbullet access token),
the recommended permissions have changed to 0600.  The "-hook" target
is always executed after the main rule's work is done; thus, the
chmod will always take place after the zed.rc file has been installed.

  https://www.gnu.org/software/automake/manual/automake.html#Extending

Signed-off-by: Chris Dunlap <cdunlap@llnl.gov>
9 years agoReplace "email" ZEDLETs with "notify" ZEDLETs
Chris Dunlap [Fri, 27 Feb 2015 00:26:25 +0000 (16:26 -0800)]
Replace "email" ZEDLETs with "notify" ZEDLETs

Several ZEDLETs already exist for sending email in reponse to a
particular zevent.  While email is ubiquitous, alternative methods may
be better suited for some configurations.  Instead of duplicating the
"email" ZEDLETs for every future notification method, it is preferable
to abstract the notification method into a function.  This has the
added benefit of reducing the amount of code duplicated between
ZEDLETs, and allowing related bugs to be fixed in a single location.

This commit replaces the existing "email" ZEDLETs with corresponding
"notify" ZEDLETs.  In addition, the ZEDLET code for sending an
email message has been moved into the zed_notify_email() function.
And this zed_notify_email() has been added to a generic zed_notify()
function for sending notifications via all available methods that
have been configured.

This commit also changes a couple of related zed.rc variables.
ZED_EMAIL_INTERVAL_SECS is changed to ZED_NOTIFY_INTERVAL_SECS,
and ZED_EMAIL_VERBOSE is changed to ZED_NOTIFY_VERBOSE.  Note that
ZED_EMAIL remains unchanged as its use is solely for the email
notification method.

Signed-off-by: Chris Dunlap <cdunlap@llnl.gov>
9 years agoCleanup ZEDLETs
Chris Dunlap [Wed, 18 Feb 2015 01:23:54 +0000 (17:23 -0800)]
Cleanup ZEDLETs

This commit factors out several common ZEDLET code blocks into
zed-functions.sh.  This shortens the length of the scripts, thereby
(hopefully) making them easier to understand and maintain.

In addition, this commit revamps the coding style used by the
scripts to be more consistent and (again, hopefully) maintainable.
It now mostly follows the Google Shell Style Guide.  I've tried to
assimilate the following resources:

  Google Shell Style Guide
  https://google-styleguide.googlecode.com/svn/trunk/shell.xml

  Dash as /bin/sh
  https://wiki.ubuntu.com/DashAsBinSh

  Filenames and Pathnames in Shell: How to do it Correctly
  http://www.dwheeler.com/essays/filenames-in-shell.html

  Common shell script mistakes
  http://www.pixelbeat.org/programming/shell_script_mistakes.html

Finally, this commit updates the exit codes used by the ZEDLETs to be
more consistent with one another.

All scripts run cleanly through ShellCheck <http://www.shellcheck.net/>.
All scripts have been tested on bash and dash.

Signed-off-by: Chris Dunlap <cdunlap@llnl.gov>
9 years agoRemove useless variable spa_active_count
Isaac Huang [Sun, 26 Apr 2015 04:25:45 +0000 (22:25 -0600)]
Remove useless variable spa_active_count

This isn't required for the Linux port because the kernel tracks
if a module is busy.  The prototype for spa_busy() is also removed
since its definition was already removed.

Signed-off-by: Isaac Huang <he.huang@intel.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3262

9 years agoMemory leak in make_root_vdev()
Isaac Huang [Sun, 26 Apr 2015 04:08:29 +0000 (22:08 -0600)]
Memory leak in make_root_vdev()

The newroot nvlist should be freed before returning.

Signed-off-by: Isaac Huang <he.huang@intel.com>
Signed-off-by: Chunwei Chen <tuxoko@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3264

9 years ago5313 Allow I/Os to be aggregated across ZIO priority classes
Justin T. Gibbs [Sat, 11 Apr 2015 18:51:06 +0000 (14:51 -0400)]
5313 Allow I/Os to be aggregated across ZIO priority classes

Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Will Andrews <willa@SpectraLogic.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>

References:
  https://www.illumos.org/issues/5313
  https://github.com/illumos/illumos-gate/commit/fe319232

Ported-by: DHE <git@dehacked.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3280

9 years ago5410 Document -S option to zfs inherit
Paul B. Henson [Thu, 1 Jan 2015 01:44:52 +0000 (17:44 -0800)]
5410 Document -S option to zfs inherit

5410 Document -S option to zfs inherit
5412 Mention -S option when zfs inherit fails on quota
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/5410
  https://github.com/illumos/illumos-gate/commit/5ff8cfa9

Ported-by: DHE <git@dehacked.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3279

9 years agoSerialize access to spa->spa_feat_stats nvlist
Ned Bass [Thu, 23 Apr 2015 19:32:59 +0000 (12:32 -0700)]
Serialize access to spa->spa_feat_stats nvlist

The function spa_add_feature_stats() manipulates the shared nvlist
spa->spa_feat_stats in an unsafe concurrent manner. Add a mutex to
protect the list.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3335

9 years agoalign zfs_autoimport_disable manpage with reality
cburroughs [Fri, 24 Apr 2015 18:03:26 +0000 (14:03 -0400)]
align zfs_autoimport_disable manpage with reality

The default was changed in #2820.

Signed-off-by: cburroughs <chris.burroughs@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3341

9 years agoFix kernel panic due to tsd_exit in ZFS_EXIT(zsb)
Chunwei Chen [Wed, 1 Apr 2015 18:18:05 +0000 (02:18 +0800)]
Fix kernel panic due to tsd_exit in ZFS_EXIT(zsb)

The following panic would occur under certain heavy load:
[ 4692.202686] Kernel panic - not syncing: thread ffff8800c4f5dd60 terminating with rrw lock ffff8800da1b9c40 held
[ 4692.228053] CPU: 1 PID: 6250 Comm: mmap_deadlock Tainted: P           OE  3.18.10 #7

The culprit is that ZFS_EXIT(zsb) would call tsd_exit() every time, which
would purge all tsd data for the thread. However, ZFS_ENTER is designed to be
reentrant, so we cannot allow ZFS_EXIT to blindly purge tsd data.

Instead, we rely on the new behavior of tsd_set. When NULL is passed as the
new value to tsd_set, it will automatically remove the tsd entry specified the
the key for the current thread.

rrw_tsd_key and zfs_allow_log_key already calls tsd_set(key, NULL) when
they're done. The zfs_fsyncer_key relied on ZFS_EXIT(zsb) to call tsd_exit() to
do clean up. Now we explicitly call tsd_set(key, NULL) on them.

Signed-off-by: Chunwei Chen <tuxoko@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3247

9 years agoSupport the "version" property on volumes via the zfs_prop_get_int() API
Tim Chase [Sun, 19 Apr 2015 03:57:36 +0000 (23:57 -0400)]
Support the "version" property on volumes via the zfs_prop_get_int() API

As of this commit, volumes do not possess the version property in
any existing OpenZFS implementation.  The zpool upgrade code, however,
uses zfs_prop_get_int() to fetch the version property of all children in
a pool.  The semantics of the function, however, demand that it only be
used for known valid properties so it returns a garbage value for volumes.

This patch causes the version of a volume to appear to callers using
the zfs_prop_get_int() API to be that of the default ZPL version for
the implementation.  In the future, should volumes gain the property,
its actual value will be used.

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Fixes #3313

9 years agoExtend PF_FSTRANS critical regions
Brian Behlendorf [Thu, 23 Apr 2015 17:09:19 +0000 (10:09 -0700)]
Extend PF_FSTRANS critical regions

Additional testing has shown that the region covered by PF_FSTRANS
needs to be extended to cover the  zpl_xattr_security_init() and
init_acl() functions.  The zpl_mark_dirty() function can also recurse
and therefore must always be protected.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Chunwei Chen <tuxoko@gmail.com>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Closes #3331

9 years agoFix formatting error in zfs(8)
DHE [Sat, 18 Apr 2015 11:07:53 +0000 (07:07 -0400)]
Fix formatting error in zfs(8)

Commit b1a3e93217e6e474e86345010469994c066cf875 accidentally
introduced an intentation error between the 'zfs receive'
and 'zfs allow' detailed documentation sections.

Signed-off-by: DHE <git@dehacked.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3312

9 years agoFix io-spare.sh to work with disk vdevs
Chris Dunlap [Fri, 10 Apr 2015 19:56:21 +0000 (12:56 -0700)]
Fix io-spare.sh to work with disk vdevs

The "zpool status" output shows the full pathname for file-type vdevs,
but only the basename component for disk-type vdevs.  In commit
bee6665, the "basename" command was dropped from altering the vdev
name used when searching the "zpool status" output.  Consequently,
hot-disk sparing for disk vdevs broke since "zpool status" output
was now being searched for the full pathname to the disk vdev.

Parsing the "zpool status" output in this manner is rather brittle.
It would be preferable to search for the vdev based on its guid.
But until that happens, this commit adds back the "basename" command
to fix the vdev name breakage.

Signed-off-by: Chris Dunlap <cdunlap@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3310

9 years agoMark additional functions as PF_FSTRANS
Brian Behlendorf [Tue, 14 Apr 2015 17:25:50 +0000 (10:25 -0700)]
Mark additional functions as PF_FSTRANS

Prevent deadlocks by disabling direct reclaim during all NFS, xattr,
ctldir, and super function calls.  This is related to 40d06e3.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tim Chase <tim@chase2k.com>
Issue #3225

9 years agoDocument bookmarks a little better in zfs(8)
Turbo Fredriksson [Thu, 9 Apr 2015 04:18:01 +0000 (06:18 +0200)]
Document bookmarks a little better in zfs(8)

Add a basic summary to zfs(8) describing bookmarks.

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3268

9 years agoRebuild init scripts on source file updates
DHE [Fri, 10 Apr 2015 15:14:47 +0000 (11:14 -0400)]
Rebuild init scripts on source file updates

The resulting script is not removed by 'make clean' or rebuilt
when the source files are changed. Users with long standing git
trees may find their init script is out of date.

Signed-off-by: DHE <git@dehacked.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3273

9 years agoAllocate zfs_znode_cache on the Linux slab
Tim Chase [Tue, 14 Apr 2015 05:06:40 +0000 (01:06 -0400)]
Allocate zfs_znode_cache on the Linux slab

The Linux slab, in general, performs better than the SPl slab in cases
where a lot of objects are allocated and fragmentation is likely present.

This patch fixes pathologically bad behavior in cases where the ARC is
filled with mostly metadata and a user program needs to allocate and
dirty enough memory which would require an insignificant amount of the
ARC to be reclaimed.

If zfs_znode_cache is on the SPL slab, the system may spin for a very
long time trying to reclaim sufficient memory.  If it is on the Linux
slab, the behavior has been observed to be much more predictible; the
memory is reclaimed more efficiently.

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #3283

9 years agoTag zfs-0.6.4
Brian Behlendorf [Thu, 9 Apr 2015 03:16:45 +0000 (20:16 -0700)]
Tag zfs-0.6.4

META file and release log updated.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoUse vmem_alloc() in spa_config_write()
Brian Behlendorf [Fri, 3 Apr 2015 16:12:02 +0000 (09:12 -0700)]
Use vmem_alloc() in spa_config_write()

The packed nvlist allocated in spa_config_write() may exceed the
warning threshold for large configurations.  Use the vmem interfaces
for this short lived allocation.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3251

9 years agoMark all ZPL and ioctl functions as PF_FSTRANS
Tim Chase [Tue, 31 Mar 2015 03:43:29 +0000 (22:43 -0500)]
Mark all ZPL and ioctl functions as PF_FSTRANS

Prevent deadlocks by disabling direct reclaim during all ZPL and ioctl
calls as well as the l2arc and adapt ARC threads.

This obviates the need for MUTEX_FSTRANS so its previous uses and
definition have been eliminated.

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3225

9 years agoUpdate zfs_pd_bytes_max default in zfs(8)
Brian Behlendorf [Tue, 31 Mar 2015 18:51:37 +0000 (11:51 -0700)]
Update zfs_pd_bytes_max default in zfs(8)

Commit b738bc5 should have updated the default value of zfs_pd_bytes_max
in the zfs(8) man page.  The correct default value is 50*1024*1024.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoIllumus 5693 - ztest fails in dbuf_verify: buf[i] == 0, due to dedup and bp_override
Matthew Ahrens [Fri, 27 Mar 2015 06:11:50 +0000 (17:11 +1100)]
Illumus 5693 - ztest fails in dbuf_verify: buf[i] == 0, due to dedup and bp_override

5693 ztest fails in dbuf_verify: buf[i] == 0, due to dedup and bp_override
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>

References:
  https://www.illumos.org/issues/5693
  https://github.com/illumos/illumos-gate/commit/7f7ace3

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3231

9 years agoIllumos 5694 - traverse_prefetcher does not prefetch enough
George Wilson [Fri, 27 Mar 2015 04:31:52 +0000 (15:31 +1100)]
Illumos 5694 - traverse_prefetcher does not prefetch enough

5694 traverse_prefetcher does not prefetch enough
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Josef 'Jeff' Sipek <josef.sipek@nexenta.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
  https://www.illumos.org/issues/5694
  https://github.com/illumos/illumos-gate/commit/34d7ce05

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3230

9 years agoAlign code with Illumos
Chris Dunlop [Fri, 27 Mar 2015 04:04:12 +0000 (15:04 +1100)]
Align code with Illumos

Align code in traverse_visitbp() with that in Illumos in preparation for
applying Illumos-5694.

No functional change: use a temporary variable pd to replace multiple
occurrences of td->td_pfd.  This increases our stack use slightly more
then normal because the function is called recursively.

Signed-off-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #3230

9 years agoIllumos 5695 - dmu_sync'ed holes do not retain birth time
Prakash Surya [Fri, 27 Mar 2015 02:03:22 +0000 (13:03 +1100)]
Illumos 5695 - dmu_sync'ed holes do not retain birth time

5695 dmu_sync'ed holes do not retain birth time
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>

References:
  https://www.illumos.org/issues/5695
  https://github.com/illumos/illumos-gate/commit/70163ac

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3229

9 years agozpool import should honor overlay property
Ned Bass [Thu, 26 Mar 2015 19:10:26 +0000 (12:10 -0700)]
zpool import should honor overlay property

Make the 'zpool import' command honor the overlay property to allow
filesystems to be mounted on a non-empty directory. As it stands now
this property is only checked by the 'zfs mount' command.  Move the
check into 'zfs_mount()` in libzpool so the property is honored for all
callers.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3227

9 years agoAdd NULL guard in zfs_zrlock_class event class
Ned Bass [Wed, 25 Mar 2015 00:22:21 +0000 (17:22 -0700)]
Add NULL guard in zfs_zrlock_class event class

The owner field could be NULL in some cases, so add a guard.  Shorten
__entry field names to fit assignment statements in 80 columns.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Fixes #3220

9 years agoAdd RHEL style kmod packages
Brian Behlendorf [Wed, 25 Mar 2015 23:59:17 +0000 (16:59 -0700)]
Add RHEL style kmod packages

Provide a Redhat specific zfs-kmod.spec file which uses the old style
kmods (not kmods2) packaging.  By using the provided kmodtool script
packages can be built which support weak modules.  This allows for the
kernel to be updated without having to rebuild the ZFS kernel modules.

Packages for RHEL/Centos/SL/TOSS which use this spec file can by built
as follows:

$ ./configure --with-spec=redhat
$ make rpms

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoRemove rpm/fedora directory
Brian Behlendorf [Fri, 27 Mar 2015 21:30:23 +0000 (14:30 -0700)]
Remove rpm/fedora directory

Originally it was thought that custom spec files might be required
for Fedora.  Happily that has turns out not to be the case.  Since
this directory just contains symlinks to the generic spec files it
can be removed.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoCheck all vdev labels in 'zpool import'
Brian Behlendorf [Fri, 20 Mar 2015 22:10:24 +0000 (15:10 -0700)]
Check all vdev labels in 'zpool import'

When using 'zpool import' to scan for available pools prefer vdev names
which reference vdevs with more valid labels.  There should be two labels
at the start of the device and two labels at the end of the device.  If
labels are missing then the device has been damaged or is in some other
way incomplete.  Preferring names with fully intact labels helps weed out
bad paths and improves the likelihood of being able to import the pool.

This behavior only applies when scanning /dev/ for valid pools.  If a
cache file exists the pools described by the cache file will be used.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Chris Dunlap <cdunlap@llnl.gov>
Closes #3145
Closes #2844
Closes #3107

9 years agodbuf_free_range() overzealously frees dbufs
Ned Bass [Wed, 25 Mar 2015 00:00:08 +0000 (17:00 -0700)]
dbuf_free_range() overzealously frees dbufs

When called to free a spill block from a dnode, dbuf_free_range() has a
bug that results in all dbufs for the dnode getting freed.  A variety of
problems may result from this bug, but a common one was a zap lookup
tripping an ASSERT because the zap buffers had been zeroed out.  This
could happen on a dataset with xattr=sa set when extended attributes are
written and removed on a directory concurrently with I/O to files in
that directory.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Fixes #3195
Fixes #3204
Fixes #3222

9 years agoSet the maximum ZVOL transfer size correctly
Tim Chase [Mon, 23 Mar 2015 17:10:19 +0000 (12:10 -0500)]
Set the maximum ZVOL transfer size correctly

ZoL had been setting max_sectors to UINT_MAX, but until Linux 3.19, it
the kernel artifically capped it at 1024 (BLK_DEF_MAX_SECTORS).
This cap was removed in torvalds/linux@34b48db.  This patch changes
it to DMU_MAX_ACCESS (in sectors) and also changes the ASSERT in
dmu_tx_hold_write() to allow the maximum transfer size.

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3212

9 years agoExecute udevadm settle before trying to import pools
Gordan Bobic [Mon, 23 Mar 2015 16:17:56 +0000 (16:17 +0000)]
Execute udevadm settle before trying to import pools

Execute udevadm settle before trying to import pools.  Otherwise the
disk device nodes may not be ready before import time.  This is
analogous to the behavior of the init scripts and systemd units.

Signed-off-by: Gordan Bobic <gordan@steel.shatteredsilicon.net>
Signed-off-by: Pavel Snajdr <snajpa@snajpa.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3213

9 years agozio_injection_enabled should not be a module option
Isaac Huang [Sun, 22 Mar 2015 05:32:54 +0000 (23:32 -0600)]
zio_injection_enabled should not be a module option

The zio_inject.c keeps zio_injection_enabled as a counter of
fault handlers, so it should not be exported to user space as
a module option.

Several EXPORT_SYMBOLs are moved from zio.c to zio_inject.c,
where the symbols are defined.

Signed-off-by: Isaac Huang <he.huang@intel.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3199

9 years agoReduce size of zfs_sb_t: allocate z_hold_mtx separately
Chris Dunlop [Mon, 16 Mar 2015 01:21:21 +0000 (12:21 +1100)]
Reduce size of zfs_sb_t: allocate z_hold_mtx separately

zfs_sb_t has grown to the point where using kmem_zalloc() for allocations
is triggering the 32k warning threshold.

We can't safely convert this entire allocation to use vmem_alloc() instead
of kmem_alloc() because the backing_dev_info structure is embedded here.
It depends on the bit_waitqueue() function which won't behave properly
when given a virtual address.

Instead, use vmem_alloc() to allocate the z_hold_mtx array separately.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Chris Dunlop <chris@onthe.net.au>
Closes #3178

9 years agoFix arc_adjust_meta() behavior
Brian Behlendorf [Tue, 17 Mar 2015 22:08:22 +0000 (15:08 -0700)]
Fix arc_adjust_meta() behavior

The goal of this function is to evict enough meta data buffers from the
ARC in order to enforce the arc_meta_limit.  Achieving this is slightly
more complicated than it appears because it is common for data buffers
to have holds on meta data buffers.  In addition, dnode meta data buffers
will be held by the dnodes in the block preventing them from being freed.
This means we can't simply traverse the ARC and expect to always find
enough unheld meta data buffer to release.

Therefore, this function has been updated to make alternating passes
over the ARC releasing data buffers and then newly unheld meta data
buffers.  This ensures forward progress is maintained and arc_meta_used
will decrease.  Normally this is sufficient, but if required the ARC
will call the registered prune callbacks causing dentry and inodes to
be dropped from the VFS cache.  This will make dnode meta data buffers
available for reclaim.  The number of total restarts in limited by
zfs_arc_meta_adjust_restarts to prevent spinning in the rare case
where all meta data is pinned.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Pavel Snajdr <snajpa@snajpa.net>
Issue #3160

9 years agoRestructure per-filesystem reclaim
Brian Behlendorf [Tue, 17 Mar 2015 22:07:47 +0000 (15:07 -0700)]
Restructure per-filesystem reclaim

Originally when the ARC prune callback was introduced the idea was
to register a single callback for the ZPL.  The ARC could invoke this
call back if it needed the ZPL to drop dentries, inodes, or other
cache objects which might be pinning buffers in the ARC.  The ZPL
would iterate over all ZFS super blocks and perform the reclaim.

For the most part this design has worked well but due to limitations
in 2.6.35 and earlier kernels there were some problems.  This patch
is designed to address those issues.

1) iterate_supers_type() is not provided by all kernels which makes
it impossible to safely iterate over all zpl_fs_type filesystems in
a single callback.  The most straight forward and portable way to
resolve this is to register a callback per-filesystem during mount.
The arc_*_prune_callback() functions have always supported multiple
callbacks so this is functionally a very small change.

2) Commit 050d22b removed the non-portable shrink_dcache_memory()
and shrink_icache_memory() functions and didn't replace them with
equivalent functionality.  This meant that for Linux 3.1 and older
kernels the ARC had no mechanism to drop dentries and inodes from
the caches if needed.  This patch adds that missing functionality
by calling shrink_dcache_parent() to release dentries which may be
pinning inodes.  This will result in all unused cache entries being
dropped which is a bit heavy handed but it's the only interface
available for old kernels.

3) A zpl_drop_inode() callback is registered for kernels older than
2.6.35 which do not support the .evict_inode callback.  This ensures
that when the last reference on an inode is dropped it is immediately
removed from the cache.  If this isn't done than inode can end up on
the global unused LRU with no mechanism available to ZFS to drop them.
Since the ARC buffers are not dropped the hottest inodes can still
be recreated without performing disk IO.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Pavel Snajdr <snajpa@snajpa.net>
Issue #3160

9 years agoFix arc_meta_max accounting
Brian Behlendorf [Tue, 17 Mar 2015 22:04:27 +0000 (15:04 -0700)]
Fix arc_meta_max accounting

The arc_meta_max value should be increased when space it consumed not when
it is returned.  This ensure's that arc_meta_max is always up to date.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Pavel Snajdr <snajpa@snajpa.net>
Issue #3160

9 years agoIncrease Linux pipe buffer size on 'zfs receive'
Richard Yao [Wed, 11 Mar 2015 18:24:46 +0000 (14:24 -0400)]
Increase Linux pipe buffer size on 'zfs receive'

I noticed when reviewing documentation that it is possible for user
space to use fctnl(fd, F_SETPIPE_SZ, (unsigned long) size) to change
the kernel pipe buffer size on Linux to increase the pipe size up to
the value specified in /proc/sys/fs/pipe-max-size. There are users using
mbuffer to improve zfs recv performance when piping over the network, so
it seems advantageous to integrate such functionality directly into the
zfs recv tool. This avoids the addition of two buffers and two copies
(one for the buffer mbuffer adds and another for the additional pipe),
so it should be more efficient.

This could have been made configurable and/or this could have changed
the value back to the original after we were done with the file
descriptor, but I do not see a strong case for doing either, so I
went with a simple implementation.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1161

9 years agoMove duplicate information about the 'zfs send -e' option.
Turbo Fredriksson [Tue, 17 Mar 2015 22:29:27 +0000 (23:29 +0100)]
Move duplicate information about the 'zfs send -e' option.

The extra one was under the 'zfs receive' command (which isn't relevant).
Instead, it should have been further up (still in the 'zfs send' option).

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3194

9 years agoUse MUTEX_FSTRANS on l2arc_buflist_mtx
Chunwei Chen [Tue, 17 Mar 2015 06:54:57 +0000 (23:54 -0700)]
Use MUTEX_FSTRANS on l2arc_buflist_mtx

Use MUTEX_FSTRANS on l2arc_buflist_mtx to prevent the following deadlock
scenario:
1. arc_release() -> hash_lock -> l2arc_buflist_mtx
2. l2arc_write_buffers() -> l2arc_buflist_mtx -> (direct reclaim) ->
   arc_buf_remove_ref() -> hash_lock

Signed-off-by: Chunwei Chen <tuxoko@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Signed-off-by: Tim Chase <tim@chase2k.com>
Issue #3160

9 years agoFix warning about AM_INIT_AUTOMAKE arguments
Hajo Möller [Thu, 12 Mar 2015 18:09:19 +0000 (19:09 +0100)]
Fix warning about AM_INIT_AUTOMAKE arguments

As of automake 1.14.2, currently shipped with Ubuntu 14.04, automake
warns about AM_INIT_AUTOMAKE having more than one argument:

configure.ac:41: warning: AM_INIT_AUTOMAKE: two- and three-arguments forms are deprecated.  For more info, see:
configure.ac:41: http://www.gnu.org/software/automake/manual/automake.html#Modernize-AM_005fINIT_005fAUTOMAKE-invocation

This commit fixes the warnings by following above link's advice, so
AM_INIT gets called with the package's name and version. As both are
defined in the META file we're parsing it with `grep`, `cut` and `tr`.

NOTE: autoconf < 1.14 not supporting m4_esyscmd_s so m4_esyscmd was
used and modified `tr` to truncate newlines, too.

Signed-off-by: Hajo M<C3><B6>ller <dasjoe@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3174

9 years agoLinux 4.0 compat: bdi_setup_and_register() __must_check
Bill McGonigle [Fri, 13 Mar 2015 16:44:42 +0000 (09:44 -0700)]
Linux 4.0 compat: bdi_setup_and_register() __must_check

Explicitly disable the unused by variable warnings by setting
__attribute__((unused)) for bdi_setup_and_register().  This is
required because the function is defined with the __must_check
attribute.

Signed-off-by: Bill McGonigle <bill-github.com-public1@bfccomputing.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3141

9 years agoIllumos 5630 - stale bonus buffer in recycled dnode_t leads to data corruption
Justin T. Gibbs [Thu, 12 Mar 2015 00:10:35 +0000 (11:10 +1100)]
Illumos 5630 - stale bonus buffer in recycled dnode_t leads to data corruption

5630 stale bonus buffer in recycled dnode_t leads to data corruption
Author: Justin T. Gibbs <justing@spectralogic.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Will Andrews <will@freebsd.org>
Approved by: Robert Mustacchi <rm@joyent.com>

References:
  https://www.illumos.org/issues/5630
  https://github.com/illumos/illumos-gate/commit/cd485b4

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Issue #3172

9 years agoIllumos 5047 - don't use atomic_*_nv if you discard the return value
Josef 'Jeff' Sipek [Thu, 12 Mar 2015 00:03:31 +0000 (11:03 +1100)]
Illumos 5047 - don't use atomic_*_nv if you discard the return value

5047 don't use atomic_*_nv if you discard the return value
Author: Josef 'Jeff' Sipek <josef.sipek@nexenta.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Reviewed by: Jason King <jason.brian.king@gmail.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>

References:
  https://www.illumos.org/issues/5047
  https://github.com/illumos/illumos-gate/commit/640c167

Porting Notes:

Several hunks from the original patch where not specific to ZFS
and thus were dropped.

Ported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Issue #3172

9 years agoMark zfs_inactive() with PF_FSTRANS
Brian Behlendorf [Mon, 9 Mar 2015 20:57:22 +0000 (13:57 -0700)]
Mark zfs_inactive() with PF_FSTRANS

Allowing direct reclaim to re-enter the VFS in the zfs_inactive()
call path has historically been problematic for ZoL.  Therefore,
in order to avoid an entire class of current and future issues
caused by this PF_FSTRANS is set for all zfs_inactive() callers.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3163

9 years agoActually source /etc/sysconfig/zfs instead of /etc/default/zfs
Hajo Möller [Mon, 9 Mar 2015 20:51:31 +0000 (21:51 +0100)]
Actually source /etc/sysconfig/zfs instead of /etc/default/zfs

Signed-off-by: Hajo M<C3><B6>ller <dasjoe@users.noreply.github.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3162

9 years agoUse cached feature info in spa_add_feature_stats()
Ned Bass [Thu, 26 Feb 2015 20:24:11 +0000 (12:24 -0800)]
Use cached feature info in spa_add_feature_stats()

Avoid issuing I/O to the pool when retrieving feature flags information.
Trying to read the ZAPs from disk means that zpool clear would hang if
the pool is suspended and recovery would require a reboot. To keep the
feature stats resident in memory, we hang a cached nvlist off of the
spa.  It is built up from disk the first time spa_add_feature_stats() is
called, and refreshed thereafter using the cached feature reference
counts. spa_add_feature_stats() gets called at pool import time so we
can be sure the cached nvlist will be available if the pool is later
suspended.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3082

9 years agoAdd ZED to zfs.redhat.in script
Chris Dunlap [Thu, 5 Mar 2015 21:11:05 +0000 (13:11 -0800)]
Add ZED to zfs.redhat.in script

This commit updates the zfs.redhat.in script to start/stop ZED.

Signed-off-by: Chris Dunlap <cdunlap@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #3153

9 years agoReplace zfs.redhat.in with zfs.lsb.in init script
Brian Behlendorf [Wed, 4 Mar 2015 19:09:17 +0000 (11:09 -0800)]
Replace zfs.redhat.in with zfs.lsb.in init script

This commit replaces the zfs.redhat.in init script with a slightly
modified version of the existing zfs.lsb.in init script.  This was
done to minimize the functional differences between platforms.
The lsb version of the script was choosen because it's heavily
tested and provides the most functionality.

Changes made for RHEL systems:
* Configuration: /etc/default/zfs -> /etc/sysconfig/zfs
* LSB functions: /lib/lsb/init-functions -> /etc/rc.d/init.d/functions
* Logging: log_begin_msg/log_end_msg -> action

Features in LSB which are now in RHEL:
* USE_DISK_BY_ID=0      - Use the by-id names
* VERBOSE_MOUNT=0       - Verbose mounts by default
* DO_OVERLAY_MOUNTS=0   - Overlay mounts by default
* MOUNT_EXTRA_OPTIONS=0 - Generic extra options

Existing RHEL features which were removed:
* Automatically mounting FSs on ZVOLs listed in /etc/fstab

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #3153

9 years agoInstall arc_summary.py
Turbo Fredriksson [Mon, 2 Mar 2015 21:28:57 +0000 (22:28 +0100)]
Install arc_summary.py

Add the arc_summary Makefile to the build system so the script is
properly included in the distribution tarball and installed.

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3147

9 years agoChange ASSERT(!"...") to cmn_err(CE_PANIC, ...)
Brian Behlendorf [Fri, 27 Feb 2015 20:53:35 +0000 (12:53 -0800)]
Change ASSERT(!"...") to  cmn_err(CE_PANIC, ...)

There are a handful of ASSERT(!"...")'s throughout the code base for
cases which should be impossible.  This patch converts them to use
cmn_err(CE_PANIC, ...) to ensure they are always enabled and so that
additional debugging is logged if they were to occur.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1445

9 years agoLinux 4.0 compat: bdi_setup_and_register()
Brian Behlendorf [Sat, 28 Feb 2015 00:09:52 +0000 (16:09 -0800)]
Linux 4.0 compat: bdi_setup_and_register()

The 'capabilities' argument which was passed to bdi_setup_and_register()
has been removed.  File systems should no longer pass BDI_CAP_MAP_COPY.
For our purposes this means there are now three different interfaces
which must be handled.  A zpl_bdi_setup_and_register() wrapper function
has been introduced to provide a single interface to the ZPL code.

* 2.6.32 - 2.6.33, bdi_setup_and_register() is not exported.
* 2.6.34 - 3.19, bdi_setup_and_register() takes 3 arguments.
* 4.0 - x.y, bdi_setup_and_register() takes 2 arguments.

I've also taken this opportunity to remove HAVE_BDI because kernels
older then 2.6.32 are no longer supported.  All kernels newer than
this will have one of the above interfaces.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Chunwei Chen <tuxoko@gmail.com>
Closes #3128

9 years agoUse MUTEX_FSTRANS mutex type
Brian Behlendorf [Thu, 26 Feb 2015 23:29:33 +0000 (15:29 -0800)]
Use MUTEX_FSTRANS mutex type

There are regions in the ZFS code where it is desirable to be able
to be set PF_FSTRANS while a specific mutex is held.  The ZFS code
could be updated to set/clear this flag in all the correct places,
but this is undesirable for a few reasons.

1) It would require changes to a significant amount of the ZFS
   code.  This would complicate applying patches from upstream.

2) It would be easy to accidentally miss a critical region in
   the initial patch or to have an future change introduce a
   new one.

Both of these concerns can be addressed by using a new mutex type
which is responsible for managing PF_FSTRANS, support for which was
added to the SPL in commit zfsonlinux/spl@9099312 - Merge branch
'kmem-rework'.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tim Chase <tim@chase2k.com>
Closes #3050
Closes #3055
Closes #3062
Closes #3132
Closes #3142
Closes #2983

9 years agoFix deadlock between zpool export and zfs list
Isaac Huang [Fri, 27 Feb 2015 05:46:45 +0000 (22:46 -0700)]
Fix deadlock between zpool export and zfs list

Pool reference count is NOT checked in spa_export_common()
if the pool has been imported readonly=on, i.e. spa->spa_sync_on
is FALSE. Then zpool export and zfs list may deadlock:

1. Pool A is imported readonly.
2. zpool export A and zfs list are run concurrently.
3. zfs command gets reference on the spa, which holds a dbuf on
   on the MOS meta dnode.
4. zpool command grabs spa_namespace_lock, and tries to evict dbufs
   of the MOS meta dnode. The dbuf held by zfs command can't be
   evicted as its reference count is not 0.
5. zpool command blocks in dnode_special_close() waiting for the
   MOS meta dnode reference count to drop to 0, with
   spa_namespace_lock held.
6. zfs command tries to get the spa_namespace_lock with a reference
   on the spa held, which holds a dbuf on the MOS meta dnode.
7. Now zpool command and zfs command deadlock each other.

Also any further zfs/zpool command will block on spa_namespace_lock
forever.

The fix is to always check pool reference count in spa_export_common(),
no matter whether the pool was imported readonly or not.

Signed-off-by: Isaac Huang <he.huang@intel.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2034

9 years agoPrevent "zpool destroy|export" when suspended
Brian Behlendorf [Fri, 27 Feb 2015 22:35:56 +0000 (14:35 -0800)]
Prevent "zpool destroy|export" when suspended

Cleanly destroying or exporting a pool requires that the pool
not be suspended.  Therefore, set the POOL_CHECK_SUSPENDED flag
for these ioctls so the utilities will output a descriptive
error message rather than block.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #2878

9 years agoAvoid dladdr() in ztest
Tim Chase [Tue, 24 Feb 2015 18:53:31 +0000 (12:53 -0600)]
Avoid dladdr() in ztest

Under Linux, at least, dladdr() doesn't reliably work for functions which
aren't in a DSO.  Add the function name to ztest_info[].

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3130

9 years agoFix possible future overflow in zfs_nicenum
Christer Ekholm [Thu, 19 Feb 2015 21:44:53 +0000 (22:44 +0100)]
Fix possible future overflow in zfs_nicenum

The function zfs_nicenum that converts number to human-readable output
uses a index to a string of letters. This patch limits the index to
the length of the string.

Signed-off-by: Christer Ekholm <che@chrekh.se>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3122

9 years agoRetire spl_module_init()/spl_module_fini()
Brian Behlendorf [Wed, 18 Feb 2015 23:39:05 +0000 (15:39 -0800)]
Retire spl_module_init()/spl_module_fini()

In the original implementation of the SPL wrappers were provided
for module initialization and cleanup.  This was done to abstract
away any compatibility code which might be needed for the SPL.

As it turned out the only significant compatibility issue was that
the default pwd during module load differed under Illumos and Linux.
Since this is such as minor thing and the wrappers complicate the
code they are being retired.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2985

9 years agoFix O_APPEND open(2) flag
Brian Behlendorf [Fri, 20 Feb 2015 18:28:25 +0000 (10:28 -0800)]
Fix O_APPEND open(2) flag

As described in flags section of open(2):

  O_APPEND:
    The  file  is  opened in append mode.  Before each write(2), the
    file offset is positioned at the end of the  file,  as  if  with
    lseek(2).   O_APPEND may lead to corrupted files on NFS filesys-
    tems if more than one process appends data to a  file  at  once.
    This is because NFS does not support appending to a file, so the
    client kernel has to simulate it, which can't be done without  a
    race condition.

This issue was originally overlooked because normally the generic
VFS code handles this for a filesystem.  However, because ZFS explictly
registers a zpl_write() function it's responsible for the seek.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3124

9 years agoFix error in dracut script if not using ZFS root
Steffen Müthing [Mon, 16 Feb 2015 03:13:26 +0000 (04:13 +0100)]
Fix error in dracut script if not using ZFS root

If we are not booting from ZFS, parse-zfs.sh fails because of
an unset variable.

Signed-off-by: Steffen M<C3><BC>thing <steffen.muething@iwr.uni-heidelberg.de>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #3110

9 years agoAdd required files to initramfs
Steffen Müthing [Mon, 16 Feb 2015 03:08:04 +0000 (04:08 +0100)]
Add required files to initramfs

The dracut module installs the udev rules and the vdev_id utility for creating
the /dev/disk/by-vdev/ names, but omits some additional utilities and the
config file required by vdev_id.

Signed-off-by: Steffen M<C3><BC>thing <steffen.muething@iwr.uni-heidelberg.de>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #3110

9 years agoSet zfs_autoimport_disable default value to 1
Dan Swartzendruber [Thu, 9 Oct 2014 20:45:46 +0000 (16:45 -0400)]
Set zfs_autoimport_disable default value to 1

When loading the ZFS kernel modules they should not populate the
spa namespace using the cache file.  This behavior isn't consistent
with other Linux kernel modules and we need to move away from it.
Removing this makes the whole startup process predictable with four
basic steps which are driven by the init system.

1) modprobe
2) zpool import
3) zfs mount
4) zfs share

This change also helps lay the groundwork for eventually removing
the kobj_* compatibility code on the kernel side.  It may need to
be preserved in userspace because libzfs_init() depends on it.
This is why the conditional must be wrapped with an #ifdef _KERNEL.

Signed-off-by: Dan Swartzendruber <dswartz@druber.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2820

9 years agoSkip bad DVAs during free by setting zfs_recover=1
Brian Behlendorf [Thu, 12 Feb 2015 23:05:21 +0000 (15:05 -0800)]
Skip bad DVAs during free by setting zfs_recover=1

When a bad DVA is encountered in metaslab_free_dva() the system
should treat it as fatal.  This indicates that somehow a damaged
DVA was written to disk and that should be impossible.

However, we have seen a handful of reports over the years of pools
somehow being damaged in this way.  Since this damage can render
otherwise intact pools unimportable, and the consequence of skipping
the bad DVA is only leaked free space, it makes sense to provide
a mechanism to ignore the bad DVA.  Setting the zfs_recover=1 module
option will cause the DVA to be ignored which may allow the pool to
be imported.

Since zfs_recover=0 by default any pool attempting to free a bad DVA
will treat it as a fatal error preserving the current behavior.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3099
Issue #3090
Issue #2720

9 years agoWrite directly to $initdir
Sören Tempel [Tue, 10 Feb 2015 20:12:27 +0000 (21:12 +0100)]
Write directly to $initdir

Simplify install() by removing the need for a temp file.

Signed-off-by: Soeren Tempel <soeren+git@soeren-tempel.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #3093

9 years agoUse test(1) in a proper way
Sören Tempel [Tue, 10 Feb 2015 19:12:13 +0000 (20:12 +0100)]
Use test(1) in a proper way

Use the correct operators to check the expected data type.

Signed-off-by: Soeren Tempel <soeren+git@soeren-tempel.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #3093

9 years agoEnhancements to zpool dry run mode.
Tim Chase [Tue, 22 Oct 2013 21:51:17 +0000 (16:51 -0500)]
Enhancements to zpool dry run mode.

In dry run mode, zpool should display more of the proposed pool
configuration for "zpool add".  This commit adds support for displaying
cache devices.

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1106

9 years agoChange VERIFY to ASSERT in mutex_destroy()
Brian Behlendorf [Thu, 30 Oct 2014 18:11:00 +0000 (11:11 -0700)]
Change VERIFY to ASSERT in mutex_destroy()

There have been multiple reports of 'zdb' tripping the VERIFY in
mutex_destroy() because pthread_mutex_destroy() returns EBUSY.

Exactly how this can happen still needs to be explained, but this
doesn't strictly need to be fatal for non-debug builds.  Therefore,
this patch converts the VERIFY to an ASSERT until the root cause
is determined and resolved.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #2027

9 years agoFix readdir for .zfs/snapshot directory
Andrey Vesnovaty [Tue, 8 Oct 2013 08:28:40 +0000 (10:28 +0200)]
Fix readdir for .zfs/snapshot directory

dmu_snapshot_list_next stores the index of the next snapshot entry to the offp
argument, which zpl_snapdir_iterate then uses for the dir_emit. This
result in an off-by-one error. Therefore a temporary variable should be
used.

This was a regression introduced in commit zfsonlinux/zfs@0f37d0c.

Signed-off-by: Andrey Vesnovaty <andrey.vesnovaty@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2930

9 years agoRetire zio_cons()/zio_dest()
Brian Behlendorf [Fri, 30 Jan 2015 19:25:19 +0000 (11:25 -0800)]
Retire zio_cons()/zio_dest()

The zio_cons() constructor and zio_dest() destructor don't exist
in the upstream Illumos code.  They were introduced as a workaround
to avoid issue #2523.  Since this issue has now been resolved this
code is being reverted to bring ZoL back in sync with Illumos.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ned Bass <bass6@llnl.gov>
Issue #3063

9 years agoRetire zio_bulk_flags
Brian Behlendorf [Fri, 6 Feb 2015 21:37:02 +0000 (13:37 -0800)]
Retire zio_bulk_flags

Long ago the zio_bulk_flags module parameter was introduced to
facilitate debugging and profiling the zio_buf_caches.  Today
this code works well and there's no compelling reason to keep
this functionality.  In fact it's preferable to revert this so
the code is more consistent with other ZFS implementations.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ned Bass <bass6@llnl.gov>
Issue #3063

9 years agoLinux 3.19 compat: file_inode was added
Jörg Thalheim [Sat, 7 Feb 2015 12:41:01 +0000 (13:41 +0100)]
Linux 3.19 compat: file_inode was added

struct access f->f_dentry->d_inode was replaced by accessor function
file_inode(f)

Signed-off-by: Joerg Thalheim <joerg@higgsboson.tk>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3084

9 years agoUse vmem_alloc() for nvlists
Brian Behlendorf [Thu, 5 Feb 2015 20:43:37 +0000 (12:43 -0800)]
Use vmem_alloc() for nvlists

Several of the nvlist functions may perform allocations larger than
the 32k warning threshold.  Convert them to use vmem_alloc() so the
best allocator is used.

Commit efcd79a retired KM_NODEBUG which was used to suppress large
allocation warnings.  Concurrently the large allocation warning threshold
was increased from 8k to 32k.  The goal was to identify the remaining
locations, such as this one, where the allocation can be larger than
32k.  This patch is expected fine tuning resulting for the kmem-rework
changes, see commit 6e9710f.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3057
Closes #3079
Closes #3081

9 years agoRevert "Don't read space maps during import for readonly pools"
Brian Behlendorf [Tue, 10 Feb 2015 00:53:42 +0000 (16:53 -0800)]
Revert "Don't read space maps during import for readonly pools"

This reverts commit 7fc8c33ede10f7104ca0e91d690d3ebb5236887b which
accidentally introduced a ztest failure.

ztest: '/usr/sbin/zdb -bcc -d -U /var/tmp/zpool.cache ztest' exit code 2
child exited with code 3

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
9 years agoProduce a full snapshot list for zfs send -p
Tim Chase [Mon, 17 Nov 2014 04:27:58 +0000 (22:27 -0600)]
Produce a full snapshot list for zfs send -p

In order to accelerate zfs receive operations in the face of many
property-containing snapshots, commit 0574855 changed the header nvlist
("fss") of a send stream to exclude snapshots which aren't part of the
stream.  This, however, would cause zfs receive -F to erroneously remove
snapshots; it would remove any snapshot which wasn't listed in the header
nvlist.

This patch restores the full list of snapshots in fss[<id>[snaps]] but
still suppresses the properties of non-sent snapshots and also removes a
consistency check in which an error is raised if a listed snapshot does
not have any properties in fss[<id>[snapprops]].

The 0574855 commit also introduced a bug in which zfs send -p of a
complete stream (zfs send -p pool/fs@snap) would exclude the snapshot
properties in fss[<id>[snapprops]].  This patch detects the last snapshot
in a series when no "from" snapshot has been specified and includes its
properties.

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2907

9 years agoDon't read space maps during import for readonly pools
Brian Behlendorf [Tue, 18 Nov 2014 22:29:04 +0000 (17:29 -0500)]
Don't read space maps during import for readonly pools

Normally when importing a pool the space maps for all top level
vdevs are read from disk.  The space maps will be required latter
when an allocation is performed and free blocks need to be located.

However, if the pool is imported readonly then we are guaranteed
that no allocations can occur.  In this case the space maps need
not be loaded..   A similar argument can be made for the DTLs
(dirty time logs).

Because a pool import will fail if the space maps cannot be read.
The ability to safely ignore them makes it more likely that a
damaged pool can be imported readonly to recover its contents.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #2831

9 years agoFix Dracut scripts to allow for blanks in pool and dataset names
Lukas Wunner [Sat, 7 Feb 2015 10:34:22 +0000 (11:34 +0100)]
Fix Dracut scripts to allow for blanks in pool and dataset names

The ability to use blanks is documented in zpool(8) and implemented
in module/zcommon/zfs_namecheck.c:valid_char().

Signed-off-by: Lukas Wunner <lukas@wunner.de>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3083

9 years agoFix loop in Dracut shutdown script
Lukas Wunner [Wed, 4 Feb 2015 09:45:19 +0000 (10:45 +0100)]
Fix loop in Dracut shutdown script

The shell executes each command of a pipeline in a subshell,
thus $ret always had the same value after the while loop that
it had before the loop (http://mywiki.wooledge.org/BashFAQ/024),
signaling success even if some of the zpools could not be exported.

Signed-off-by: Lukas Wunner <lukas@wunner.de>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3083

9 years agoIllumos 5311 - traverse_dnode may report success when it should not
Justin T. Gibbs [Sun, 23 Nov 2014 03:14:24 +0000 (19:14 -0800)]
Illumos 5311 - traverse_dnode may report success when it should not

5311 traverse_dnode may report success when it should not
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Will Andrews <willa@spectralogic.com>
Approved by: Dan McDonald <danmcd@omniti.com>

References:
  https://github.com/illumos/illumos-gate/commit/2a89c2c
  https://www.illumos.org/issues/5311

Ported by: DHE <git@dehacked.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2970

9 years agoFix SA header size accounting
Ned Bass [Wed, 4 Feb 2015 21:57:50 +0000 (13:57 -0800)]
Fix SA header size accounting

The functions sa_find_sizes() and sa_build_layout() fail to account
for the additional 2 bytes of SA header space when calculating whether
a variable size attribute might spill over. They may consequently
determine that an attribute will fit in the bonus buffer along with a
spill block pointer, when in reality the attribute would be partially
overwritten by the spill block pointer if spill over occurs. This also
causes an inconsistency between the SA header size and the number of
variable size attributes in the layout, tripping an assertion when
debugging is on. The following reproducer demonstrates the problem.

  ln -s $(perl -e 'print "z" x 20') file
  setfattr -h -n trusted.foo -v $(perl -e 'print "z" x 200') file

Even though sa_find_sizes() computes the index of the attribute where
spill-over will occur, sa_build_layouts() discards the result and
recomputes it itself. As it turns out, both functions get it wrong.
Since this computation is awkward and, as history has shown, easy to
screw up, let's just do it in one place. This patch fixes the bug in
sa_find_sizes() and updates sa_build_layout() to use the result
computed there.

Also improve the comments in sa_find_sizes().

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tim Chase <tim@chase2k.com>
Closes #3070

9 years agoSkip evicting dbufs when walking the dbuf hash
Brian Behlendorf [Fri, 6 Feb 2015 01:00:05 +0000 (17:00 -0800)]
Skip evicting dbufs when walking the dbuf hash

When a dbuf is in the DB_EVICTING state it may no longer be on the
dn_dbufs list.  In which case it's unsafe to call DB_DNODE_ENTER.
Therefore, any dbuf which is found in this safe must be skipped.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2553
Closes #2495

9 years agoFix build error when make deb
Chunwei Chen [Fri, 6 Feb 2015 07:22:10 +0000 (15:22 +0800)]
Fix build error when make deb

After 53698a4, the following error occurs when make deb.

  CCLD     zed
../../lib/libzfs/.libs/libzfs.so: undefined reference to `get_system_hostid'

Add libzpool.la to zed/Makefile.am to fix this

Signed-off-by: Chunwei Chen <tuxoko@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3080

9 years agoRead spl_hostid module parameter before gethostid()
Chunwei Chen [Fri, 23 Jan 2015 08:05:04 +0000 (16:05 +0800)]
Read spl_hostid module parameter before gethostid()

If spl_hostid is set via module parameter, it's likely different from
gethostid(). Therefore, the userspace tool should read it first before
falling back to gethostid().

Signed-off-by: Chunwei Chen <tuxoko@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3034

9 years agoSpurious ENOMEM returns when reading dbufs kstat
Tim Chase [Tue, 3 Feb 2015 05:55:20 +0000 (23:55 -0600)]
Spurious ENOMEM returns when reading dbufs kstat

Commit 7b2d78a046aa4695d434478a439a9438521d73af fixed some improper uses
of snprintf(), however, in __dbuf_stats_hash_table_data() the return
value of snprintf is propagated to the caller.  This caused spurious
ENOMEM errors when reading the dbufs kstat.

This commit causes the actual number of characters written to be returned.

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3072

9 years agofix l2arc compression buffers leak
avg [Thu, 6 Nov 2014 11:08:02 +0000 (11:08 +0000)]
fix l2arc compression buffers leak

Commit log from FreeBSD:

    We have observed that arc_release() can be called concurrently with a
    l2arc in-flight write.  Also, we have observed that arc_hdr_destroy()
    can be called from arc_write_done() for a zio with ZIO_FLAG_IO_REWRITE
    flag in similar circumstances.

    Previously the l2arc headers would be freed while leaking their
    associated compression buffers.  Now the buffers are placed on
    l2arc_free_on_write list for delayed freeing.  This is similar to
    what was already done to arc buffers that were supposed to be freed
    concurrently with in-flight writes of those buffers.

    In addition to fixing the discovered leaks this change also adds
    some protective code to assert that a compression buffer associated
    with a l2arc header is never leaked.

    A new kstat l2_cdata_free_on_write is added.  It keeps a count
    of delayed compression buffer frees which previously would have
    been leaks.

    Tested by:  Vitalij Satanivskij <satan@ukr.net> et al
    Requested by:       many
    MFC after:  2 weeks
    Sponsored by:       HybridCluster / ClusterHQ

References:
    https://illumos.org/issues/5222
    https://github.com/freebsd/freebsd/commit/b98f85d
    http://thread.gmane.org/gmane.os.freebsd.current/155757/focus=155781
    http://lists.open-zfs.org/pipermail/developer/2014-January/000455.html
    http://lists.open-zfs.org/pipermail/developer/2014-February/000523.html

Ported-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3029

9 years agoUse zio buffers in zil_itx_create()
Brian Behlendorf [Thu, 29 Jan 2015 23:09:51 +0000 (15:09 -0800)]
Use zio buffers in zil_itx_create()

The zil_itx_create() function uses the vmem_alloc() allocator for
its buffers because when logging a write that buffer may be as large
as 64K.  This is non-optimal because we may need to allocate many of
of these buffers and this interface has the potential to be slow.
Instead, use zio_data_buf_alloc() which is specifically designed to
be able to efficiently allocate a wide range of buffer sizes.

In addition, do some cleanup and use the zil_itx_destroy() function
to always free an itx structure.  This way we're always sure the
right allocation functions are used.  Notice that in the current
code kmem_free() and vmem_free() were both used.  This happened to
work because these wrappers map to the same internal SPL function.

This was identified as a potential problem when a low-end memory
constrained system began logging the following warnings.  There
was no deadlock here just repeated allocation failures resulting
in increased latency.

Possible memory allocation deadlock: size=65792 lflags=0x42d0
Pid: 20118, comm: kvm Tainted: P           O 3.2.0-0.bpo.4-amd64
Call Trace:
 [<ffffffffa040b834>] ? spl_kmem_alloc_impl+0x115/0x127 [spl]
 [<ffffffffa040b84f>] ? spl_kmem_alloc_debug+0x9/0x36 [spl]
 [<ffffffffa05d8a0b>] ? zil_itx_create+0x2d/0x59 [zfs]
 [<ffffffffa05c71e6>] ? zfs_log_write+0x13a/0x2f0 [zfs]
 [<ffffffffa05d41bc>] ? zfs_write+0x85b/0x9bb [zfs]
 [<ffffffffa05e37ec>] ? zpl_aio_write+0xca/0x110 [zfs]
 [<ffffffff811088e5>] ? do_sync_readv_writev+0xa3/0xde
 [<ffffffff81108f41>] ? do_readv_writev+0xaf/0x125
 [<ffffffff81109055>] ? sys_pwritev+0x55/0x9a
 [<ffffffff813721d2>] ? system_call_fastpath+0x16/0x1b

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Closes #3059