]> git.proxmox.com Git - mirror_zfs.git/log
mirror_zfs.git
2 years agoExtract workflows dependencies
Damian Szuberski [Mon, 7 Feb 2022 19:44:17 +0000 (20:44 +0100)]
Extract workflows dependencies

- Move build dependencies moved to
  `.github/workflows/build-dependencies.txt` shared among workflows.

- Change `ubuntu-latest` -> `ubuntu-20.04` to avoid unexpected
  runner environment updates in `zloop` workflow.

- Change `ubuntu-20.04` -> `ubuntu-latest` to track changes in
  runner environment in `checkstyle` workflow.

- Kernel buffer is flushed before ZTS invocation to avoid storing
  the same data after each test case run.

- `make` is invoked with consistent set of options to reduce
  clutter in logs.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes #13037

2 years agozvol: make calls to platform ops static
Christian Schwarz [Mon, 7 Feb 2022 18:24:38 +0000 (19:24 +0100)]
zvol: make calls to platform ops static

There's no need to make the platform ops dynamic dispatch.

This change replaces the dynamic dispatch with static calls to the
platform-specific functions.
To avoid name collisions, prefix all platform-specific functions
with `zvol_os_`.
I actually find `zvol_..._os` slightly nicer to read in the calling
code, but having it as a prefix is useful.

Advantage:
- easier jump-to-definition / grepping
- potential benefits to static analysis
- better legibility

Future work: also prefix remaining `static` functions in zvol_os.c.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Christian Schwarz <christian.schwarz@nutanix.com>
Closes #12965

2 years agoAdd more control/visibility to spa_load_verify().
Alexander Motin [Fri, 4 Feb 2022 21:06:38 +0000 (16:06 -0500)]
Add more control/visibility to spa_load_verify().

Use error thresholds from policy to control whether to scrub data
and/or metadata.  If threshold is set to UINT64_MAX, then caller
probably does not care about result and we may skip that part.

By default import neither set the data error threshold nor read
the error counter, so skip the data scrub for faster import.
Metadata are still scrubbed and fail if even single error found.

While there just for symmetry return number of metadata errors in
case threshold is not set to zero and we haven't reached it.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Pavel Zakharov <pavel.zakharov@delphix.com>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Closes #13022

2 years agozfs_set_prop_nvlist: make it easier to spot the call to dsl_props_set
Christian Schwarz [Fri, 4 Feb 2022 19:52:10 +0000 (20:52 +0100)]
zfs_set_prop_nvlist: make it easier to spot the call to dsl_props_set

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Christian Schwarz <christian.schwarz@nutanix.com>
Closes #12963

2 years agodsl_dir_tempreserve_impl: remove unused `deferred` variable
Christian Schwarz [Fri, 4 Feb 2022 18:33:34 +0000 (19:33 +0100)]
dsl_dir_tempreserve_impl: remove unused `deferred` variable

The following commit moved the users of `deferred` into function
dsl_pool_unreserved_space:

    commit d2734cce68cf740e015312314415f9034c67851c
    Author: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
    Date:   Fri Dec 16 14:11:29 2016 -0800

        OpenZFS 9166 - zfs storage pool checkpoint

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <freqlabs@FreeBSD.org>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Christian Schwarz <christian.schwarz@nutanix.com>
Closes #13056

2 years agoZTS: Update enospc_002_pos test case
Brian Behlendorf [Fri, 4 Feb 2022 17:36:46 +0000 (09:36 -0800)]
ZTS: Update enospc_002_pos test case

The on-disk cost of creating a snapshot or bookmark is sufficiently low
that it is difficult to make it reliably fail even when the pool is
"full".  In order to avoid false positives remove these two checks from
the test case.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #13060

2 years agoFix clearing set-uid and set-gid bits on a file when replying a write
Pawel Jakub Dawidek [Thu, 3 Feb 2022 22:37:57 +0000 (14:37 -0800)]
Fix clearing set-uid and set-gid bits on a file when replying a write

POSIX requires that set-uid and set-gid bits to be removed when an
unprivileged user writes to a file and ZFS does that during normal
operation.

The problem arrises when the write is stored in the ZIL and replayed.
During replay we have no access to original credentials of the process
doing the write, so zfs_write() will be performed with the root
credentials. When root is doing the write set-uid and set-gid bits
are not removed from the file.

To correct that, log a separate TX_SETATTR entry that removed those bits
on first write to such file.

Idea from: Christian Schwarz

Add test for ZIL replay of setuid/setgid clearing.

Improve various edge cases when clearing setid bits:
- The setid bits can be readded during a single write, so make sure to check
  for them on every chunk write.
- Log TX_SETATTR record at most once per transaction group (if the setid bits
  are keep coming back).
- Move zfs_log_setattr() outside of zp->z_acl_lock.

Reviewed-by: Dan McDonald <danmcd@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Co-authored-by: Christian Schwarz <me@cschwarz.com>
Signed-off-by: Pawel Jakub Dawidek <pawel@dawidek.net>
Closes #13027

2 years agoAdd `--enable-asan` and `--enable-ubsan` switches
Damian Szuberski [Thu, 3 Feb 2022 22:35:38 +0000 (23:35 +0100)]
Add `--enable-asan` and `--enable-ubsan` switches

`configure` now accepts `--enable-asan` and `--enable-ubsan` switches
which results in passing `-fsanitize=address`
and `-fsanitize=undefined`, respectively, to the compiler. Those
flags are enabled in GitHub workflows for ZTS and zloop. Errors
reported by both instrumentations are corrected, except for:

- Memory leak reporting is (temporarily) suppressed. The cost of
  fixing them is relatively high compared to the gains.

- Checksum computing functions in `module/zcommon/zfs_fletcher*`
  have UBSan errors suppressed. It is completely impractical
  to enforce 64-byte payload alignment there due to performance
  impact.

- There's no ASan heap poisoning in `module/zstd/lib/zstd.c`. A custom
  memory allocator is used there rendering that measure
  unfeasible.

- Memory leaks detection has to be suppressed for `cmd/zvol_id`.
  `zvol_id` is run by udev with the help of `ptrace(2)`. Tracing is
  incompatible with memory leaks detection.

Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes #12928

2 years agozed-functions.sh: escape newline to produce valid json
Phil Kauffman [Thu, 3 Feb 2022 22:31:57 +0000 (16:31 -0600)]
zed-functions.sh: escape newline to produce valid json

This was discovered when using Discords Slack compatible webhook.

Slack webhooks works without the escape, however Discord rightly refuses
the POST as it contains invalid JSON.

https://discord.com/developers/docs/resources/webhook#execute-slackcompatible-webhook

Valid (while escaping the newline:
```
+ msg_json='{"text": "*ZFS scrub_finish error for test on quartz*\nZFS has detected a data error:\n\n   eid: 124\n class: scrub_finish\n  host: quartz\n  time: \n error: \n objid: :\n  pool: test\n"}'
```

Invalid (no escape):
```
+ msg_json='{"text": "*ZFS scrub_finish error for test on quartz*
ZFS has detected a data error:\n\n   eid: 124\n class: scrub_finish\n  host: quartz\n  time: \n error: \n objid: :\n  pool: test\n"}'
```
The new line gets rendered and not sent inside the JSON as intended.

```
++ curl -X POST https://discord.com/api/webhooks/{webhook.id}/{webhook.token}/slack --header 'Content-Type: application/json' --data-binary '{"text": "*ZFS scrub_finish error for test on quartz*
ZFS has detected a data error:\n\n   eid: 124\n class: scrub_finish\n  host: quartz\n  time: \n error: \n objid: :\n  pool: test\n"}'
+ msg_out='{"message": "Cannot send an empty message", "code": 50006}'
```

Test method:
`root@quartz:/etc/zfs/zed.d# export ZED_ZEDLET_DIR=/etc/zfs/zed.d; export ZEVENT_EID=124; export ZEVENT_SUBCLASS=scrub_finish; export ZEVENT_POOL=test; export ZED_NOTIFY_DATA=1; bash -x ./data-notify.sh`

Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Philip Kauffman <philip@kauffman.me>
Closes #13049

2 years agoAdd enumerated vdev names to 'zpool iostat -v' and 'zpool list -v'
Akash B [Thu, 3 Feb 2022 22:29:29 +0000 (03:59 +0530)]
Add enumerated vdev names to 'zpool iostat -v' and 'zpool list -v'

This commit adds enumerated names to disambiguate between the
different vdevs. Previously only 'zpool status' showed enumerated
vdev names, now 'zpool list -v' and 'zpool iostat -v' also shows
the enumerated vdev names.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Dipak Ghosh <dipak.ghosh@hpe.com>
Signed-off-by: Akash B <akash-b@hpe.com>
Closes #12510
Closes #13031

2 years agoReport dnodes with faulty bonuslen
George Amanakis [Thu, 3 Feb 2022 22:28:19 +0000 (23:28 +0100)]
Report dnodes with faulty bonuslen

In files created/modified before 4254acb there may be a corruption of
xattrs which is not reported during scrub and normal send/receive. It
manifests only as an error when raw sending/receiving. This happens
because currently only the raw receive path checks for discrepancies
between the dnode bonus length and the spill pointer flag.

In case we encounter a dnode whose bonus length is greater than the
predicted one, we should report an error. Modify in this regard
dnode_sync() with an assertion at the end, dump_dnode() to error out,
dsl_scan_recurse() to report errors during a scrub, and zstream to
report a warning when dumping. Also added a test to verify spill blocks
are sent correctly in a raw send.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Amanakis <gamanakis@gmail.com>
Closes #12720
Closes #13014

2 years agoNotice if the test-runner dies
наб [Wed, 2 Feb 2022 22:17:46 +0000 (23:17 +0100)]
Notice if the test-runner dies

Currently, we seem to only care if the results collector errors.
We also should care if the test-runner died.

Authored-by: Rich Ercolani <rincebrain@gmail.com>
Co-authored-by: Rich Ercolani <rincebrain@gmail.com>
Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12998

2 years agoFix trivial calloc(3) arguments order
Tomohiro Kusumi [Wed, 2 Feb 2022 19:27:35 +0000 (04:27 +0900)]
Fix trivial calloc(3) arguments order

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <freqlabs@FreeBSD.org>
Signed-off-by: Tomohiro Kusumi <kusumi.tomohiro@gmail.com>
Closes #13052

2 years agoSimplify resume token generation
Ryan Moeller [Mon, 8 Nov 2021 22:35:05 +0000 (22:35 +0000)]
Simplify resume token generation

* Improve naming.
* Reduce indentation.
* Avoid boilerplate logic duplication.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Factor out lzc_flags_from_resume_nvl
Ryan Moeller [Mon, 13 Sep 2021 15:11:43 +0000 (15:11 +0000)]
libzfs_sendrecv: Factor out lzc_flags_from_resume_nvl

Improve the readability of zfs_send_resume_impl by moving resume nvl
decoding into a separate helper function.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Refactor find_redact_book
Ryan Moeller [Mon, 13 Sep 2021 15:10:56 +0000 (15:10 +0000)]
libzfs_sendrecv: Refactor find_redact_book

Factor out get_bookmarks, find_redact_pair, and get_redact_complete
helper functions to improve the readability of find_redact_book.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Fix some comment style nits
Ryan Moeller [Mon, 13 Sep 2021 14:51:02 +0000 (14:51 +0000)]
libzfs_sendrecv: Fix some comment style nits

* Capitalize and punctuate complete sentences.
* Add a blank line between functions.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Style pass on dump_filesystems
Ryan Moeller [Wed, 18 Aug 2021 21:01:21 +0000 (21:01 +0000)]
libzfs_sendrecv: Style pass on dump_filesystems

* Add a high level comment.
* Eliminate unnecessarily void arg.
* Capitalize and punctuate complete sentences in comments.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Style pass on dump_filesystem
Ryan Moeller [Wed, 18 Aug 2021 18:01:04 +0000 (18:01 +0000)]
libzfs_sendrecv: Style pass on dump_filesystem

* Add high level comments.
* Eliminate unnecessarily void arg.
* Avoid unnecessary line wrapping.
* Initialize sdd fields with the correct types.
* Remove extra whitespace.
* Refactor replication checks for clarity.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Style pass on dump_snapshot
Ryan Moeller [Wed, 18 Aug 2021 17:04:46 +0000 (17:04 +0000)]
libzfs_sendrecv: Style pass on dump_snapshot

* Add a high level comment.
* Avoid unnecessary line wrapping.
* Simplify size accounting logic.
* Eliminate unnecessary buffer on the stack.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Style pass on send_print_verbose
Ryan Moeller [Thu, 12 Aug 2021 16:50:50 +0000 (16:50 +0000)]
libzfs_sendrecv: Style pass on send_print_verbose

* Add missing dgettext calls.
* Avoid unnecessary line wraps.
* Factor out duplicated parsable check.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Pull header line out of loop
Ryan Moeller [Thu, 12 Aug 2021 16:48:56 +0000 (16:48 +0000)]
libzfs_sendrecv: Pull header line out of loop

This makes the header print before the sleep as well, which is fine.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Initialize in case of failure
Ryan Moeller [Thu, 5 Aug 2021 15:32:57 +0000 (15:32 +0000)]
libzfs_sendrecv: Initialize in case of failure

In zfs_send_progress, initialize \*bytes_written and \*blocks_visited
in case we have to return early due to ioctl failure.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Style pass on dump_ioctl
Ryan Moeller [Thu, 5 Aug 2021 15:29:24 +0000 (15:29 +0000)]
libzfs_sendrecv: Style pass on dump_ioctl

* Don't bother building a debug nvlist if we can't return it.
* Save errno after ioctl failure in case snprintf clobbers it.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Style pass on zfs_send_space
Ryan Moeller [Thu, 5 Aug 2021 15:28:32 +0000 (15:28 +0000)]
libzfs_sendrecv: Style pass on zfs_send_space

* Reduce indentation.
* Move locals closer to use.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Style pass on send_iterate_fs
Ryan Moeller [Tue, 27 Jul 2021 16:20:15 +0000 (16:20 +0000)]
libzfs_sendrecv: Style pass on send_iterate_fs

* Capitalize and punctuate complete sentences in comments.
* Separate out a group of locals to add a comment on their purpose.
* Remove unnecessary line wrapping.
* Make it clear that dds_origin is a string by using explicit character
  comparison to check for an empty string, rather than implictly
  treating it as a boolean.
* Reorganize manipulation of props and holds nvlists to improve
  clarity.
* There's no need to initialize the snapname buffer with zeros, we're
  immediately overwriting it.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Style pass on send_iterate_prop
Ryan Moeller [Tue, 27 Jul 2021 16:18:29 +0000 (16:18 +0000)]
libzfs_sendrecv: Style pass on send_iterate_prop

* Add a high level comment.
* Move locals closer to point of use.
* Use fnv* routines rather than explicit verification of success.
* Factor out duplicated code by introducing isspacelimit to clarify
  behavior.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Style pass on send_iterate_snap
Ryan Moeller [Tue, 27 Jul 2021 13:49:59 +0000 (13:49 +0000)]
libzfs_sendrecv: Style pass on send_iterate_snap

* Add a high level comment.
* Use local variables to reduce line wrapping.
* Remove extra braces and insert space for clarity.
* Assert precondition that the dataset name contains '@' for sanity.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Fix leaked holds nvlist
Ryan Moeller [Tue, 27 Jul 2021 13:48:42 +0000 (13:48 +0000)]
libzfs_sendrecv: Fix leaked holds nvlist

There is no need to allocate a holds nvlist.  lzc_get_holds does that
for us.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Avoid extra avl_find
Ryan Moeller [Tue, 27 Jul 2021 13:47:27 +0000 (13:47 +0000)]
libzfs_sendrecv: Avoid extra avl_find

avl_add does avl_find internally, then avl_insert.  We're already doing
the avl_find, so using avl_insert directly avoids repeating the search.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Simplify out guid temporary
Ryan Moeller [Tue, 27 Jul 2021 13:46:35 +0000 (13:46 +0000)]
libzfs_sendrecv: Simplify out guid temporary

De-clutter the clode and make it clear the guid is only used here.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs_sendrecv: Use size_t for payload_len
Ryan Moeller [Tue, 27 Jul 2021 13:45:28 +0000 (13:45 +0000)]
libzfs_sendrecv: Use size_t for payload_len

We won't be passing a negative value here, so make it clear.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12967

2 years agolibzfs: mount: don't leak mnt_param_t if mnt_func fails
наб [Wed, 12 Jan 2022 23:50:15 +0000 (00:50 +0100)]
libzfs: mount: don't leak mnt_param_t if mnt_func fails

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12968

2 years agolibtpool: remove useless CSTYLED
наб [Wed, 12 Jan 2022 16:38:03 +0000 (17:38 +0100)]
libtpool: remove useless CSTYLED

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12968

2 years agolinux: libspl: zone: () -> (void)
наб [Wed, 12 Jan 2022 16:37:40 +0000 (17:37 +0100)]
linux: libspl: zone: () -> (void)

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12968

2 years agozpool: misc. cleanup
наб [Wed, 12 Jan 2022 16:37:08 +0000 (17:37 +0100)]
zpool: misc. cleanup

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12968

2 years agolibzfs: share_proto: constify, staticify
наб [Wed, 12 Jan 2022 16:36:36 +0000 (17:36 +0100)]
libzfs: share_proto: constify, staticify

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12968

2 years agozpool: main: print hostids with PRIx64
наб [Wed, 12 Jan 2022 16:29:50 +0000 (17:29 +0100)]
zpool: main: print hostids with PRIx64

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12968

2 years agolibzfs: const correctness
наб [Wed, 12 Jan 2022 16:28:41 +0000 (17:28 +0100)]
libzfs: const correctness

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12968

2 years agozpool: main: const correctness, use time_t for time
наб [Wed, 12 Jan 2022 16:26:04 +0000 (17:26 +0100)]
zpool: main: const correctness, use time_t for time

The cast will explode on 32-bit big-endian architectures

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12968

2 years agospl: Don't check FreeBSD rwlocks for double initialization (#13019)
Mark Johnston [Mon, 31 Jan 2022 18:58:45 +0000 (13:58 -0500)]
spl: Don't check FreeBSD rwlocks for double initialization (#13019)

This checking breaks KMSAN since it effectively loads from uninitialized
memory to see if the lock is already initialized.  This happens in
dnode_cons() for example.  This checking is not very useful, partly due
to UMA's memory trashing, and is already disabled for mutexes.  Make
mutexes and rwlocks consistent: remove double-initialization checking
for rwlocks, and pass SX_NEW to disable the same checking in
lock_init().

No functional change intended, this affects only debug builds.

As a side note, kmem cache constructors/destructors are implemented
suboptimally on FreeBSD.  FreeBSD's slab allocator, UMA, supports two
pairs of constructors/destructors: ctor/dtor and init/fini.  The former
are called upon every allocation and free of an item, while the latter
are called when an item is imported or released from a zone,
respectively.  That is, when a slab is allocated to a particular cache,
it is subdivided into items, and init is called on each.  fini is called
when the slab is being prepared to be freed back to the system.  The
intent is for them to initialize static fields such as locks, which
do not need to be initialized upon each allocation of an item.

In illumos, kmem_cache constructors/destructors correspond to UMA's
init/fini callbacks.  However, in the SPL they are implemented as UMA
ctor/dtors, meaning that they get called far more often than necessary.
This may be difficult to fix, since new code may assume the kmem cache
ctor/dtors are in fact called upon each allocation/free, and there
doesn't seem to be a clear way to implement the intended semantics on
Linux.

Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Mark Johnston <markj@FreeBSD.org>
Closes #13019

2 years agoZTS: explicitly strip whitespace for broken wc(1) implementations
наб [Sat, 29 Jan 2022 00:59:52 +0000 (01:59 +0100)]
ZTS: explicitly strip whitespace for broken wc(1) implementations

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #13032

2 years agolibzfs: sendrecv: fix unused, remove argsused
наб [Fri, 28 Jan 2022 15:34:48 +0000 (16:34 +0100)]
libzfs: sendrecv: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12906

2 years agoReplace *CTASSERT() with _Static_assert()
наб [Fri, 21 Jan 2022 17:01:55 +0000 (18:01 +0100)]
Replace *CTASSERT() with _Static_assert()

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12993

2 years agoClean up CSTYLEDs
наб [Fri, 21 Jan 2022 16:07:15 +0000 (17:07 +0100)]
Clean up CSTYLEDs

69 CSTYLED BEGINs remain, appx. 30 of which can be removed if cstyle(1)
had a useful policy regarding
  CALL(ARG1,
   ARG2,
   ARG3);
above 2 lines. As it stands, it spits out *both*
  sysctl_os.c: 385: continuation line should be indented by 4 spaces
  sysctl_os.c: 385: indent by spaces instead of tabs
which is very cool

Another >10 could be fixed by removing "ulong" &al. handling.
I don't foresee anyone actually using it intentionally
(does it even exist in modern headers? why did it in the first place?).

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12993

2 years agoicp: asm_linkage.h: clean out unused bits, CSTYLED
наб [Fri, 21 Jan 2022 13:40:12 +0000 (14:40 +0100)]
icp: asm_linkage.h: clean out unused bits, CSTYLED

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12993

2 years agolibspl: include: ia32: remove
наб [Fri, 21 Jan 2022 13:38:43 +0000 (14:38 +0100)]
libspl: include: ia32: remove

It hosts only asm_linkage.h, which is entirely unused,
and has slightly diverged from the one that's actually used
(module/icp/include/sys/ia32/asm_linkage.h)

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12993

2 years agotests: simplify check_bg_procs_limit_num(), inline into setup
наб [Mon, 17 Jan 2022 18:11:06 +0000 (19:11 +0100)]
tests: simplify check_bg_procs_limit_num(), inline into setup

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12979

2 years agotests: prune remaining xargs(1), add missing zfs-project -c0 note
наб [Mon, 27 Dec 2021 17:29:38 +0000 (18:29 +0100)]
tests: prune remaining xargs(1), add missing zfs-project -c0 note

-c0 suppresses diagnoses ‒ it's not just -c but with NULs; cf.
  http://build.zfsonlinux.org/builders/Debian%2010%20x86_64%20%28TEST%29/builds/10605/steps/shell_4/logs/log
search for the second "zfs project -s -p" instance

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12979

2 years agotests: simplify find_vfstab_dev()
наб [Mon, 17 Jan 2022 15:40:45 +0000 (16:40 +0100)]
tests: simplify find_vfstab_dev()

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12979

2 years agoMakefile: simplify away xargs(1)
наб [Mon, 17 Jan 2022 15:37:16 +0000 (16:37 +0100)]
Makefile: simplify away xargs(1)

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12979

2 years agomodule: Makefile: simplify clean and install jobs
наб [Mon, 27 Dec 2021 01:21:38 +0000 (02:21 +0100)]
module: Makefile: simplify clean and install jobs

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12979

2 years agoFreeBSD: Fix zvol_cdev_open locking
Ryan Moeller [Wed, 26 Jan 2022 19:23:39 +0000 (14:23 -0500)]
FreeBSD: Fix zvol_cdev_open locking

First open locking changes were correctly applied to zvol_geom_open but
incorrectly applied to zvol_cdev_open, causing spa_namespace_lock to be
held indefinitely.

Make the first open locking in zvol_cdev_open match zvol_geom_open.

Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes #13016

2 years agoRPM: Add missing BuildRequires for PAM component
ColMelvin [Tue, 25 Jan 2022 21:14:43 +0000 (15:14 -0600)]
RPM: Add missing BuildRequires for PAM component

When the optional PAM binaries are included in a build, ./configure will
look for security/pam_modules.h and - if it doesn't find it - recommend
the user install `libpam0g-dev`.  On Red Hat systems, `pam-devel` is the
package that supplies this requirement; `libpam0g-dev` does not exist.

By encoding this requirement in the spec file, we give packagers more
appropriate (and timely) recommendations for completing the build.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Chris Lindee <chris.lindee+github@gmail.com>
Closes #13001

2 years agoLinux <4.8 compat: submit_bio() rw arg
Finix1979 [Tue, 25 Jan 2022 21:12:49 +0000 (05:12 +0800)]
Linux <4.8 compat: submit_bio() rw arg

When using the two argument version of submit_bio() in kernel's prior
to 4.8 the first argument should be specified.  It's used by block
dump to report the bio direction.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Finix Yan <yancw@info2soft.com>
Closes #13006

2 years agoLinux 5.17 compat: PDE_DATA() renamed to pde_data()
наб [Mon, 24 Jan 2022 19:11:07 +0000 (20:11 +0100)]
Linux 5.17 compat: PDE_DATA() renamed to pde_data()

Upstream commit 359745d78351c6f5442435f81549f0207ece28aa
("proc: remove PDE_DATA() completely")

Link: https://lore.kernel.org/all/20211124081956.87711-2-songmuchun@bytedance.com/T/#u
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #13004
Closes #12989

2 years agoLinux 5.17 compat: dequeue_signal() takes a 4th argument
наб [Wed, 19 Jan 2022 20:33:50 +0000 (21:33 +0100)]
Linux 5.17 compat: dequeue_signal() takes a 4th argument

Linux 5.17's dequeue_signal() takes an additional enum pid_type *
output argument

Upstream commit 5768d8906bc23d512b1a736c1e198aa833a6daa4
("signal: Requeue signals in the appropriate queue")

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12989

2 years agoLinux 5.17 compat: detect complete_and_exit() rename
наб [Wed, 19 Jan 2022 20:10:49 +0000 (21:10 +0100)]
Linux 5.17 compat: detect complete_and_exit() rename

Linux 5.17 sees a rename from complete_and_exit()
to kthread complete_and_exit()

Upstream commit cead18552660702a4a46f58e65188fe5f36e9dfe
("exit: Rename complete_and_exit to kthread_complete_and_exit")

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12989

2 years agolibspl: cast to uintptr_t instead of !!ing
наб [Tue, 25 Jan 2022 01:05:42 +0000 (02:05 +0100)]
libspl: cast to uintptr_t instead of !!ing

This led to these two warning types:
  debug.h:139:67: warning: the address of ‘ARC_anon’
  will always evaluate as ‘true’ [-Waddress]
    139 | #define ASSERT3P(x, y, z)
              ((void) sizeof (!!(x)), (void) sizeof (!!(z)))
        |                                               ^
  arc.c:1591:2: note: in expansion of macro ‘ASSERT3P’
   1591 |  ASSERT3P(hdr->b_l1hdr.b_state, ==, arc_anon);
        |  ^~~~~~~~
and
  arc.h:66:44: warning: ‘<<’ in boolean context,
  did you mean ‘<’? [-Wint-in-bool-context]
     66 | #define HDR_GET_LSIZE(hdr)
              ((hdr)->b_lsize << SPA_MINBLOCKSHIFT)
  debug.h:138:46: note: in definition of macro ‘ASSERT3U’
    138 | #define ASSERT3U(x, y, z)
              ((void) sizeof (!!(x)), (void) sizeof (!!(z)))
        |                        ^
  arc.c:1760:12: note: in expansion of macro ‘HDR_GET_LSIZE’
   1760 |   ASSERT3U(HDR_GET_LSIZE(hdr), !=, 0);
        |            ^~~~~~~~~~~~~

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #13009

2 years agoAdd explicit timeout to test step
Rich Ercolani [Mon, 24 Jan 2022 23:54:52 +0000 (18:54 -0500)]
Add explicit timeout to test step

If we die from timeout of the whole GH action run, we don't run the
collect step afterward, which can make it hard to investigate the
timeout.

If we timeout first in the test action, though, it qualifies as
failure, and collects appropriately.

(330 minutes seems like an acceptable tradeoff between the 6h
timeout by default on the action and the 4h and change "functional"
usually takes.)

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12999

2 years agoAdd support for FALLOC_FL_ZERO_RANGE
Rich Ercolani [Wed, 19 Jan 2022 05:12:15 +0000 (00:12 -0500)]
Add support for FALLOC_FL_ZERO_RANGE

For us, I think it's always just FALLOC_FL_PUNCH_HOLE with a fake
mustache on.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Coleman Kane <ckane@colemankane.org>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12975

2 years agoLinux 5.16 compat: Added mapping for iov_iter_fault_in_readable
Rich Ercolani [Fri, 14 Jan 2022 10:07:33 +0000 (05:07 -0500)]
Linux 5.16 compat: Added mapping for iov_iter_fault_in_readable

Linux decided to rename this for some reason. At some point, we
should probably invert this mapping, but for now...

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Coleman Kane <ckane@colemankane.org>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12975

2 years agoLinux 5.16 compat: Added add_disk check for return
Rich Ercolani [Fri, 14 Jan 2022 10:08:33 +0000 (05:08 -0500)]
Linux 5.16 compat: Added add_disk check for return

add_disk went from void to must-check int return.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Coleman Kane <ckane@colemankane.org>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12975

2 years agoLinux 5.16 compat: Check slab.h for kvmalloc
Rich Ercolani [Fri, 14 Jan 2022 10:09:08 +0000 (05:09 -0500)]
Linux 5.16 compat: Check slab.h for kvmalloc

As it says on the tin - the folio work moved a bunch out of mm.h.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Coleman Kane <ckane@colemankane.org>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12975

2 years agoFix test-runner on FreeBSD
наб [Fri, 21 Jan 2022 23:37:46 +0000 (00:37 +0100)]
Fix test-runner on FreeBSD

CLOCK_MONOTONIC_RAW is only a thing on Linux and macOS. I'm not
actually sure why the previous hardcoding of a constant didn't
error out, but when we removed it, it sure does now.

Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Co-authored-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12995

2 years agoFix handling of errors from dmu_write_uio_dbuf() on FreeBSD
Mark Johnston [Fri, 21 Jan 2022 19:54:05 +0000 (14:54 -0500)]
Fix handling of errors from dmu_write_uio_dbuf() on FreeBSD

FreeBSD's implementation of zfs_uio_fault_move() returns EFAULT when a
page fault occurs while copying data in or out of user buffers.  The VFS
treats such errors specially and will retry the I/O operation (which may
have made some partial progress).

When the FreeBSD and Linux implementations of zfs_write() were merged,
the handling of errors from dmu_write_uio_dbuf() changed such that
EFAULT is not handled as a partial write.  For example, when appending
to a file, the z_size field of the znode is not updated after a partial
write resulting in EFAULT.

Restore the old handling of errors from dmu_write_uio_dbuf() to fix
this.  This should have no impact on Linux, which has special handling
for EFAULT already.

Reviewed-by: Andriy Gapon <avg@FreeBSD.org>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Mark Johnston <markj@FreeBSD.org>
Closes #12964

2 years agoIntroduce a flag to skip comparing the local mac when raw sending
George Amanakis [Fri, 21 Jan 2022 19:41:17 +0000 (20:41 +0100)]
Introduce a flag to skip comparing the local mac when raw sending

Raw receiving a snapshot back to the originating dataset is currently
impossible because of user accounting being present in the originating
dataset.

One solution would be resetting user accounting when raw receiving on
the receiving dataset. However, to recalculate it we would have to dirty
all dnodes, which may not be preferable on big datasets.

Instead, we rely on the os_phys flag
OBJSET_FLAG_USERACCOUNTING_COMPLETE to indicate that user accounting is
incomplete when raw receiving. Thus, on the next mount of the receiving
dataset the local mac protecting user accounting is zeroed out.
The flag is then cleared when user accounting of the raw received
snapshot is calculated.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Amanakis <gamanakis@gmail.com>
Closes #12981
Closes #10523
Closes #11221
Closes #11294
Closes #12594
Issue #11300

2 years agoAvoid memory allocations in the ARC eviction thread
Mark Johnston [Fri, 21 Jan 2022 18:28:13 +0000 (13:28 -0500)]
Avoid memory allocations in the ARC eviction thread

When the eviction thread goes to shrink an ARC state, it allocates a set
of marker buffers used to hold its place in the state's sublists.

This can be problematic in low memory conditions, since
1) the allocation can be substantial, as we allocate NCPU markers;
2) on at least FreeBSD, page reclamation can block in
   arc_wait_for_eviction()

In particular, in stress tests it's possible to hit a deadlock on
FreeBSD when the number of free pages is very low, wherein the system is
waiting for the page daemon to reclaim memory, the page daemon is
waiting for the ARC eviction thread to finish, and the ARC eviction
thread is blocked waiting for more memory.

Try to reduce the likelihood of such deadlocks by pre-allocating markers
for the eviction thread at ARC initialization time.  When evicting
buffers from an ARC state, check to see if the current thread is the ARC
eviction thread, and use the pre-allocated markers for that purpose
rather than dynamically allocating them.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: George Amanakis <gamanakis@gmail.com>
Signed-off-by: Mark Johnston <markj@FreeBSD.org>
Closes #12985

2 years agolibspl: ASSERT*: !! for sizeof
наб [Fri, 21 Jan 2022 18:20:11 +0000 (19:20 +0100)]
libspl: ASSERT*: !! for sizeof

sizeof(bitfield.member) is invalid, and this shows up in some FreeBSD
build configurations: work around this by !!ing ‒
this makes the sizeof target the ! result type (_Bool), instead

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Fixes: 42aaf0e ("libspl: ASSERT*: mark arguments as used")
Closes #12984
Closes #12986

2 years agoFix problem with zdb -d
Paul Zuchowski [Thu, 20 Jan 2022 17:28:55 +0000 (12:28 -0500)]
Fix problem with zdb -d

zdb -d <pool>/<objset ID> does not work when
other command line arguments are included i.e.
zdb -U <cachefile> -d <pool>/<objset ID>
This change fixes the command line parsing
to handle this situation.  Also fix issue
where zdb -r <dataset> <file> does not handle
the root <dataset> of the pool. Introduce -N
option to force <objset ID> to be interpreted
as a numeric objsetID.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Paul Zuchowski <pzuchowski@datto.com>
Closes #12845
Closes #12944

2 years agolibefi: remove efi_type()
наб [Tue, 18 Jan 2022 22:40:43 +0000 (23:40 +0100)]
libefi: remove efi_type()

All it is right now is some #if 0ed Solaris code that returns ENOSYS,
and is only applicable for the Solaris blockdev layer.
In the Illumos gate, there's a single user: rmformat(1);
I recommend a read of the manual as a blast from the past, but, well

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Issue #12844
Closes #12969

2 years agomodule/*.ko: prune .data, global .rodata
наб [Fri, 14 Jan 2022 23:37:55 +0000 (00:37 +0100)]
module/*.ko: prune .data, global .rodata

Evaluated every variable that lives in .data (and globals in .rodata)
in the kernel modules, and constified/eliminated/localised them
appropriately. This means that all read-only data is now actually
read-only data, and, if possible, at file scope. A lot of previously-
global-symbols became inlinable (and inlined!) constants. Probably
not in a big Wowee Performance Moment, but hey.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12899

2 years agoClarify `failmode=wait` documentation
Brian Behlendorf [Fri, 14 Jan 2022 23:30:07 +0000 (15:30 -0800)]
Clarify `failmode=wait` documentation

Nowhere in the description of the failmode property does it
clearly state how to bring a suspended pool back online.
Add a few words to property description and the zpool-clear(8)
man page.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12907
Closes #9395

2 years agoFreeBSD: Touch up comments in zvol_os
Ryan Moeller [Thu, 23 Dec 2021 19:04:07 +0000 (19:04 +0000)]
FreeBSD: Touch up comments in zvol_os

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes #12934

2 years agoFreeBSD: Fix zvol_*_open() locking
Ryan Moeller [Thu, 23 Dec 2021 19:03:23 +0000 (19:03 +0000)]
FreeBSD: Fix zvol_*_open() locking

These are the changes for FreeBSD corresponding to the changes made for
Linux in #12863, see that PR for details.

Changes from #12863 are applied for zvol_geom_open and zvol_cdev_open
on FreeBSD.  This also adds a check for the zvol dying which we had
in zvol_geom_open but was missing in zvol_cdev_open.  The check causes
the open to fail early with ENXIO when we are in the middle of changing
volmode.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes #12934

2 years agoFreeBSD: Fix leaked strings in libspl mnttab
Ryan Moeller [Fri, 14 Jan 2022 20:38:32 +0000 (15:38 -0500)]
FreeBSD: Fix leaked strings in libspl mnttab

The FreeBSD implementations of various libspl functions for getting
mounted device information were found to leak several strings which
were being allocated in statfs2mnttab but never freed.

The Solaris getmntany(3C) and related interfaces are expected to return
strings residing in static buffers that need to be copied rather than
freed by the caller.

Use static thread-local storage to stash the mnttab structure strings
from FreeBSD's statfs info rather than strings allocated on the heap by
strdup(3).

While here, remove some stray commented out lines.

Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes #12961

2 years agoman: zfs.4: miscellaneous cleanup
наб [Thu, 13 Jan 2022 19:09:03 +0000 (20:09 +0100)]
man: zfs.4: miscellaneous cleanup

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12941

2 years agolinux: libzfs: mount: fix uninitialised flags
наб [Thu, 13 Jan 2022 19:07:54 +0000 (20:07 +0100)]
linux: libzfs: mount: fix uninitialised flags

They're later |=d with constants, but never reset

Caught by valgrind while investigating
https://github.com/openzfs/zfs/pull/12928#issuecomment-1007496550

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12954

2 years agoAdd ShellCheck's `--enable=all` inside `scripts/`
Damian Szuberski [Thu, 13 Jan 2022 18:09:19 +0000 (19:09 +0100)]
Add ShellCheck's `--enable=all` inside `scripts/`

Strengthen static code analysis for shell scripts.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes #12914

2 years agoRemoved Python 2 and Python 3.5- support
Damian Szuberski [Thu, 13 Jan 2022 16:51:12 +0000 (17:51 +0100)]
Removed Python 2 and Python 3.5- support

Deprecation of Python versions below 3.6 gives opportunity to unify the
build and install requirements for OpenZFS packages. The minimal
supported Python version is 3.6 as this is the most recent Python
package CentOS/RHEL 7 users can get.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes #12925

2 years agoRemove VERIFY() in vdev_props_set_sync()
Mark Maybee [Thu, 13 Jan 2022 00:15:30 +0000 (17:15 -0700)]
Remove VERIFY() in vdev_props_set_sync()

Reviewed-by: Allan Jude <allan@klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Mark Maybee <mark.maybee@delphix.com>
Closes #12951

2 years agolz4: Cherrypick fix for CVE-2021-3520
Rich Ercolani [Thu, 13 Jan 2022 00:14:36 +0000 (19:14 -0500)]
lz4: Cherrypick fix for CVE-2021-3520

There should be no risk of us accidentally hitting this since
we'd need maliciously malformed data to wind up in the pipeline,
or a very unfortunate random bit flip at exactly the right moment.
Still since we can handle it we should.

Reviewed-by: Igor Kozhukhov <igor@dilos.org>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Adam Moss <c@yotes.com>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12947

2 years agoUpdated the lz4 decompressor
Rich Ercolani [Fri, 7 Jan 2022 18:36:49 +0000 (13:36 -0500)]
Updated the lz4 decompressor

As an experiment, I stole the lz4 decompressor from
upstream lz4 (1.9.3), and landed it.

Feedback suggested that keeping the vendor lz4 code isolated and
unlinted was probably reasonable, so I lobbed it into its own file.

It also seemed reasonable to put the mostly-untouched* code into
lz4.c proper, and relegate the integrated and ZFS-specific code to
lz4_zfs.c.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12805

2 years agoREADME: Update OpenZFS website url
Ryan Hirasaki [Fri, 7 Jan 2022 00:25:01 +0000 (18:25 -0600)]
README: Update OpenZFS website url

This change is to first replace the OpenZFS website in the README to
point to openzfs.org as this is what open-zfs.org redirects to.
Along with replacing the URL, the protocol is also upgraded
from http to https.

These changes should prevent web browsers such as Firefox from
complaining about visiting a http site, if the proper security
settings are enabled, when it will still end up on a https page
after the redirect.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Hirasaki <ryanhirasaki@gmail.com>
Closes #12939

2 years agoRemove sha1 hashing from OpenZFS, it's not used anywhere.
Tino Reichardt [Mon, 3 Jan 2022 16:43:11 +0000 (17:43 +0100)]
Remove sha1 hashing from OpenZFS, it's not used anywhere.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Attila Fülöp <attila@fueloep.org>
Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12895
Closes #12902

2 years agomodule: icp: remove solaris crypto and cryptoadm ioctl definitions
наб [Wed, 22 Dec 2021 22:58:32 +0000 (23:58 +0100)]
module: icp: remove solaris crypto and cryptoadm ioctl definitions

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Attila Fülöp <attila@fueloep.org>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12895
Closes #12902

2 years agomodule: icp: rip out the Solaris loadable module architecture
наб [Wed, 22 Dec 2021 14:27:43 +0000 (15:27 +0100)]
module: icp: rip out the Solaris loadable module architecture

After progressively folding away null cases, it turns out there's
/literally/ nothing there, even if some things are part of the
Solaris SPARC DDI/DKI or the seventeen module types (some doubled for
32-bit userland), or the entire modctl syscall definition.
Nothing.

Initialisation is handled in illumos-crypto.c,
which calls all the initialisers directly

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Attila Fülöp <attila@fueloep.org>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12895
Closes #12902

2 years agoAdd ShellCheck's `--enable=all` inside `cmd/`
Damian Szuberski [Fri, 7 Jan 2022 00:07:54 +0000 (01:07 +0100)]
Add ShellCheck's `--enable=all` inside `cmd/`

The only exception is `cmd/vdev_id/vdev_id` which might be a subject of
refactoring (see #12084)

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes #12912

2 years agol2arc_write_buffers: remove redundant asserts
Christian Schwarz [Thu, 6 Jan 2022 22:39:22 +0000 (23:39 +0100)]
l2arc_write_buffers: remove redundant asserts

Probably introduced inadvertently in b525630 (Native Encryption).

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Amanakis <gamanakis@gmail.com>
Signed-off-by: Christian Schwarz <christian.schwarz@nutanix.com>
Closes #12935

2 years agoAdd ShellCheck's `--enable=all` inside `etc/`
Damian Szuberski [Thu, 6 Jan 2022 22:36:04 +0000 (23:36 +0100)]
Add ShellCheck's `--enable=all` inside `etc/`

Strengthen static code analysis for shell scripts.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes #12913

2 years agomodule: Makefile: flatten subdir loop, use $PWD instead of `pwd`
наб [Thu, 23 Dec 2021 20:57:44 +0000 (21:57 +0100)]
module: Makefile: flatten subdir loop, use $PWD instead of `pwd`

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Issue #12899

2 years agoFreeBSD: remove unused variable
наб [Thu, 23 Dec 2021 20:47:43 +0000 (21:47 +0100)]
FreeBSD: remove unused variable

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Issue #12899

2 years agoconfig: check for parallel(1), use it for cstyle
наб [Thu, 23 Dec 2021 20:26:19 +0000 (21:26 +0100)]
config: check for parallel(1), use it for cstyle

Before:
$ time make cstyle
real    0m23.118s
user    0m23.002s
sys     0m0.114s

After:
$ time make cstyle
real    0m4.577s
user    0m31.487s
sys     0m0.699s

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Issue #12899

2 years agolibfetch: unquote @LIBFETCH_SONAME@ subst
наб [Thu, 6 Jan 2022 19:26:40 +0000 (20:26 +0100)]
libfetch: unquote @LIBFETCH_SONAME@ subst

@LIBFETCH_SONAME@ is no longer quoted. The C define still is.

Ref: 153f7c9f72082d7ef5ee27fcbec1bcb94ba88151
Ref: https://github.com/openzfs/zfs/pull/12835#discussion_r776833743
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Damian Szuberski <szuberskidamian@gmail.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12922

2 years agozvol: remove unused variable
наб [Mon, 27 Dec 2021 18:33:58 +0000 (19:33 +0100)]
zvol: remove unused variable

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12917

2 years agofm: remove unused variables
наб [Mon, 27 Dec 2021 18:33:38 +0000 (19:33 +0100)]
fm: remove unused variables

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12917

2 years agozvol: remove unused variable
наб [Mon, 27 Dec 2021 18:33:16 +0000 (19:33 +0100)]
zvol: remove unused variable

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12917

2 years agoFreeBSD: vfsops: use setgen for error case
наб [Thu, 6 Jan 2022 19:15:08 +0000 (20:15 +0100)]
FreeBSD: vfsops: use setgen for error case

Fix from https://github.com/openzfs/zfs/pull/12844#discussion_r774179413

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12905

2 years agoRevert "zfs list: Allow more fields in ZFS_ITER_SIMPLE mode"
Paul Dagnelie [Thu, 6 Jan 2022 19:12:53 +0000 (11:12 -0800)]
Revert "zfs list: Allow more fields in ZFS_ITER_SIMPLE mode"

This reverts commit f6a0dac84af2fba9c306a3a307ea7aafcbe32d2b.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #12938

2 years agoman: speling
chrisrd [Thu, 6 Jan 2022 19:00:01 +0000 (06:00 +1100)]
man: speling

Fix spelling.

Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Chris Dunlop <chris@onthe.net.au>
Closes #12911