Brian Behlendorf [Tue, 24 Sep 2024 00:07:29 +0000 (17:07 -0700)]
ZTS: CodeQL Action v3 update
Switch from v2 to v3 CodeQL Actions. The v2 actions will no longer
be supported as of Dec '24 so we need to move to v3. According to
the release notes they should be functionally equivalent.
Note that the only difference between v2 and v3 of the CodeQL
Action is the node version they support, ... For example 3.22.11
was the first v3 release and is functionally identical to 2.22.11.
Rob Norris [Sun, 5 May 2024 04:04:57 +0000 (14:04 +1000)]
linux: log a scary warning when used with an experimental kernel
Since the person using the kernel may not be the person who built it,
show a warning at module load too, in case they aren't aware that it
might be weird.
Reviewed-by: Robert Evans <evansr@google.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Signed-off-by: Rob Norris <robn@despairlabs.com> Sponsored-by: https://despairlabs.com/sponsor/
Closes #15986
Rob Norris [Tue, 12 Mar 2024 09:38:12 +0000 (20:38 +1100)]
config/kernel: enforce maximum kernel version, with escape hatch
META lists the maximum kernel version we consider to be fully supported.
However, we don't enforce this.
Sometimes we ship experimental patches for a newer kernel than we're
ready to support or, less often, we compile just fine against a newer
kernel. Invariably, something doesn't quite work properly, and it's
difficult for users to understand that they're actually running against
a kernel that we're not yet ready to support.
This commit tries to improve this situation. First, it simply enforces
Linux-Maximum, by having configure bail out if you try to compile
against a newer version that.
Then, it adds the --enable-linux-experimental switch to configure. When
supplied, this disables enforcing the maximum version, allowing the user
to attempt to build against a kernel with version higher than
Linux-Maximum.
Finally, if the switch is supplied _and_ configure is run against a
higher kernel version, it shows a big warning message when configure
finishes, and defines HAVE_LINUX_EXPERIMENTAL for the build. This allows
us to add code to modify runtime behaviour as well.
Reviewed-by: Robert Evans <evansr@google.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Signed-off-by: Rob Norris <robn@despairlabs.com> Sponsored-by: https://despairlabs.com/sponsor/
Closes #15986
The qemu-9-summary-page.sh script reads the file env.txt in the
first lines. When the module didn't build, this file was not copied
into the tarfile - causing the scipt to abort.
Fix: copy needed files into the tarfile in case of module build
failures. The fix ignores also empty tarfiles in future.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
Closes #16555
Alexander Motin [Sat, 21 Sep 2024 17:36:25 +0000 (13:36 -0400)]
Reduce and handle EAGAIN errors on AIO label reads
At least FreeBSD has a limit of 256 simultaneous AIO requests per
process. Attempt to issue more results in EAGAIN errors. Since we
issue 4 requests per disk/partition from 2xCPUs threads, it is
quite easy to reach that limit on large systems, that results in
random pool import failures. It annoyed me for quite a while on
a system with 64 CPUs and 70+ partitioned disks.
This patch from one side limits the number of threads to avoid the
error, while from another should softly fall back to sync reads in
case of error. It takes into account _SC_AIO_MAX as a system-wide
AIO limit and _SC_AIO_LISTIO_MAX as a closest value to per-process
limit. The last not exactly right, but it is the best I found.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored by: iXsystems, Inc.
Closes #16551
Add compatibility file for GRUB versions up to v2.06
GRUB is not able to detect ZFS pool if snaphsot of top level boot
pool is created. This issue is observed with GRUB versions up to
v2.06 if extensible_dataset feature is enabled on ZFS boot pool.
compatibility=grub2-2.06 would enable all read-only compatible
zpool features except extensible_dataset and other features that
depend on it.
The existing grub2 compatibility file is now renamed to grub2-2.12 to
reflect the appropriate grub2 version. grub2-2.12 lists all read-only
features that can be enabled on boot pool for grub2 with version 2.12
onwards.
A new symlink grub2 is created that currently points to the grub2-2.12
compatibility file.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Umer Saleem <usaleem@ixsystems.com>
Closes #13873
Closes #15261
Closes #15909
ZTS: Fix skipping over comment lines in zpool_create.shlib
In zpool_create.shlib, check_feature_set iterates over all features
mentioned in provided compatibility file to check if only those
features are enabled on the pool.
This commit fixes skipping over comment lines correctly. Otherwise,
the test case fails as comment lines are also treated as feature names
by check_feature_set function.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Umer Saleem <usaleem@ixsystems.com>
Closes #15909
Rob Norris [Sat, 21 Sep 2024 17:03:54 +0000 (03:03 +1000)]
FreeBSD: restore zfs_znode_update_vfs()
I accidentally removed this in c22d56e3e, and didn't notice because it
doesn't fail the build, but does fail to load into the kernel because it
can't link it.
Sponsored-by: https://despairlabs.com/sponsor/ Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: George Melikov <mail@gmelikov.ru> Signed-off-by: Rob Norris <robn@despairlabs.com>
Closes #16554
Brian Behlendorf [Fri, 20 Sep 2024 22:48:12 +0000 (15:48 -0700)]
Add SIMD metadata in /proc on Linux follow up
This change accidentally broke the FreeBSD build due to
a conflict between the simd_stat_init()/simd_stat_fini()
macros on FreeBSD and the extern function prototype.
Reviewed-by: Tony Hutter <hutter2@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #16552
Too many times, people's performance problems have amounted to
"somehow your SIMD support isn't working", and determining that
at runtime is difficult to describe to people.
This adds a /proc/spl/kstat/zfs/simd node, which exposes
metadata about which instructions ZFS thinks it can use,
on AArch64 and x86_64 Linux, to make investigating things
like this much easier.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #16530
This commit changes the workflow of the github actions.
- Ubuntu 20.04, 22.04, 24.04 will be tested via QEMU now
- remove unused scripts of this commit: b7bc334d1
- re-add the zloop standalone testings via zloop.yml
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
Closes #16549
George Melikov [Mon, 3 Jun 2024 17:40:25 +0000 (20:40 +0300)]
arc_hdr_authenticate: make explicit error
On compression we could be more explicit here for cases
where we can not recompress the data.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Co-authored-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes #9416
George Melikov [Wed, 29 May 2024 20:50:25 +0000 (23:50 +0300)]
ZLE compression: don't use BPE_PAYLOAD_SIZE
ZLE compressor needs additional bytes to process
d_len argument efficiently.
Don't use BPE_PAYLOAD_SIZE as d_len with it
before we rework zle compressor somehow.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes #9416
George Melikov [Tue, 10 Sep 2019 20:34:53 +0000 (23:34 +0300)]
zio_compress: introduce max size threshold
Now default compression is lz4, which can stop
compression process by itself on incompressible data.
If there are additional size checks -
we will only make our compressratio worse.
New usable compression thresholds are:
- less than BPE_PAYLOAD_SIZE (embedded_data feature);
- at least one saved sector.
Old 12.5% threshold is left to minimize affect
on existing user expectations of CPU utilization.
If data wasn't compressed - it will be saved as
ZIO_COMPRESS_OFF, so if we really need to recompress
data without ashift info and check anything -
we can just compress it with zero threshold.
So, we don't need a new feature flag here!
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes #9416
I think cli_root/zdb/zdb_backup.ksh runs also an FreeBSD and I needed to
include the sysutils/coreutils package for the FreeBSD tests within the
QEMU patchset.
This could be reverted, when this pull request gets upstream
Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
Closes #16543
Rob Norris [Wed, 28 Aug 2024 12:28:24 +0000 (22:28 +1000)]
zfs_znode: lift common code to a single shared file
For now, userspace has no znode implementation. Some of the property and
path handling code is used there though and is the same on all
platforms, so we only need a single copy of it.
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Rob Norris <robn@despairlabs.com> Sponsored-by: https://despairlabs.com/sponsor/
Closes #16492
Rob Norris [Wed, 28 Aug 2024 12:22:51 +0000 (22:22 +1000)]
arc_os: split userspace and Linux kernel code
The Linux arc_os.c carries userspace and kernel code, with very little
overlap between the two. This lifts the userspace parts out into a
separate arc_os.c for libzpool and removes it from the Linux side.
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Rob Norris <robn@despairlabs.com> Sponsored-by: https://despairlabs.com/sponsor/
Closes #16492
Rob Norris [Thu, 19 Sep 2024 02:55:48 +0000 (12:55 +1000)]
linux/abd_os: remove kernel version check for compound page support
All kernels we support have compound pages that work the way we would
like. However, this code is new and this knowledge was hard won, so I'd
like to leave the description and option there for a little while, even
if it can only be disabled with a recompile.
Sponsored-by: https://despairlabs.com/sponsor/
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de> Signed-off-by: Rob Norris <robn@despairlabs.com>
Closes #16545
Specializing cityhash4 on 32-bit architectures can reduce the size
of stack frames as well as instruction count. This is a tiny but
useful optimization, since some callers invoke it frequently.
When specializing into 1/2/3/4-arg versions, the stack usage
(in bytes) on some 32-bit arches are listed as follows:
Shengqi Chen [Tue, 27 Aug 2024 07:52:33 +0000 (15:52 +0800)]
dmu_objset: replace dnode_hash impl with cityhash4
As mentioned in PR #16131, replacing CRC-based hash with cityhash4
could slightly improve the performance by eliminating memory access.
Replacing algorightm is safe since the hash result is not persisted.
Don Brady [Wed, 18 Sep 2024 18:36:48 +0000 (12:36 -0600)]
Avoid fault diagnosis if multiple vdevs have errors
When multiple drives are throwing errors, it is likely not
a drive failing but rather a failure above the drives, like
a controller. The active cases context of the drive's peers
is now considered when making a diagnosis.
Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc.
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Don Brady <don.brady@klarasystems.com>
Closes #16531
Rob Norris [Sun, 1 Sep 2024 02:34:17 +0000 (12:34 +1000)]
zfs_file: implement zfs_file_deallocate for FreeBSD 14
FreeBSD 14 gained a `VOP_DEALLOCATE` VFS operation and a `fspacectl`
syscall to use it. At minimum, these zero the given region, and if the
underlying filesystem supports it, can make the region sparse. We can
use this to get TRIM-like behaviour for file vdevs.
Sponsored-by: https://despairlabs.com/sponsor/
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de> Signed-off-by: Rob Norris <robn@despairlabs.com>
Closes #16496
Rob Norris [Mon, 16 Sep 2024 01:11:29 +0000 (11:11 +1000)]
config: fix page_mapping test
It always failed from "unused variable" warnings-errors. The resulting
`#define page_mapping(...)` happend to work because it always overrode
the kernel's function prototype, but that's brittle.
Sponsored-by: https://despairlabs.com/sponsor/ Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de> Signed-off-by: Rob Norris <robn@despairlabs.com>
Closes #16479
Rob Norris [Mon, 5 Aug 2024 12:19:15 +0000 (22:19 +1000)]
config: remove HAVE_GET_LINK_COOKIE
As far as I can tell, this never made it to a real release. It was
introduced in 6b2553918d8b and removed a couple of weeks later in fceef393a538. This was all part of the development of what would become
4.5. So I assume this was OpenZFS chasing upstream development at the
time.
fceef393a538 viro 2015-12-30 switch ->get_link() to delayed_call, kill ->put_link() cd3417c8fc95 viro 2015-12-29 kill free_page_put_link() 0d0def49d05a viro 2015-12-08 teach nfs_get_link() to work in RCU mode 1a384eaac265 viro 2015-12-08 teach proc_self_get_link()/proc_thread_self_get_link() to work in RCU mode 6a6c99049635 viro 2015-12-08 teach shmem_get_link() to work in RCU mode d3883d4f9344 viro 2015-12-08 teach page_get_link() to work in RCU mode 6b2553918d8b viro 2015-12-08 replace ->follow_link() with new method that could stay in RCU mode
Sponsored-by: https://despairlabs.com/sponsor/ Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de> Signed-off-by: Rob Norris <robn@despairlabs.com>
Closes #16479
projects / mirror_zfs.git / log