FreeBSD: Fix integer conversion for vnlru_free{,_vfsops}()

When reviewing #13875, I noticed that our FreeBSD code has an issue
where it converts from `int64_t` to `int` when calling
`vnlru_free{,_vfsops}()`. The result is that if the int64_t is `1 <<
36`, the int will be 0, since the low bits are 0. Even when some low
bits are set, a value such as `((1 << 36) + 1)` would truncate to 1,
which is wrong.

There is protection against this on 32-bit platforms, but on 64-bit
platforms, there is no check to protect us, so we add a check.

Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes #13882

diff --git a/module/os/freebsd/zfs/arc_os.c b/module/os/freebsd/zfs/arc_os.c
index 77af092e1ed4397bc05a3842dc194fed568d6a46..590d1c04b9a5e324ec3d391304f8469921bf2dac 100644 (file)
--- a/module/os/freebsd/zfs/arc_os.c
+++ b/module/os/freebsd/zfs/arc_os.c
@@ -161,6 +161,12 @@ arc_prune_task(void *arg)
        int64_t nr_scan = (intptr_t)arg;
 
        arc_reduce_target_size(ptob(nr_scan));
+
+#ifndef __ILP32__
+       if (nr_scan > INT_MAX)
+               nr_scan = INT_MAX;
+#endif
+
 #if __FreeBSD_version >= 1300139
        sx_xlock(&arc_vnlru_lock);
        vnlru_free_vfsops(nr_scan, &zfs_vfsops, arc_vnlru_marker);