pve-common.git
4 days agoProcFSTools: read_proc_stat: add more cpu stats from /proc/stat master
Dominik Csapak [Wed, 28 Jul 2021 12:12:43 +0000 (14:12 +0200)]
ProcFSTools: read_proc_stat: add more cpu stats from /proc/stat

those fields might be interesting to users. At the moment, this is
only used in the external metrics export.

These fields exist in the kernel since:
* irq - 2.6.0
* softirq - 2.6.0
* steal - 2.6.11
* guest - 2.6.24
* guest_nice - 2.6.33

so they must all exist

also add a 'total' field which simply sums up the (non-guest) fields

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2 weeks agointerfaces: do not warn about FD if it was not set explicitly
Thomas Lamprecht [Fri, 16 Jul 2021 10:54:14 +0000 (12:54 +0200)]
interfaces: do not warn about FD if it was not set explicitly

As that would be weird, the use explicitly did not configure it,
which is OK, and we'd then warn due to *our* fallback to `0`.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 weeks agointerfaces: code cleanup and use some more actual telling variable names
Thomas Lamprecht [Fri, 16 Jul 2021 10:52:29 +0000 (12:52 +0200)]
interfaces: code cleanup and use some more actual telling variable names

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 weeks agointerfaces: improve bridge_fd handling
Fabian Grünbichler [Fri, 16 Jul 2021 07:40:50 +0000 (09:40 +0200)]
interfaces: improve bridge_fd handling

and ignore values with a warning that are outside of the kernels
expected range.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2 weeks agofix #2831: never set bridge_fd to 0 with STP on
Fabian Grünbichler [Fri, 16 Jul 2021 07:40:49 +0000 (09:40 +0200)]
fix #2831: never set bridge_fd to 0 with STP on

it's an invalid combination that causes the network reload/setup to
fail. unfortunately, this is not caught by ifupdown2 itself, but only
rejected by the kernel with ERANGE over netlink.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2 weeks agobump version to 7.0-5
Thomas Lamprecht [Wed, 14 Jul 2021 09:50:51 +0000 (11:50 +0200)]
bump version to 7.0-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 weeks agofix #3527: cgroup: drop file buffers from memory usage
Wolfgang Bumiller [Wed, 14 Jul 2021 07:42:15 +0000 (09:42 +0200)]
fix #3527: cgroup: drop file buffers from memory usage

matches the behavior of lxcfs

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
5 weeks agobump version to 7.0-4
Thomas Lamprecht [Mon, 28 Jun 2021 12:57:24 +0000 (14:57 +0200)]
bump version to 7.0-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 weeks agofix #3153: INotify: adding comment of interface to inet6 section when this is the...
Lorenz Stechauner [Fri, 25 Jun 2021 12:29:37 +0000 (14:29 +0200)]
fix #3153: INotify: adding comment of interface to inet6 section when this is the only section

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
5 weeks agotools: followup: fix comment length and rename to upid_normalize_status_type
Thomas Lamprecht [Mon, 28 Jun 2021 12:51:54 +0000 (14:51 +0200)]
tools: followup: fix comment length and rename to upid_normalize_status_type

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 weeks agoPVE/Tools: add 'upid_get_status_type'
Dominik Csapak [Thu, 24 Jun 2021 07:10:11 +0000 (09:10 +0200)]
PVE/Tools: add 'upid_get_status_type'

as a single point where we get the type of upid status

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
5 weeks agoPVE/JSONSchema: add pve-task-status-type
Dominik Csapak [Thu, 24 Jun 2021 07:10:10 +0000 (09:10 +0200)]
PVE/JSONSchema: add pve-task-status-type

to have a format that contains the possible worker task states

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
5 weeks agoinotify/network: indentation fix
Thomas Lamprecht [Thu, 24 Jun 2021 13:36:04 +0000 (15:36 +0200)]
inotify/network: indentation fix

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 weeks agobump version to 7.0-3
Thomas Lamprecht [Wed, 23 Jun 2021 10:08:00 +0000 (12:08 +0200)]
bump version to 7.0-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 weeks agosystemd: allow SendSIGKILL and TimeoutStopUSec dbus properties
Stefan Reiter [Mon, 21 Jun 2021 16:35:42 +0000 (18:35 +0200)]
systemd: allow SendSIGKILL and TimeoutStopUSec dbus properties

Used in qemu-server for avoiding KillMode 'none'. SendSIGKILL is a
boolean, so we need to use dbus_boolean to serialize it.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
6 weeks agoSysFSTools: change 'product' to 'device'
Dominik Csapak [Mon, 21 Jun 2021 13:55:16 +0000 (15:55 +0200)]
SysFSTools: change 'product' to 'device'

so it is more consistend with the source (it comes from the file
'device') as well as the subsytem_device field

the only place we use that field is in the same file in pci_dev_bind_to_vfio,
which we also change here, so that should not be a breaking change
(in qemu-server we only really use the existance and the has_fl_reset
flag)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
6 weeks agoSysFSTools: add verbose flag to pci_device_info
Dominik Csapak [Mon, 21 Jun 2021 13:55:15 +0000 (15:55 +0200)]
SysFSTools: add verbose flag to pci_device_info

to also get the subsystem_vendor and device, as well as the
iommu group and mediated device support

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
6 weeks agobump version to 7.0-2
Thomas Lamprecht [Thu, 17 Jun 2021 14:41:57 +0000 (16:41 +0200)]
bump version to 7.0-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
6 weeks agotools: add upid_status_is_error function
Fabian Ebner [Wed, 12 May 2021 12:32:50 +0000 (14:32 +0200)]
tools: add upid_status_is_error function

There's also support for ending a task with warnings now, so the logic "status
not 'OK' means error" does not work anymore.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
6 weeks agod/control: record new dependency for libnetaddr-ip-perl
Thomas Lamprecht [Thu, 17 Jun 2021 13:20:14 +0000 (15:20 +0200)]
d/control: record new dependency for libnetaddr-ip-perl

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
6 weeks agonetwork: add unique_ips function
Fabian Ebner [Mon, 10 May 2021 12:18:16 +0000 (14:18 +0200)]
network: add unique_ips function

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
6 weeks agonetwork: add canonical_ip function
Fabian Ebner [Mon, 10 May 2021 12:18:15 +0000 (14:18 +0200)]
network: add canonical_ip function

Net::IP doesn't seem to have a function for it and normalizing to the full
quad-form is less then ideal if we inted to output IPv6 addresses returned by
that function at some point.

Instead, use NetAddr::IP, which is already used in pve-network.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
6 weeks agonetwork: is_ip_in_cidr: avoid warning when versions don't match
Fabian Ebner [Mon, 10 May 2021 12:18:14 +0000 (14:18 +0200)]
network: is_ip_in_cidr: avoid warning when versions don't match

is_ip_in_cidr('fd80:1::10', '127.0.0.1/24') would result in
    Use of uninitialized value in numeric eq (==)
as overlaps() returns undef in such a case.

Note that there are (albeit few) existing callers that don't specify $version.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
6 weeks agonetwork: is_ip_in_cidr: correctly handle the CIDR being a singleton range
Fabian Ebner [Mon, 10 May 2021 12:18:13 +0000 (14:18 +0200)]
network: is_ip_in_cidr: correctly handle the CIDR being a singleton range

i.e.  is_ip_in_cidr('127.0.0.1', '127.0.0.1/32', 4) should return 1;

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
6 weeks agotools: download from url: add option to allow overriding existing files
Thomas Lamprecht [Wed, 16 Jun 2021 10:41:33 +0000 (12:41 +0200)]
tools: download from url: add option to allow overriding existing files

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
6 weeks agotools: download_file_from_url: move check for existing file outside eval
Lorenz Stechauner [Wed, 16 Jun 2021 09:35:58 +0000 (11:35 +0200)]
tools: download_file_from_url: move check for existing file outside eval

it is not necessary to include this block in the eval which when it
fails tries to unlink $tmpdest, because in the check for the existing
file $tmpdest is not used.

6 weeks agotools: download_file_from_url: adapt error messages to start at new line
Lorenz Stechauner [Wed, 16 Jun 2021 09:35:57 +0000 (11:35 +0200)]
tools: download_file_from_url: adapt error messages to start at new line

the front end expects the error message to be the first part of the
last line. putting the new line at the beginning of the die message
does not work, either.

https://lists.proxmox.com/pipermail/pve-devel/2021-June/048676.html

6 weeks agoinotify: also detect VLAN id from "vlan\d+" ifaces
Thomas Lamprecht [Tue, 15 Jun 2021 14:45:30 +0000 (16:45 +0200)]
inotify: also detect VLAN id from "vlan\d+" ifaces

We support also vlanX, with X being a integer for the VLAN id, as
valid vlan iface name, so support that too here.

and make the dev name check for definedness, even if "0" is currently
not really supported (officially) it is still a valid iface name for
the kernel (which takes any byte).

The VLAN id is in the range of [2, 4094] (inclusive) so defined check
is not required there.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
6 weeks agotools: download_file_from_url: fix typo
Lorenz Stechauner [Tue, 15 Jun 2021 14:08:34 +0000 (16:08 +0200)]
tools: download_file_from_url: fix typo

6 weeks agoinotify: read_interfaces: add vlan-id and vlan-raw-device on dot notation vlan interfaces
Aaron Lauterer [Mon, 12 Apr 2021 13:14:38 +0000 (15:14 +0200)]
inotify: read_interfaces: add vlan-id and vlan-raw-device on dot notation vlan interfaces

Setting the vlan-id and vlan-raw-device value for vlan devices that
follow the dot notaton (interface.vlan) aligns how dot notation vlan
devices and vlan devices that use the explicit vlan-id and
vlan-raw-device options, available with ifupdown2, are represented in
API return values.

Previously the type for both was 'vlan' but only the latter showed more
details.

Setting these values here should not have any influence on how the
interfaces file is being written as these two values are already
filtered in __write_etc_network_interfaces for dot notation devices.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
6 weeks agoSyscalls/Tools: add renameat2
Wolfgang Bumiller [Wed, 9 Jun 2021 13:18:45 +0000 (15:18 +0200)]
Syscalls/Tools: add renameat2

Mostly for the ability to atomically swap files.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
6 weeks agotools, rest env: sort use statements
Thomas Lamprecht [Tue, 15 Jun 2021 12:24:47 +0000 (14:24 +0200)]
tools, rest env: sort use statements

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
6 weeks agotools: get_file_hash: add use statements for Digest module
Thomas Lamprecht [Tue, 15 Jun 2021 12:22:18 +0000 (14:22 +0200)]
tools: get_file_hash: add use statements for Digest module

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
6 weeks agotools: download_file_from_url: handle interrupts
Thomas Lamprecht [Tue, 15 Jun 2021 12:20:49 +0000 (14:20 +0200)]
tools: download_file_from_url: handle interrupts

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
6 weeks agotools: download_file_from_url: improve UX and avoid cyclic dependencies
Thomas Lamprecht [Tue, 15 Jun 2021 12:11:07 +0000 (14:11 +0200)]
tools: download_file_from_url: improve UX and avoid cyclic dependencies

plus some refactoring

* drop worker, cannot be done here (RPCEnv is in pve-access-control)
* actually output the wrong "got" hash on mismatch
* die on existing file with mismatched
* drop double array for passing cmd
* drop `/usr/bin` prefix
* adapt rename error message
* add error handling for unlinking the temp. file

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
6 weeks agotools: cleanup usage line
Thomas Lamprecht [Tue, 15 Jun 2021 11:12:58 +0000 (13:12 +0200)]
tools: cleanup usage line

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
6 weeks agotools: add download_file_from_url
Lorenz Stechauner [Mon, 14 Jun 2021 09:05:51 +0000 (11:05 +0200)]
tools: add download_file_from_url

adds a common function to download arbitrary files from urls.

code is based on
manager:PVE/API2/Nodes.pm:aplinfo

Security notice: this function does not perform any permission
checking. The callee has to make sure, that only authorized users may
use this function.

Caution: This function is able to download files from internal
networks (which would not be visible/accessible from outside), the
callee needs to ensure that unprivileged (e.g., non root@pam or the
like) can only pass OK URLs (e.g., resolving to public routable IPs)

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
6 weeks agoRevert "daemon: add compat code for pmgproxy 6.x"
Stoiko Ivanov [Fri, 11 Jun 2021 15:55:52 +0000 (17:55 +0200)]
Revert "daemon: add compat code for pmgproxy 6.x"

This reverts commit a3777dce67cf17cafa82deeeedd9e6067eeb2e2e.

With the upcoming release of pmg-api 7.0 we included the changes for
configuring a LISTEN_IP, thus this compatibility code is not needed
anymore.

Quickly tested with current pmg-api master

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 months agotests: run with static TZ value
Fabian Grünbichler [Wed, 19 May 2021 09:49:19 +0000 (11:49 +0200)]
tests: run with static TZ value

else tests may randomly fail depending on which local TZ is set..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2 months agobump version to 7.0-1
Thomas Lamprecht [Sun, 9 May 2021 15:33:35 +0000 (17:33 +0200)]
bump version to 7.0-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agobuildsys: change upload dist to bullseye
Thomas Lamprecht [Sun, 9 May 2021 15:33:31 +0000 (17:33 +0200)]
buildsys: change upload dist to bullseye

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agod/control: bump debhelper compat to >= 12
Thomas Lamprecht [Sun, 9 May 2021 15:32:30 +0000 (17:32 +0200)]
d/control: bump debhelper compat to >= 12

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agobump version to 6.4-3
Thomas Lamprecht [Fri, 7 May 2021 14:24:39 +0000 (16:24 +0200)]
bump version to 6.4-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agodaemon: add compat code for pmgproxy 6.x
Stoiko Ivanov [Wed, 5 May 2021 14:36:24 +0000 (16:36 +0200)]
daemon: add compat code for pmgproxy 6.x

The changes to the listening behavior introduced with PVE 6.4 break
backwardscompatibility w.r.t. listening address and logging, which
should not be changed without explictly notifying the user.

This patch re-adds the family parameter, which is still used by
pmgproxy and based on its existence creates the socket as before.

compared to the IO::Socket::IP->new call used before
390fc10dc4a696dd30646cbdd018ad08d855175f, the only change is the
renaming of 'LocalAddr' to 'LocalHost' (which are synonymous in
IO::Socket::IP [0])

It can simply be reverted with the release of pmg-api 7.0 (where
we'll record the change in the release-notes and upgrade-page)

[0] https://perldoc.perl.org/IO::Socket::IP
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 months agodaemon: explicitly bind to wildcard address.
Stoiko Ivanov [Wed, 5 May 2021 14:36:23 +0000 (16:36 +0200)]
daemon: explicitly bind to wildcard address.

with the recent change in pve-manager pveproxy (and spiceproxy)
try binding to '::' per default. This fails for hosts having disabled
ipv6 via kernel commandline.

Our desired behavior of binding on '::' and only falling back to
'0.0.0.0' in case this is not supported is not directly possible with
IO::Socket::IP->new (or rather by Socket::GetAddrInfo, which at least
on my system always returns the v4 wildcard-address first).

the code now binds to:
* the provided $host if not undef
* '::' if $host is not set
* '0.0.0.0' if $host is not set and binding on '::' yields undef,
  which means that it failed to create a socket which normally means
  that IPv6 is disabled

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agodaemon: drop Domain parameter from create_reusable_socket
Stoiko Ivanov [Wed, 5 May 2021 14:36:22 +0000 (16:36 +0200)]
daemon: drop Domain parameter from create_reusable_socket

The Domain parameter for IO::Socket::IP is not used/needed.
It is needed to create a IP Socket when calling IO::Socket->new,
but here we call IO::Socket::IP-new directly (see [0]).

[0] https://perldoc.perl.org/IO::Socket::IP

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 months agoschema: check format: parse list formats as arrays
Fabian Ebner [Thu, 18 Mar 2021 08:44:18 +0000 (09:44 +0100)]
schema: check format: parse list formats as arrays

Previously, the returned value would be only the last element or undef in case
of an empty list. There's only a handful of callers of check_format() that look
at the return value and AFAICT none of the exisitng ones is for a -list format.
But best to avoid any future surprises.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2 months agoindentation fix
Fabian Ebner [Thu, 18 Mar 2021 08:44:17 +0000 (09:44 +0100)]
indentation fix

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
3 months agobump version to 6.4-2
Thomas Lamprecht [Mon, 26 Apr 2021 17:34:26 +0000 (19:34 +0200)]
bump version to 6.4-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agoINotify: add support for dummy interfaces type
Alexandre Derumier [Sun, 22 Nov 2020 16:37:25 +0000 (17:37 +0100)]
INotify: add support for dummy interfaces type

They can be used like loopback, but not limited to only 1 interface.
It's needed for bgp with multipath/ecmp to have a unique src ip

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
3 months agoREST handler: make API return validation opt-in
Thomas Lamprecht [Sun, 4 Apr 2021 15:05:56 +0000 (17:05 +0200)]
REST handler: make API return validation opt-in

It has not shown any real value in the last decade+ it was enabled,
and it can actually add quite some performance overhead. E.g., if an
API endpoint returns a few 100k of relatively simple entries we can
easily require several seconds, even tens of seconds, to run the
return validation - making it easier to run into timeouts along the
transmit path to the client.

The CLI handler has it still enabled, normally there's no timeout
there as no HTTP transmit path is involved, and d.csapak had a slight
preference for that in an off-list discussion.

The actual implementations in PMG or PVE could enable it too if
running under debug mode.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agocode cleanup
Thomas Lamprecht [Sun, 4 Apr 2021 15:04:56 +0000 (17:04 +0200)]
code cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agobump version to 6.4-1
Thomas Lamprecht [Fri, 23 Apr 2021 12:59:57 +0000 (14:59 +0200)]
bump version to 6.4-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agoallow workers to count warnings and finish tasks in a WARNINGS state
Fabian Ebner [Fri, 9 Apr 2021 08:44:35 +0000 (10:44 +0200)]
allow workers to count warnings and finish tasks in a WARNINGS state

as is already supported by the UI (and PBS).

A nice bonus is that warn() can be used by both workers and non-workers. For
workers, the output is redirected/duplicated as set up by {fork,tee}_worker(),
and non-erroring workers that issued a warning will end in a WARNINGS state.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
3 months agopbs: keep a separate $USE_CRYPT_PARAMS list per command exe
Thomas Lamprecht [Fri, 23 Apr 2021 12:15:13 +0000 (14:15 +0200)]
pbs: keep a separate $USE_CRYPT_PARAMS list per command exe

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agopbs: rework client exe handling and error message
Thomas Lamprecht [Fri, 23 Apr 2021 12:14:36 +0000 (14:14 +0200)]
pbs: rework client exe handling and error message

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agoPBSClient: use crypt params for file 'list' and 'extract'
Stefan Reiter [Thu, 22 Apr 2021 15:34:51 +0000 (17:34 +0200)]
PBSClient: use crypt params for file 'list' and 'extract'

Necessary for accessing encrypted backups.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 months agoPBSClient: add file_restore_extract function
Stefan Reiter [Thu, 22 Apr 2021 15:34:50 +0000 (17:34 +0200)]
PBSClient: add file_restore_extract function

*_prepare creates a fifo for streaming data back to clients directly,
filefile_restore_extract blocks and should be called from a background
worker - while it is running outcoming data can be read from the FIFO.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 months agoPBSClient: add file_restore_list command
Stefan Reiter [Thu, 22 Apr 2021 15:34:49 +0000 (17:34 +0200)]
PBSClient: add file_restore_list command

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 months agoPBSClient: adapt error message to include full package names
Stefan Reiter [Thu, 22 Apr 2021 15:34:48 +0000 (17:34 +0200)]
PBSClient: adapt error message to include full package names

More helpful for a user to know what they're missing.

Suggested-by: Dominic Jäger <d.jaeger@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 months agodaemon: create_reusable_socket: listen on IPv6 and IPv4
Oguz Bektas [Wed, 24 Mar 2021 13:20:36 +0000 (14:20 +0100)]
daemon: create_reusable_socket: listen on IPv6 and IPv4

When the 'Domain' configuration of IO::Socket::IP is set to PF_INET6,
it will prefer IPv6, but since we don't set 'V6Only' it will also
listen on IPv4 interfaces[0].

Drop the now obsoleted '$family' parameter.

[0]: https://perldoc.perl.org/IO::Socket::IP#REPLACING-IO::Socket-DEFAULT-BEHAVIOUR

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agoPBSClient: allow running other binaries
Stefan Reiter [Wed, 21 Apr 2021 11:15:31 +0000 (13:15 +0200)]
PBSClient: allow running other binaries

...such as proxmox-file-restore.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 months agoJSONSchema: don't cycle-check 'download' responses
Stefan Reiter [Wed, 21 Apr 2021 11:15:30 +0000 (13:15 +0200)]
JSONSchema: don't cycle-check 'download' responses

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 months agofixup: remove double braces
Stefan Reiter [Mon, 19 Apr 2021 08:43:50 +0000 (10:43 +0200)]
fixup: remove double braces

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 months agoschema: pull out abstract 'id-pair' verifier
Fabian Grünbichler [Tue, 13 Apr 2021 12:16:29 +0000 (14:16 +0200)]
schema: pull out abstract 'id-pair' verifier

we'll need another one for guest bridge IDs

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 months agotools: getaddrinfo: code/indentation cleanup
Thomas Lamprecht [Tue, 6 Apr 2021 08:33:51 +0000 (10:33 +0200)]
tools: getaddrinfo: code/indentation cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 months agoget_options: don't set optional positional params to `undef`
Wolfgang Bumiller [Tue, 16 Mar 2021 10:24:24 +0000 (11:24 +0100)]
get_options: don't set optional positional params to `undef`

Currently this happened if (and only if) at least one
positional parameter was passed.

We run into this with
  `pmgconfig cert delete <type> [<restart>]`
vs
  `pvenode cert delete [<restart>]`

where in the PVE case the `restart` option was simply
omitted, whereas for PMG due to the existence of `<type>`
the `restart` option was explicitly passedset in the $opts
hash but ended up being `undef`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
4 months agobump version to 6.3-5
Thomas Lamprecht [Tue, 9 Mar 2021 07:35:17 +0000 (08:35 +0100)]
bump version to 6.3-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 months agoformat: fix render_bytes with CLIFormatter
Stefan Reiter [Wed, 17 Feb 2021 13:32:16 +0000 (14:32 +0100)]
format: fix render_bytes with CLIFormatter

CLIFormatter passes in an options hash as a second paramter, which so
far was ignored. Now that we treat it as a precision parameter, check if
it is a hash and extract the option from it before using it.

Otherwise perl puts HASH(0x...) into the format and sprintf chokes.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
5 months agonet: ip from CIDR: sort result
Thomas Lamprecht [Thu, 25 Feb 2021 12:07:56 +0000 (13:07 +0100)]
net: ip from CIDR: sort result

this should not be a real problem yet, but the perl keys hash pseudo
random order was an issue way to often to bet on that..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 months agonetwork: get_local_ip_from_cidr: return unique IPs
Fabian Ebner [Mon, 22 Feb 2021 13:24:07 +0000 (14:24 +0100)]
network: get_local_ip_from_cidr: return unique IPs

to avoid erroring out when "multiple" addresses are configured on live
migration, when in fact it's the same IP multiple times.

Seems like the same problem for a caller in pve-cluster was fixed by checking
the uniqueness afterwards, see commit 266041169beb36c8892ca54265e2d91335307ffb
in pve-cluster. But there doesn't seem to be any caller relying on the current
behavior, and no additional information other than the addresses are returned,
so fix it here.

Reported here:
https://forum.proxmox.com/threads/live-migration-of-vms-via-full-mesh-network-could-not-get-ip-multiple-addresses-configured-on-local-node-for-network.84585/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
5 months agobump version to 6.3-4
Thomas Lamprecht [Fri, 19 Feb 2021 14:50:29 +0000 (15:50 +0100)]
bump version to 6.3-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 months agofix #3259: always free certificate file after reading it
Stoiko Ivanov [Thu, 11 Feb 2021 18:19:47 +0000 (19:19 +0100)]
fix #3259: always free certificate file after reading it

This commit mirrors e3c4007bc94956962c03da8c6853415d2fdf057c in
pve-cluster. The fix there got lost with a refactoring for reusing the
code here in pve-common in c92b771669e68a8b49906b8a0a68b533750567b0

Tested by repeadetly reading https://$pve-node:8006/nodes on a
virtual testcluster.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
5 months agoprocfs: whitespace/indendation cleanup
Thomas Lamprecht [Fri, 19 Feb 2021 14:35:06 +0000 (15:35 +0100)]
procfs: whitespace/indendation cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 months agoregister email-or-username format
Fabian Ebner [Mon, 15 Feb 2021 12:24:56 +0000 (13:24 +0100)]
register email-or-username format

To be used for the mailto vzdump parameter.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
5 months agosendmail: use more complete email regex and shellquote
Fabian Ebner [Mon, 15 Feb 2021 12:24:55 +0000 (13:24 +0100)]
sendmail: use more complete email regex and shellquote

Shellquote is needed for '~', and while it doesn't help with '-',
there should be no problem, because options are separated from mailto
since commit 216a3f4f131693dc4bbad5e06e96a61baef5f5e9.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
5 months agobump version to 6.3-3
Thomas Lamprecht [Mon, 8 Feb 2021 15:09:16 +0000 (16:09 +0100)]
bump version to 6.3-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 months agocgroup: get_pressure_stat: handle v1 controller gracefully
Thomas Lamprecht [Mon, 8 Feb 2021 15:04:18 +0000 (16:04 +0100)]
cgroup: get_pressure_stat: handle v1 controller gracefully

and return the default zeroed $res in case we cannot get any path

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 months agouse unified parser for pressure stats
Thomas Lamprecht [Mon, 8 Feb 2021 15:03:35 +0000 (16:03 +0100)]
use unified parser for pressure stats

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 months agoadd get_pressure_stat
Alexandre Derumier [Sun, 7 Feb 2021 13:36:45 +0000 (14:36 +0100)]
add get_pressure_stat

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
5 months agoformat: drop unused JSONSchema module use
Thomas Lamprecht [Mon, 8 Feb 2021 14:03:29 +0000 (15:03 +0100)]
format: drop unused JSONSchema module use

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
5 months agoformat: handle undef, 0, and decimals in render_duration
Stefan Reiter [Mon, 8 Feb 2021 11:15:08 +0000 (12:15 +0100)]
format: handle undef, 0, and decimals in render_duration

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
5 months agoextract PVE::Format from PVE::CLIFormatter for reuse
Stefan Reiter [Mon, 8 Feb 2021 11:15:07 +0000 (12:15 +0100)]
extract PVE::Format from PVE::CLIFormatter for reuse

and add some tests

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
7 months agoSectionConfig: parse_config: add errors to result
Fabian Ebner [Mon, 21 Dec 2020 13:48:19 +0000 (14:48 +0100)]
SectionConfig: parse_config: add errors to result

so that callers can know about them. This is useful in places where we'd rather
abort then continue with a faulty configuration. For example, when reading the
storage configuration before executing a backup job.

Originally-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
7 months agobuildsys: use plain cp instead of rsync
Thomas Lamprecht [Mon, 14 Dec 2020 18:51:37 +0000 (19:51 +0100)]
buildsys: use plain cp instead of rsync

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
7 months agobump version to 6.3-2
Thomas Lamprecht [Thu, 3 Dec 2020 15:53:22 +0000 (16:53 +0100)]
bump version to 6.3-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
7 months agofollowup comment/code cleanups
Thomas Lamprecht [Thu, 3 Dec 2020 15:51:30 +0000 (16:51 +0100)]
followup comment/code cleanups

actually explain why we set to undef and not use `delete`.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
7 months agotools: add extract_sensitive_params
Dominik Csapak [Wed, 2 Dec 2020 09:21:04 +0000 (10:21 +0100)]
tools: add extract_sensitive_params

moved and generalized from pve-storage, since we'll need it
in more places

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
7 months agoPBSClient: add get_repository to generate repository urls from configs
Dominik Csapak [Thu, 3 Dec 2020 11:43:38 +0000 (12:43 +0100)]
PBSClient: add get_repository to generate repository urls from configs

we can use this everywhere we need to have a repository url

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
7 months agobuildsys: clean more
Thomas Lamprecht [Thu, 3 Dec 2020 15:39:10 +0000 (16:39 +0100)]
buildsys: clean more

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 months agobump version to 6.3-1
Thomas Lamprecht [Fri, 27 Nov 2020 14:30:25 +0000 (15:30 +0100)]
bump version to 6.3-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 months agoInotify: delete vlan-id for "iface.X" vlan iface.
Alexandre Derumier [Thu, 26 Nov 2020 09:18:46 +0000 (10:18 +0100)]
Inotify: delete vlan-id for "iface.X" vlan iface.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
8 months agosubscription: use more specific machine repo definition for PVE/PMG
Thomas Lamprecht [Fri, 27 Nov 2020 14:22:34 +0000 (15:22 +0100)]
subscription: use more specific machine repo definition for PVE/PMG

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 months agoinotify: apt auth: sort longest machine entry first and allow deletion
Thomas Lamprecht [Fri, 27 Nov 2020 14:21:37 +0000 (15:21 +0100)]
inotify: apt auth: sort longest machine entry first and allow deletion

plus a few code cleanups.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 months agobump version to 6.2-6
Thomas Lamprecht [Tue, 17 Nov 2020 15:08:04 +0000 (16:08 +0100)]
bump version to 6.2-6

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 months agorest: register method: allow minus in path template parameter names
Thomas Lamprecht [Tue, 17 Nov 2020 15:05:20 +0000 (16:05 +0100)]
rest: register method: allow minus in path template parameter names

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 months agorest: register method: non-capturing group for path regex outer group
Thomas Lamprecht [Tue, 17 Nov 2020 15:05:04 +0000 (16:05 +0100)]
rest: register method: non-capturing group for path regex outer group

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 months agobump version to 6.2-5
Thomas Lamprecht [Tue, 17 Nov 2020 13:29:25 +0000 (14:29 +0100)]
bump version to 6.2-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 months agoPBS client: backup tree: avoid over generic param has
Thomas Lamprecht [Tue, 17 Nov 2020 12:04:53 +0000 (13:04 +0100)]
PBS client: backup tree: avoid over generic param has

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 months agoPBS client: get snapshots: avoid over generic param has
Thomas Lamprecht [Tue, 17 Nov 2020 12:04:43 +0000 (13:04 +0100)]
PBS client: get snapshots: avoid over generic param has

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>