]> git.proxmox.com Git - pve-cluster.git/log
pve-cluster.git
2 years agofix #439: datacenter config: add infrastructure for next-id range configuration
Thomas Lamprecht [Thu, 24 Mar 2022 13:59:38 +0000 (14:59 +0100)]
fix #439: datacenter config: add infrastructure for next-id range configuration

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agodatacenter config: fix indentation style
Thomas Lamprecht [Thu, 24 Mar 2022 13:55:50 +0000 (14:55 +0100)]
datacenter config: fix indentation style

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agodatacenter config: refactor ref-checking for serialization
Thomas Lamprecht [Thu, 24 Mar 2022 12:09:57 +0000 (13:09 +0100)]
datacenter config: refactor ref-checking for serialization

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agodatacenter config: import parse_property_string
Thomas Lamprecht [Thu, 24 Mar 2022 12:07:50 +0000 (13:07 +0100)]
datacenter config: import parse_property_string

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agofix #3957: spell 'occurred' correctly
Fabian Grünbichler [Fri, 25 Mar 2022 08:07:42 +0000 (09:07 +0100)]
fix #3957: spell 'occurred' correctly

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2 years agoCluster: fix typo
Dominik Csapak [Mon, 14 Mar 2022 09:03:07 +0000 (10:03 +0100)]
Cluster: fix typo

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2 years agobump version to 7.1-3
Thomas Lamprecht [Wed, 15 Dec 2021 14:24:26 +0000 (15:24 +0100)]
bump version to 7.1-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoclusterlog: fix segfault / wrong iteration bounds
Fabian Grünbichler [Tue, 14 Dec 2021 10:19:12 +0000 (11:19 +0100)]
clusterlog: fix segfault / wrong iteration bounds

the clusterlog struct is a basic ring buffer:

struct clog_base {
    uint32_t size; // total size of this clog_base
    uint32_t cpos; // index into data, starts counting at start of clog_base, initially 0
    char data[];
};

an entry consists of indices of the next and previous entries and
various fields (fixed-length ones omitted here):

typedef struct {
uint32_t prev; // index of previous entry, or 0 if none exists
uint32_t next; // index of next entry
[..] // fixed-length fields
uint8_t node_len;
uint8_t ident_len;
uint8_t tag_len;
uint32_t msg_len;
char data[]; // node+ident+tag+msg - variable-length fields
} clog_entry_t;

the next and prev indices are calculated when allocating a new entry,
and the position of the current entry 'cpos' is updated accordingly
(clog_alloc_entry):
- size of the entry is padded with up to 7 bytes
- first entry goes to index 8
- second and subsequent entries go to the current entry's 'next' index
- if the current entry's 'next' index is out of bonds, the first entry
  is overwritten => wrap-around
- the 'prev' index of the new entry is set to cpos
- cpos is set to the index of the new entry
- the 'next' index of the new entry is set to its index+padded size

when iterating over the entries, the following bounds are used to follow
the 'prev' links starting at the current entry:

while (cpos && (cpos <= clog->cpos || cpos > (clog->cpos + CLOG_MAX_ENTRY_SIZE))) {

while this handles a not-yet-wrapped around ring buffer (cpos would be 0
when reaching the first entry), and tries to handle wrap-arounds by
terminating when reaching a 'red-zone' of 'CLOG_MAX_ENTRY_SIZE' starting
at the current entry (this covers the current entry which was already
visited as first entry during the iteration, and the next entry after it
which might have been overwritten) - but it's possible that entries line
up so that the wrap-around 'prev' index of the first entry points to a
location *before* the current entry.

for example, looking at clog_base with S being the size field, C being
the cpos field, followed by the actual data. N/P are the next/prev
indices of the entry at C, Q denotes the 'prev' index of the first entry
in the data array, and 'R' the red zone used for the loop check in case
of wrap-around.

first, fill up the buffer with six large entries:

Q                               P      C      N
|                               |      |      |
|                               |      |      |
v                               v      v      v
+-+-+------+------+------+------+------+------+-+
| | |      |      |      |      |      |      |x|
| | |   1  |   2  |   3  |   4  |   5  |   6  |x|
| | |      |      |      |      |      |      |x|
+-+-+------+------+------+------+------+------+-+
 S C                                    RRRRRRRRRRR

iterating from C backwards ends up at Q being 0, terminating the loop
without a wrap-around after having visit 6->1

now the next (in this example, smaller) entry that gets allocated/insert
needs to wrap around, because the empty space at the end (denoted by
XXX) is too small:

    C      N                          QP
    |      |                          ||
    |      |                          ||
    v      v                          vv
+-+-+------+------+------+------+------+------+-+
| | |      |      |      |      |      |      |x|
| | |   7  |   2  |   3  |   4  |   5  |   6  |x|
| | |      |      |      |      |      |      |x|
+-+-+------+------+------+------+------+------+-+
 S C RRRRRRRRRRR

iterating backwards from C terminates the loop when reaching the red
zone, with the (second) entry no longer being considered since it partly
overlaps it. only 7->3 are visited.

adding more entries we end up with the following layout:

                                   P  QC   N
                                   |  ||   |
                                   |  ||   |
                                   v  vv   v
+-+-+------+---+---+---+---+---+---+---+---+--+-+
| | |      |   |   |   |   |   |   |   |   |##|x|
| | |   7  | 8 | 9 |10 |11 |12 |13 |14 |15 |#6|x|
| | |      |   |   |   |   |   |   |   |   |##|x|
+-+-+------+---+---+---+---+---+---+---+---+--+-+
 S C                                    RRRRRRRRRRR

with # denoting space previously occupied the last large entry (#6)
which is still unmodified (the rest of that entry's data has been
overwritten by entries #14 and #15).

iterating from C (to the left/P) the loop ends up at entry #7, follows
the link to Q (which satisfies the loop bounds as Q < C), and the data
starting at (invalid index) Q gets interpreted as an entry. it is
possible (though even more unlikely than the partial overwrite case)
that Q and C line up perfectly, which would cause the loop to become an
infinite loop. the loop *should* terminate after having visited 15-7,
without wrapping around.

note that the actual sizes of the entries are not relevant, the
requirements are:
- entry before last wrap-around must be big enough that entry of current
  index can overtake it without another wrap-around
- method that does iteration must be called before next wrap-around

the fix is obviously trivial once the issue became apparent - when
wrapping around during iteration, additionally check that we are not
jumping across the red zone into already invalidated parts of data.

clusterlog_merge is technically not affected since it aborts before a
wrap-around anyway, but it doesn't hurt to have the checks consistently
in case this ever changes.

thanks to @kev1904 on our community forums for reporting and providing the data
to nail the cause down fast!

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2 years agoclusterlog: segfault reproducer
Fabian Grünbichler [Tue, 14 Dec 2021 10:19:11 +0000 (11:19 +0100)]
clusterlog: segfault reproducer

see next commit for details.

get_state mimics the code path triggered in the wild, the other two are
affected just the same.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2 years agod/copyright: update years
Thomas Lamprecht [Thu, 18 Nov 2021 07:48:22 +0000 (08:48 +0100)]
d/copyright: update years

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agobuildsys: fix variable names
Thomas Lamprecht [Mon, 15 Nov 2021 13:43:37 +0000 (14:43 +0100)]
buildsys: fix variable names

wrong was only the DBG one but make the LIB one use an underscore
separator too...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agobump version to 7.1-2
Thomas Lamprecht [Mon, 15 Nov 2021 13:32:00 +0000 (14:32 +0100)]
bump version to 7.1-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agodatacenter.cfg: code style fixes
Thomas Lamprecht [Mon, 15 Nov 2021 13:30:02 +0000 (14:30 +0100)]
datacenter.cfg: code style fixes

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agodatacenter.cfg: fix fall back for undefined config
Thomas Lamprecht [Mon, 15 Nov 2021 13:27:12 +0000 (14:27 +0100)]
datacenter.cfg: fix fall back for undefined config

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Reported-by: Oguz Bektas <o.bektas@proxmox.com>
2 years agobump version to 7.1-1
Thomas Lamprecht [Mon, 15 Nov 2021 07:24:36 +0000 (08:24 +0100)]
bump version to 7.1-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agofix #3596: handle delnode of offline node
Fabian Grünbichler [Fri, 12 Nov 2021 08:45:27 +0000 (09:45 +0100)]
fix #3596: handle delnode of offline node

the recommended way is to first shutdown, then delnode, and never let it
come back online, in which case corosync-cfgtool won't be able to kill
the removed (offline) node.

also, the order was wrong - if we first update corosync.conf to remove
the node entry from the nodelist, corosync doesn't know about the nodeid
anymore, so killing will fail even if the node is still online.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2 years agobump version to 7.0-5
Thomas Lamprecht [Thu, 11 Nov 2021 16:22:03 +0000 (17:22 +0100)]
bump version to 7.0-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agosysctl: disable net.ipv4.igmp_link_local_mcast_reports
Alexandre Derumier [Wed, 6 Oct 2021 08:32:20 +0000 (10:32 +0200)]
sysctl: disable net.ipv4.igmp_link_local_mcast_reports

currently, when veth or tap interfaces are plugged to bridge,
an igmp v3 report is broadcasted to the network, with the
bridge mac adddress.

Users have reported problems with hetzner for example, blocking the server
because of the unknown mac flooding the network.
https://forum.proxmox.com/threads/proxmox-claiming-mac-address.52601/page-6#post-421676

some traces:

ip addr:

190: fwbr109i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 22:5f:0b:cb:ac:42 brd ff:ff:ff:ff:ff:ff

ebtable log:
Oct  6 09:46:24 kvmformation3 kernel: [437256.753355] MAC-FLOOD-F IN=fwpr109p0 OUT=eno1 MAC source = 22:5f:0b:cb:ac:42 MAC dest = 01:00:5e:00:00:16 proto = 0x0800 IP SRC=0.0.0.0 IP DST=224.0.0.22, IP tos=0xC0, IP proto=2

tcpdump -e -i eno1 igmp
09:53:23.914825 22:5f:0b:cb:ac:42 (oui Unknown) > 01:00:5e:00:00:16 (oui Unknown), ethertype IPv4 (0x0800), length 54: 0.0.0.0 > igmp.mcast.net: igmp v3 report, 1 group record(s)

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agosysctl snippet: move to /usr/lib and prefix with 10-
Stoiko Ivanov [Thu, 11 Nov 2021 15:42:09 +0000 (16:42 +0100)]
sysctl snippet: move to /usr/lib and prefix with 10-

following best-practices according to `sysctl.d(5)`:
* Packages should install their configuration files in /usr/lib/ ...
* It is recommended to prefix all filenames with a two-digit number
  and a dash ...

the conffile removal is inspired by how it was done in `procps` (one
of the few packages in the debian repository, which did this
transition) and by following `dpkg-maintscript-helper(1)` and
`deb-conffiles(5)` (the former recommending the latter)

The choice of 10- as prefix is due to pve-container shipping its
snippet with that prefix already. other packages use higher numbers
(e.g. systemd - 50-)

Tested on 2 VMs (one with modifications, the other without) - worked
as advertised (the modified file was kept as
/etc/sysctl.d/pve.conf.dpkg-old and the upgrade notified me of the
change)

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agocluster: small code/style cleanups
Thomas Lamprecht [Thu, 11 Nov 2021 16:08:16 +0000 (17:08 +0100)]
cluster: small code/style cleanups

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoadd webauthn configuration to datacenter.cfg
Wolfgang Bumiller [Tue, 9 Nov 2021 11:27:06 +0000 (12:27 +0100)]
add webauthn configuration to datacenter.cfg

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2 years agobump version to 7.0-4
Thomas Lamprecht [Tue, 9 Nov 2021 17:30:14 +0000 (18:30 +0100)]
bump version to 7.0-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoadd 'jobs.cfg' to observed files
Dominik Csapak [Mon, 8 Nov 2021 13:07:52 +0000 (14:07 +0100)]
add 'jobs.cfg' to observed files

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Dylan Whyte <d.whyte@proxmox.com>
Tested-By: Aaron Lauterer <a.lauterer@proxmox.com>
2 years agoapi: join info: return explicit error code for no-cluster
Thomas Lamprecht [Sun, 7 Nov 2021 19:45:39 +0000 (20:45 +0100)]
api: join info: return explicit error code for no-cluster

allows an API client to more easily differ between this OK "error"
and an actual exception.

Note that I'd rather now just return undef or an empty object for the
no cluster case (not to sure about the original reasons about the die
anymore), but that would be a breaking change, and in fact it would
break current pve-manager versions out there, so schedule that for
the next major release (if we still want it then)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoaddnode: code reduction
Thomas Lamprecht [Sun, 7 Nov 2021 19:43:58 +0000 (20:43 +0100)]
addnode: code reduction

no semantic change intended

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agosetup: gen pve cert: code-style & indentation fixes
Thomas Lamprecht [Fri, 5 Nov 2021 13:27:04 +0000 (14:27 +0100)]
setup: gen pve cert: code-style & indentation fixes

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoRevert "d/control: better handle fuse3 transition"
Thomas Lamprecht [Wed, 6 Oct 2021 13:32:25 +0000 (15:32 +0200)]
Revert "d/control: better handle fuse3 transition"

This reverts commit a9592e415670131e35cd149eafd0993ddb526536.

3 years agodc.cfg: Add notes to datacenter config
Dominic Jäger [Wed, 22 Sep 2021 10:42:17 +0000 (12:42 +0200)]
dc.cfg: Add notes to datacenter config

Similar to notes for nodes.
datacenter.cfg normally uses key-value pairs defined in the schema.
We bypass this to allow potentially long comments at the top.

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agod/control: better handle fuse3 transition
Thomas Lamprecht [Tue, 20 Jul 2021 06:22:35 +0000 (08:22 +0200)]
d/control: better handle fuse3 transition

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump version to 7.0-3
Thomas Lamprecht [Thu, 1 Jul 2021 10:42:57 +0000 (12:42 +0200)]
bump version to 7.0-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmxcfs: bump basic FS limits, 1 MiB per-file, 128 MiB total
Thomas Lamprecht [Wed, 30 Jun 2021 10:06:16 +0000 (12:06 +0200)]
pmxcfs: bump basic FS limits, 1 MiB per-file, 128 MiB total

We have some users running into issues in some cases, like syncing
huge user base through LDAP into users.cfg or having a few thousands+
of HA services, as then the per-file limit is exhausted.

Bumping that one provides only half of the solution as the total
limit of 30 MiB would only allow a few files getting that big, or
reduce the amount left over for actual guest configurations quite a
bit.

So also bump the total filesystem limit from 30 MiB to 128 MiB, so by
a factor of ~4 and in the same spirit bump the maximal numbers of
inodes (i.e., different files) from 10k to 256k, which pmxcfs can
handle still rather easily (tested with touch) and would allow to max
out the full FS limit with 512 byte files, which fits small guest
configs, so sounds like an OK proportioned limit.

That should give use quite some wiggle room again, and should be
relatively safe as most of our access is rather small and on a few
files only, only root has full access anyway and that user can break
everything already, so not much lost here.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopve-cluster.service: remove ceph.service
Aaron Lauterer [Mon, 3 May 2021 10:00:11 +0000 (12:00 +0200)]
pve-cluster.service: remove ceph.service

The ceph.service file has been removed in pve-manager commit be244f1.
Therefore, there is no need to reference it anymore. This also avoids
showing the `ceph.service` as a `not found` unit.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
3 years agobump version to 7.0-2
Thomas Lamprecht [Thu, 20 May 2021 09:21:06 +0000 (11:21 +0200)]
bump version to 7.0-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agod/control: add missing libtest-mockmodule-perl b-d
Fabian Grünbichler [Tue, 18 May 2021 12:37:35 +0000 (14:37 +0200)]
d/control: add missing libtest-mockmodule-perl b-d

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agoget tasklist: unpack null-terminated C string before decoding as JSON
Thomas Lamprecht [Sat, 15 May 2021 11:00:46 +0000 (13:00 +0200)]
get tasklist: unpack null-terminated C string before decoding as JSON

This was always an "issue", but with Perl 5.28, from our Debian Buster
based release, decode_json just ignored the \0 NUL byte.

For example:
```
perl -w -MJSON -e 'my $raw = "[]\0"; print to_json(decode_json($raw), {pretty=>1});'

```
will get you the following error on perl 5.32
```
garbage after JSON object, at character offset 2 (before "\x{0}") at -e line 1.
```

Note, I did not find anything related in the perldelta aricles for
the 28 -> 30 or 30 -> 32 update, the first one made a bigger jump for
the JSON module version used, so possibly a change there.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoget tasklist: code cleanup
Thomas Lamprecht [Sat, 15 May 2021 11:00:10 +0000 (13:00 +0200)]
get tasklist: code cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump version to 7.0-1
Thomas Lamprecht [Sun, 9 May 2021 15:58:45 +0000 (17:58 +0200)]
bump version to 7.0-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobuildsys: fix IPCC.so linkage...
Thomas Lamprecht [Sun, 9 May 2021 17:31:13 +0000 (19:31 +0200)]
buildsys: fix IPCC.so linkage...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobuildsys: change upload dist to bullseye
Thomas Lamprecht [Sun, 9 May 2021 15:58:40 +0000 (17:58 +0200)]
buildsys: change upload dist to bullseye

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agod/control: bump debhelper compat to >= 12
Thomas Lamprecht [Sun, 9 May 2021 15:56:33 +0000 (17:56 +0200)]
d/control: bump debhelper compat to >= 12

dh_systemd was enabled by default since level 10, with level 12 the
compat plugin does not exists anymore so enabling it manually results
in an error.

The dh_strip override is now obsolete too, as users need to go
through 5.4 AND 6.4 anyway on upgrade, and new installations do not
matter here.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agod/control: adapt libqb SO-Version dependency change
Thomas Lamprecht [Sun, 9 May 2021 15:37:05 +0000 (17:37 +0200)]
d/control: adapt libqb SO-Version dependency change

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump version to 6.4-1
Thomas Lamprecht [Mon, 26 Apr 2021 14:01:12 +0000 (16:01 +0200)]
bump version to 6.4-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopvecm: fix typo in description for 'updatecerts'
Oguz Bektas [Mon, 1 Feb 2021 14:47:01 +0000 (15:47 +0100)]
pvecm: fix typo in description for 'updatecerts'

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
3 years agocfs lock: avoid confusing lock prefix on error
Thomas Lamprecht [Thu, 22 Apr 2021 19:38:26 +0000 (21:38 +0200)]
cfs lock: avoid confusing lock prefix on error

we have lots of forum posts where users think that the locking was
the error, not the actual error message from the called code.

This has limited value as general-applied prefix, if a code requires
the lockid or whatever to be included in the error message they can
already do so, so just re-raise the error and be done, at least if it
is a error from the code and not from the lock setup,.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmxcfs: db: tell query planner that prepared statement are long living
Thomas Lamprecht [Thu, 22 Apr 2021 08:46:42 +0000 (10:46 +0200)]
pmxcfs: db: tell query planner that prepared statement are long living

SQLITE_PREPARE_PERSISTENT
    The SQLITE_PREPARE_PERSISTENT flag is a hint to the query planner
    that the prepared statement will be retained for a long time and
    probably reused many times. Without this flag,
    sqlite3_prepare_v3() and sqlite3_prepare16_v3() assume that the
    prepared statement will be used just once or at most a few times
    and then destroyed using sqlite3_finalize() relatively soon. The
    current implementation acts on this hint by avoiding the use of
    lookaside memory so as not to deplete the limited store of
    lookaside memory. Future versions of SQLite may act on this hint
    differently.
-- https://sqlite.org/c3ref/c_prepare_normalize.html#sqlitepreparepersistent

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmxcfs: db: use SQLITE_STATIC to avoid memory copies
Thomas Lamprecht [Thu, 22 Apr 2021 08:18:58 +0000 (10:18 +0200)]
pmxcfs: db: use SQLITE_STATIC to avoid memory copies

we can trust that we own *value and *name until the sqlite statement
was executed, so use the STATIC bind flag to tell sqlite that it does
not need to make it's own copy in the bind statement.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmxcfs: more debug info on backend write and duplicate inode checks + cleanup
Thomas Lamprecht [Thu, 22 Apr 2021 08:17:50 +0000 (10:17 +0200)]
pmxcfs: more debug info on backend write and duplicate inode checks + cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agorename sdn/.version to sdn/.running-config
Alexandre Derumier [Mon, 28 Sep 2020 08:33:39 +0000 (10:33 +0200)]
rename sdn/.version to sdn/.running-config

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoadd sdn/dns.cfg
Alexandre Derumier [Mon, 28 Sep 2020 08:33:38 +0000 (10:33 +0200)]
add sdn/dns.cfg

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoadd priv/ipam.db
Alexandre Derumier [Mon, 28 Sep 2020 08:33:37 +0000 (10:33 +0200)]
add priv/ipam.db

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoadd sdn/ipams.cfg
Alexandre Derumier [Mon, 28 Sep 2020 08:33:36 +0000 (10:33 +0200)]
add sdn/ipams.cfg

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoadd sdn/subnets.cfg
Alexandre Derumier [Mon, 28 Sep 2020 08:33:35 +0000 (10:33 +0200)]
add sdn/subnets.cfg

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmxcfs: tests: make add_test signature backward compatible
Thomas Lamprecht [Sun, 31 Jan 2021 02:21:48 +0000 (03:21 +0100)]
pmxcfs: tests: make add_test signature backward compatible

we still need to be able to build with the libcheck version from
buster..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmxcfs: status: catch possible allocation error
Thomas Lamprecht [Sun, 31 Jan 2021 02:08:56 +0000 (03:08 +0100)]
pmxcfs: status: catch possible allocation error

even if not really realistic to happen in Linux

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmxcfs: tests: fix type in add_test signature
Thomas Lamprecht [Sun, 31 Jan 2021 02:08:17 +0000 (03:08 +0100)]
pmxcfs: tests: fix type in add_test signature

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmxcfs: tests: replace depreacated fail_unless with ck_assert
Thomas Lamprecht [Sun, 31 Jan 2021 02:07:43 +0000 (03:07 +0100)]
pmxcfs: tests: replace depreacated fail_unless with ck_assert

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobuildsys: drop legacy docgen control variable
Thomas Lamprecht [Mon, 14 Dec 2020 20:42:07 +0000 (21:42 +0100)]
buildsys: drop legacy docgen control variable

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobuildsys: fix linkage, place library definitions after objects
Thomas Lamprecht [Mon, 14 Dec 2020 20:37:26 +0000 (21:37 +0100)]
buildsys: fix linkage, place library definitions after objects

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agopmxcfs: do not grant LXC configs o+r permissions anymore
Thomas Lamprecht [Wed, 7 Oct 2020 07:56:05 +0000 (09:56 +0200)]
pmxcfs: do not grant LXC configs o+r permissions anymore

This was initially done because of some hook reading the config from
an unprivileged namespace when using unprivileged containers.

But, we nowadays do not do this anymore, either setup stuff before to
or use another source for getting required information (e.g., our
autodev hook uses "/var/lib/lxc/$vmid/devices").

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agopmxcfs: add some removal reminders for openvz
Thomas Lamprecht [Wed, 7 Oct 2020 08:07:09 +0000 (10:07 +0200)]
pmxcfs: add some removal reminders for openvz

Removing them now could count as compat breakage, for users which
still depend on some of this weird behavior it's nicer if we do this
more explicitly with 7.0

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agobump version to 6.2-1
Thomas Lamprecht [Wed, 30 Sep 2020 11:48:54 +0000 (13:48 +0200)]
bump version to 6.2-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agopmxcfs: update copyright in license header
Thomas Lamprecht [Wed, 30 Sep 2020 11:31:12 +0000 (13:31 +0200)]
pmxcfs: update copyright in license header

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agopmxcfs: protect CPG operations with mutex
Fabian Grünbichler [Wed, 30 Sep 2020 11:21:31 +0000 (13:21 +0200)]
pmxcfs: protect CPG operations with mutex

cpg_mcast_joined (and transitively, cpg_join/leave) are not thread-safe.
pmxcfs triggers such operations via FUSE and CPG dispatch callbacks,
which are running in concurrent threads.

accordingly, we need to protect these operations with a mutex, otherwise
they might return CS_OK without actually doing what they were supposed
to do (which in turn can lead to the dfsm taking a wrong turn and
getting stuck in a supposedly short-lived state, blocking access via
FUSE and getting whole clusters fenced).

huge thanks to Alexandre Derumier for providing the initial bug report
and quite a lot of test runs while debugging this issue.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agopmxcfs sync: properly check for corosync error
Fabian Grünbichler [Fri, 25 Sep 2020 12:53:49 +0000 (14:53 +0200)]
pmxcfs sync: properly check for corosync error

dfsm_send_state_message_full always returns != 0, since it returns
cs_error_t which starts with CS_OK at 1, with values >1 representing
errors.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agopvecm: pass correct nodename to finish_join
Fabian Grünbichler [Tue, 12 May 2020 08:59:05 +0000 (10:59 +0200)]
pvecm: pass correct nodename to finish_join

only cosmetic, but printing the wrong nodename might cause confusion.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agofix #2727: pass correct format for linkX
Fabian Grünbichler [Tue, 12 May 2020 08:59:04 +0000 (10:59 +0200)]
fix #2727: pass correct format for linkX

to unbreak joining via SSH with an explicit link address.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agostyle nit: use post-if not and'ed expression
Thomas Lamprecht [Mon, 18 May 2020 12:45:03 +0000 (14:45 +0200)]
style nit: use post-if not and'ed expression

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agobump version to 6.1-8
Thomas Lamprecht [Sun, 3 May 2020 16:19:31 +0000 (18:19 +0200)]
bump version to 6.1-8

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoprepare observed-files basedire: fix mounted check
Thomas Lamprecht [Sun, 3 May 2020 16:19:18 +0000 (18:19 +0200)]
prepare observed-files basedire: fix mounted check

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agobump version to 6.1-8
Thomas Lamprecht [Sat, 2 May 2020 17:14:37 +0000 (19:14 +0200)]
bump version to 6.1-8

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agosdn: add sdn/.version file
Alexandre Derumier [Mon, 13 Apr 2020 06:12:57 +0000 (08:12 +0200)]
sdn: add sdn/.version file

4 years agosdn: remove .new files
Alexandre Derumier [Mon, 13 Apr 2020 06:12:56 +0000 (08:12 +0200)]
sdn: remove .new files

4 years agobump version to 6.1-7
Thomas Lamprecht [Thu, 30 Apr 2020 15:33:56 +0000 (17:33 +0200)]
bump version to 6.1-7

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoupdatecerts: create base directories of observed files
Thomas Lamprecht [Thu, 30 Apr 2020 15:30:44 +0000 (17:30 +0200)]
updatecerts: create base directories of observed files

replaces the random hacks where we do some hail-mary mkdir in a
writer or the like, to ensure that the directory structure exists and
we can write safely.

more central and safer would be pmxcfs itself, but to late in the
release cycle to do that now.

Chicken out if pmxcfs is not mounted, we don't want to trash it's
(future) mountpoint..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agobump version to 6.1-6
Thomas Lamprecht [Thu, 30 Apr 2020 12:32:17 +0000 (14:32 +0200)]
bump version to 6.1-6

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoAdd ACME plugin config file to cluster files
Wolfgang Link [Thu, 16 Apr 2020 05:18:18 +0000 (07:18 +0200)]
Add ACME plugin config file to cluster files

Signed-off-by: Wolfgang Link <w.link@proxmox.com>
4 years agocfs_lock: make it more clear that an error happened during not with the cfs-lock
Thomas Lamprecht [Wed, 29 Apr 2020 09:13:59 +0000 (11:13 +0200)]
cfs_lock: make it more clear that an error happened during not with the cfs-lock

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agofix trailing whitespace error
Thomas Lamprecht [Wed, 29 Apr 2020 09:11:23 +0000 (11:11 +0200)]
fix trailing whitespace error

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agocfs_lock: re-raise exceptions
Fabian Grünbichler [Wed, 29 Apr 2020 08:52:51 +0000 (10:52 +0200)]
cfs_lock: re-raise exceptions

so that API paths that raise an exception while holding a CFS lock
properly propagate that exception to the client, instead of the
stringified version with added noise about locks added to the front.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agocfs_lock: add firewall lock helper
Fabian Grünbichler [Wed, 29 Apr 2020 08:52:48 +0000 (10:52 +0200)]
cfs_lock: add firewall lock helper

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agobump version to 6.1-5
Thomas Lamprecht [Wed, 25 Mar 2020 16:38:49 +0000 (17:38 +0100)]
bump version to 6.1-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agojoin API version: factor out checks and add info to outputs
Thomas Lamprecht [Wed, 25 Mar 2020 16:31:31 +0000 (17:31 +0100)]
join API version: factor out checks and add info to outputs

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoAdd cluster join API version check
Stefan Reiter [Thu, 9 Jan 2020 15:31:36 +0000 (16:31 +0100)]
Add cluster join API version check

Adds API call GET /cluster/config/apiversion to retrieve remote clusters
join-API version (0 is assumed for versions without this endpoint). Also
available via CLI as 'pvecm apiver'.

Introduce API_AGE similar to storage plugin API, but with two ages for
cluster/joinee roles. Currently, all versions are intercompatible.

For future usage, a new 'addnode' parameter 'apiversion' is introduced,
to allow introducing API breakages for joining nodes as well.

As a first compatibility check, use new fallback method only if
available. This ensures full compatibility between nodes/clusters with
and without new fallback behaviour.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agopvecm: qdevice setup: fix check for odd node count
Aaron Lauterer [Tue, 24 Mar 2020 16:16:42 +0000 (17:16 +0100)]
pvecm: qdevice setup: fix check for odd node count

With Perl 5.26 the behavior of `scalar(%hash)` changed [0] causing the
check for odd numbers to never evaluate to true. Allowing odd sized
clusters to set up a QDevice. The algorithm was not changed to LMS if
forced to still create the QDevice.

Instead of showing the bucket info of the referenced hash it did show
the hash reference. Dereferencing it will again return the number of
items present in the hash.

[0] https://perldoc.perl.org/perl5260delta.html#scalar(%25hash)-return-signature-changed

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
4 years agoassert joinable: add warning if forced to continue
Thomas Lamprecht [Wed, 19 Feb 2020 08:32:36 +0000 (09:32 +0100)]
assert joinable: add warning if forced to continue

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agopvecm: exec corosync commands directly
Thomas Lamprecht [Wed, 19 Feb 2020 08:32:07 +0000 (09:32 +0100)]
pvecm: exec corosync commands directly

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoAdd verification and fallback to cluster join/addnode
Stefan Reiter [Thu, 9 Jan 2020 15:31:35 +0000 (16:31 +0100)]
Add verification and fallback to cluster join/addnode

Verify that the config of the new node is valid and compatible with the
cluster (i.e. that the links for the new node match the currently
configured nodes).

Additionally, fallback is provided via a new parameter to addnode,
'new_node_ip'. Previously, fallback was handled on the joining node, by
setting it's local IP as 'link0', however, a cluster with only one link,
but numbered 1-7 is still valid, and a fallback is possible, but the old
code would now fail.

Instead, pass the locally resolved IP via a seperate parameter
(resolving the IP on the cluster side is impractical, as IP resolution
could fail or provide a wrong IP for Corosync).

For compatibility reasons, allow fallback to occur via the old
method as well, but mark with FIXME for future removal.

Fallback fails in case the cluster has more than one link, in this case
only the user can know which NIC/IP corresponds to which cluster link.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agoEnable support for up to 8 corosync links
Stefan Reiter [Thu, 9 Jan 2020 15:31:34 +0000 (16:31 +0100)]
Enable support for up to 8 corosync links

add_corosync_link_properties/extract_corosync_link_args are introduced
as helpers to avoid hardcoding links in parameters=>properties on
several occasions, while still providing autocompletion with pvecm by
being seperate parameters instead of an array.

Maximum number of links is given as constant MAX_LINK_COUNT, should it
change in the future.

All necessary functions have been updated to
use the new $links array format instead of seperate $link0/$link1
parameters, and call sites changed accordingly.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agocorosync: add verify_conf
Stefan Reiter [Thu, 9 Jan 2020 15:31:33 +0000 (16:31 +0100)]
corosync: add verify_conf

It does some basic sanity checking, warns the user about encryption
settings and unresolved hostnames, and finally makes sure that all nodes
have the same links configured (as well as comparing the configured
links to specified interfaces, if there are any).

A corosync.conf that has been created and modified strictly through our
API should *always* be valid.

verify_conf is called in 'addnode', warnings and errors are returned via
the API to be displayed in the task log of the node asking to join. If a
verification error occurs, it is handled specially via a "raise" outside
of any lock code that strips extra information from an Exception
instance. This ensures that multi-line formatted errors can be returned.
Warnings are always returned as array, to be printed on the caller.

Includes testing.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agobump version to 6.1-4
Thomas Lamprecht [Tue, 28 Jan 2020 14:24:48 +0000 (15:24 +0100)]
bump version to 6.1-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agod/control: drop shlibs:Depends for architecture independent packages
Thomas Lamprecht [Tue, 28 Jan 2020 14:26:38 +0000 (15:26 +0100)]
d/control: drop shlibs:Depends for architecture independent packages

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoadd some dev test scripts for evaluating a few IPC calls hands on
Thomas Lamprecht [Mon, 27 Jan 2020 17:15:56 +0000 (18:15 +0100)]
add some dev test scripts for evaluating a few IPC calls hands on

simple stuff and not a real unit or regression test, still useful
when checking how/if something works or benchmarking..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agotoken IPC: mark pointed data as const when only pointer is mutable
Thomas Lamprecht [Mon, 27 Jan 2020 17:23:40 +0000 (18:23 +0100)]
token IPC: mark pointed data as const when only pointer is mutable

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agopmxcfs: add verify_token IPC request
Fabian Grünbichler [Tue, 21 Jan 2020 12:53:57 +0000 (13:53 +0100)]
pmxcfs: add verify_token IPC request

Add a new IPC request which takes a token string and matches it with
the priv/token.cfg shadow file, this allows non-root processes with
the privilege of doing IPC requests, to verify tokens without being
able to read the full token list itself.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
[ Thomas: solved merge conflict in observer files struct ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agobump version to 6.1-3
Thomas Lamprecht [Fri, 17 Jan 2020 09:30:15 +0000 (10:30 +0100)]
bump version to 6.1-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoAdd "cpu-models.conf" to observed files
Stefan Reiter [Thu, 16 Jan 2020 15:40:47 +0000 (16:40 +0100)]
Add "cpu-models.conf" to observed files

Holds information about custom CPU models, section config style. Used in
qemu-server.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agoFix #2553: Prevent the Deadlock by aligning the lockorder
Kevin Greßlehner [Tue, 14 Jan 2020 11:48:28 +0000 (11:48 +0000)]
Fix #2553: Prevent the Deadlock by aligning the lockorder

Overview:
Every once in a while the /etc/pve directory freezes. ("ls" and "df"
does not work) Therefore the most pve components do not work anymore.
(webinterface is not answering, shell commands do not work) This
mostly happens during snapshots, which happen frequently in my case.
The workaround / temporary solution is to restart the pve-cluster
service.

Steps to reproduce:
Make frequent snapshots/snapshot-deletes on a default installation.
The /etc/pve direcetory will freeze at some point.

Cause;
When a snapshot is made, it eventually invokes memdb_rename
(memdb.c:1103), which at first locks the memdb->mutex at memdb.c:1122
and then invokes the methods vmlist_different_vm_exists
(memdb.c:1147) or vmlist_register_vm (memdb.c:1233).  These methods
are defined in status.c and want to lock the mutex lock of
(status.c:689 and status.c:669.

The deadlock appears when cfs_create_guest_conf_propertiy_msg aquires
the status.c mutex lock while memdb_rename aquires the memdb.c mutex
lock at the same time. Then cfs_create_guest_conf_propertiy_msg wants
to lock the memdb.c lock at memdb_read (which is held by
memdb_rename) and vmlist_different_vm_exists or vmlist_register_vm
wants to lock the status.c lock (which is held by
cfs_create_guest_conf_propertiy_msg). Both methods are waiting for
each other to unlock their locks -> deadlock.

Fix:
Fix by aligning the lockorder of the memdb and status mutex lock
calls.

Lock &memdb->mutex in memdb_read and refer to a new method
"memdb_read_nolock" in memdb.c which doesn't handle locks by itself.
This method then handles the stuff which was originally in
memdb_read. Therefore everything except
cfs_create_guest_conf_property_msg uses memdb_read (which handles the
locking itself), and cfs_create_guest_conf_property_msg prelocks
&memdb->mutex and invokes memdb_read_nolock.

Signed-off-by: Kevin Greßlehner <kevin_gressi@live.at>
[ added more info from bug report & fixed indentation/line endings ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agogrammar fix: s/does not exists/does not exist/g
Thomas Lamprecht [Fri, 13 Dec 2019 11:05:28 +0000 (12:05 +0100)]
grammar fix: s/does not exists/does not exist/g

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agodc.cfg verbose description: fix missing whitespace & reword
Thomas Lamprecht [Sat, 30 Nov 2019 14:50:22 +0000 (15:50 +0100)]
dc.cfg verbose description: fix missing whitespace & reword

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agobump version to 6.1-2
Thomas Lamprecht [Wed, 27 Nov 2019 16:19:51 +0000 (17:19 +0100)]
bump version to 6.1-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>