From: Fiona Ebner <f.ebner@proxmox.com>
Date: Fri, 30 Jun 2023 11:59:42 +0000 (+0200)
Subject: pvecm: updatecerts: wait for quorum
X-Git-Url: https://git.proxmox.com/?p=pve-cluster.git;a=commitdiff_plain;h=97dc596c68e8c2db6655c604ef7810fa80125594

pvecm: updatecerts: wait for quorum

Mostly useful for the updatecerts call triggered via the ExecStartPre
hook for pveproxy.service.

When starting a node that's part of a cluster, there is a time window
between the start of pve-cluster.service and when quorum is reached
(from the node's perspective). pveproxy.service is ordered after
pve-cluster.service, but that does not prevent the ExecStartPre hook
from being executed before the node is part of the quorate partition.

The pvecm updatecerts command won't do much without quorum. Generating
local (non-pmxcfs) files is still done before waiting on quorum.

In particular, it might happen that the base directories for observed
files will not get created during/after the upgrade from Proxmox VE 7
to 8 (reported in the community forum [0] and reproduced right away in
a virtual test cluster).

Waiting on quorum should highly increase the chances for successful
execution of the ExecStartPre hook.

[0]: https://forum.proxmox.com/threads/129644/

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---

diff --git a/src/PVE/CLI/pvecm.pm b/src/PVE/CLI/pvecm.pm
index 2d0e046..4bd76f9 100755
--- a/src/PVE/CLI/pvecm.pm
+++ b/src/PVE/CLI/pvecm.pm
@@ -6,6 +6,8 @@ use warnings;
 use Cwd qw(getcwd);
 use File::Path;
 use File::Basename;
+use Time::HiRes qw(usleep);
+
 use PVE::Tools qw(run_command);
 use PVE::Cluster;
 use PVE::INotify;
@@ -576,6 +578,12 @@ __PACKAGE__->register_method ({
 	# no-good for ExecStartPre as it fails the whole service in this case
 	PVE::Tools::run_fork_with_timeout(30, sub {
 	    PVE::Cluster::Setup::generate_local_files();
+
+	    for (my $i = 0; !PVE::Cluster::check_cfs_quorum(1); $i++) {
+		print "waiting for pmxcfs mount to appear and get quorate...\n" if $i % 50 == 0;
+		usleep(100 * 1000);
+	    }
+
 	    PVE::Cluster::Setup::updatecerts_and_ssh($param->@{qw(force silent)});
 	    PVE::Cluster::prepare_observed_file_basedirs();
 	});