replication: update last_sync before removing old replication snapshots

If pvesr was terminated after finishing with the new sync and after
removing old replication snapshots, but before it could write the new
state, the next replication would fail. It would wrongly interpret the
actual last replication snapshot as stale, remove it, and (if no other
snapshots are present) attempt a full sync, which would fail.

Reported in the community forum [0], this was brought to light by the
new pvescheduler before it learned graceful reload.

It's not possible to simply preserve a last remaining snapshot in
prepare(), because prepare() is also used for valid removals. Instead,
update last_sync early enough. Stale snapshots will still be removed
on the next run if there are any.

[0]: https://forum.proxmox.com/threads/100154

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>

diff --git a/src/PVE/Replication.pm b/src/PVE/Replication.pm
index 051dfd950e9e838baa4318cf8668d548faa9b9b4..de652f26a94becbf3912e4725123cf1d18995fb3 100644 (file)
--- a/src/PVE/Replication.pm
+++ b/src/PVE/Replication.pm
@@ -372,6 +372,9 @@ sub replicate {
        die $err;
     }
 
+    # Ensure that new sync is recorded before removing old replication snapshots.
+    PVE::ReplicationState::record_sync_end($jobcfg, $state, $start_time);
+
     # remove old snapshots because they are no longer needed
     $cleanup_local_snapshots->($last_snapshots, $last_sync_snapname);
 

diff --git a/src/PVE/ReplicationState.pm b/src/PVE/ReplicationState.pm
index 81a1b31381f371f70139cc7812f0b76c3602979f..8efe0e26f25fb641569531df1d02d35438c0c27b 100644 (file)
--- a/src/PVE/ReplicationState.pm
+++ b/src/PVE/ReplicationState.pm
@@ -159,6 +159,13 @@ sub delete_guest_states {
     PVE::Tools::lock_file($state_lock, 10, $code);
 }
 
+sub record_sync_end {
+    my ($jobcfg, $state, $start_time) = @_;
+
+    $state->{last_sync} = $start_time;
+    write_job_state($jobcfg, $state);
+}
+
 sub record_job_end {
     my ($jobcfg, $state, $start_time, $duration, $err) = @_;