For now we have two important resources types - virtual machines and
containers. One basic idea here is that we can bundle related software
For now we have two important resources types - virtual machines and
containers. One basic idea here is that we can bundle related software
service from other services, like it was done with `rgmanager`. In
general, a HA managed resource should not depend on other resources.
service from other services, like it was done with `rgmanager`. In
general, a HA managed resource should not depend on other resources.
The HA stack now tries to start the resources and keeps it
running. Please note that you can configure the ``requested''
The HA stack now tries to start the resources and keeps it
running. Please note that you can configure the ``requested''
LRM only executes actions when it holds its lock, we can mark a failed node
as fenced if we can acquire its lock. This lets us then recover any failed
HA services securely without any interference from the now unknown failed node.
LRM only executes actions when it holds its lock, we can mark a failed node
as fenced if we can acquire its lock. This lets us then recover any failed
HA services securely without any interference from the now unknown failed node.
After the LRM gets in the active state it reads the manager status
file in `/etc/pve/ha/manager_status` and determines the commands it
has to execute for the services it owns.
After the LRM gets in the active state it reads the manager status
file in `/etc/pve/ha/manager_status` and determines the commands it
has to execute for the services it owns.
parallel and are limited to at most 4 by default. This default setting
may be changed through the datacenter configuration key `max_worker`.
When finished the worker process gets collected and its result saved for
parallel and are limited to at most 4 by default. This default setting
may be changed through the datacenter configuration key `max_worker`.
When finished the worker process gets collected and its result saved for
a specific setup. For example may 4 live migrations happen at the same
time, which can lead to network congestions with slower networks and/or
big (memory wise) services. Ensure that also in the worst case no congestion
a specific setup. For example may 4 live migrations happen at the same
time, which can lead to network congestions with slower networks and/or
big (memory wise) services. Ensure that also in the worst case no congestion
-Each command requested by the CRM is uniquely identifiable by an UID, when
-the worker finished its result will be processed and written in the LRM
+Each command requested by the CRM is uniquely identifiable by a UID, when
+the worker finishes its result will be processed and written in the LRM
status file `/etc/pve/nodes/<nodename>/lrm_status`. There the CRM may collect
it and let its state machine - respective the commands output - act on it.
The actions on each service between CRM and LRM are normally always synced.
status file `/etc/pve/nodes/<nodename>/lrm_status`. There the CRM may collect
it and let its state machine - respective the commands output - act on it.
The actions on each service between CRM and LRM are normally always synced.
then executes this action *one time* and writes back the result, also
identifiable by the same UID. This is needed so that the LRM does not
then executes this action *one time* and writes back the result, also
identifiable by the same UID. This is needed so that the LRM does not
With the exception of the `stop` and the `error` command,
those two do not depend on the result produced and are executed
always in the case of the stopped state and once in the case of
With the exception of the `stop` and the `error` command,
those two do not depend on the result produced and are executed
always in the case of the stopped state and once in the case of
available and try to always enforce the requested state. For example, a
service with the requested state 'started' will be started if its not
already running. If it crashes it will be automatically started again.
available and try to always enforce the requested state. For example, a
service with the requested state 'started' will be started if its not
already running. If it crashes it will be automatically started again.
When an node leaves the cluster quorum, its state changes to unknown.
If the current CRM then can secure the failed nodes lock, the services
When an node leaves the cluster quorum, its state changes to unknown.
If the current CRM then can secure the failed nodes lock, the services
The resource configuration file `/etc/pve/ha/resources.cfg` stores
the list of resources managed by `ha-manager`. A resource configuration
The resource configuration file `/etc/pve/ha/resources.cfg` stores
the list of resources managed by `ha-manager`. A resource configuration
really important task, because without, it would not be possible to
recover a resource on another node.
really important task, because without, it would not be possible to
recover a resource on another node.
it may have still access to shared resources. This is really
dangerous! Imagine that every network but the storage one broke. Now,
while not reachable from the public network, the VM still runs and
it may have still access to shared resources. This is really
dangerous! Imagine that every network but the storage one broke. Now,
while not reachable from the public network, the VM still runs and
should be triggered on the same node and how often a service should be
relocated so that it gets a try to be started on another node.
The aim of this policy is to circumvent temporary unavailability of shared
should be triggered on the same node and how often a service should be
relocated so that it gets a try to be started on another node.
The aim of this policy is to circumvent temporary unavailability of shared