+We thus wanted to integrate a simpler method in the HA Manager first, namely
+self fencing with watchdogs.
+
+Watchdogs are widely used in critical and dependable systems since the
+beginning of micro controllers, they are often independent and simple
+integrated circuit which programs can use to watch them. After opening they need to
+report periodically. If, for whatever reason, a program becomes unable to do
+so the watchdogs triggers a reset of the whole server.
+
+Server motherboards often already include such hardware watchdogs, these need
+to be configured. If no watchdog is available or configured we fall back to the
+Linux Kernel softdog while still reliable it is not independent of the servers
+Hardware and thus has a lower reliability then a hardware watchdog.
+
+Configure Hardware Watchdog
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+By default all watchdog modules are blocked for security reasons as they are
+like a loaded gun if not correctly initialized.
+If you have a hardware watchdog available remove its kernel module from the
+blacklist, load it with insmod and restart the `watchdog-mux` service or reboot
+the node.
+
+Recover Fenced Services
+~~~~~~~~~~~~~~~~~~~~~~~
+
+After a node failed and its fencing was successful we start to recover services
+to other available nodes and restart them there so that they can provide service
+again.
+
+The selection of the node on which the services gets recovered is influenced
+by the users group settings, the currently active nodes and their respective
+active service count.
+First we build a set out of the intersection between user selected nodes and
+available nodes. Then the subset with the highest priority of those nodes
+gets chosen as possible nodes for recovery. We select the node with the
+currently lowest active service count as a new node for the service.
+That minimizes the possibility of an overload, which else could cause an
+unresponsive node and as a result a chain reaction of node failures in the
+cluster.
+
+[[ha_manager_groups]]