rhel-systemd: Restart openvswitch service if a daemon crashes

rhel-systemd: Restart openvswitch service if a daemon crashes

Currently if either ovsdb-server or ovs-vswitchd is crashing the
daemon is not restarting leaving the system in faulty state.
This patch will detect the daemon crash and will restart the
openvswitch service.

Here is a (bit to wide) table showing the behavior before and after
the patch. Note that only the Crash behavior has changed:

Before patch:
                               |       Process Status        |             systemctl <> status            |
                               | ovs-vswitchd | ovsdb-server | openvswitch  | ovs-vswitchd | ovsdb-server |
                               +--------------+--------------+--------------+--------------+--------------+
systemctl start openvswitch*   | started      | started      |active,exited |active,running|active,running|
Crash vswitchd                 | crashed      | stopped      |inactive, dead|failed        |inactive,dead |
Crash ovsdb                    | stopped      | crashed      |inactive, dead|inactive,dead |failed        |
systemctl restart openvswitch  | re-started   | re-started   |active,exited |active,running|active,running|
systemctl restart ovs-vswitchd | re-started   | re-started   |active,exited |active,running|active,running|
systemctl restart ovsdb-server | re-started   | re-started   |active,exited |active,running|active,running|
systemctl stop openvswitch     | stopped      | stopped      |inactive, dead|inactive,dead |inactive,dead |
systemctl stop ovs-vswitchd    | stopped      | stopped      |inactive, dead|inactive,dead |inactive,dead |
systemctl stop ovsdb-server    | stopped      | stopped      |inactive, dead|inactive,dead |inactive,dead |
systemctl start ovs-vswitchd*  | started      | started      |inactive, dead|active,running|active,running|
systemctl start ovsdb-server*  | not started  | started      |inactive, dead|inactive, dead|active,running|

With patch:
                               |       Process Status        |             systemctk <> status            |
                               | ovs-vswitchd | ovsdb-server | openvswitch  | ovs-vswitchd | ovsdb-server |
                               +--------------+--------------+--------------+--------------+--------------+
systemctl start openvswitch*   | started      | started      |active,exited |active,running|active,running|
Crash vswitchd                 | crash,started| re-started   |active,exited |active,running|active,running|
Crash ovsdb                    | re-started   | crash,started|active,exited |active,running|active,running|
systemctl restart openvswitch  | re-started   | re-started   |active,exited |active,running|active,running|
systemctl restart ovs-vswitchd | re-started   | re-started   |active,exited |active,running|active,running|
systemctl restart ovsdb-server | re-started   | re-started   |active,exited |active,running|active,running|
systemctl stop openvswitch     | stopped      | stopped      |inactive, dead|inactive,dead |inactive,dead |
systemctl stop ovs-vswitchd    | stopped      | stopped      |inactive, dead|inactive,dead |inactive,dead |
systemctl stop ovsdb-server    | stopped      | stopped      |inactive, dead|inactive,dead |inactive,dead |
systemctl start ovs-vswitchd*  | started      | started      |inactive, dead|active,running|active,running|
systemctl start ovsdb-server*  | not started  | started      |inactive, dead|inactive, dead|active,running|

* These commands where executed when no ovs related processes where
  running. All other commands where executed when OVS was successfully
  running.

Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
Acked-by: Markos Chandras <mchandras@suse.de>
Acked-by: Aaron Conole <aconole@redhat.com>
Acked-by: Flavio Leitner <fbl@sysclose.org>
Signed-off-by: Russell Bryant <russell@ovn.org>