1 info 0 hardware: starting simulation
2 info 20 cmdlist: execute power node1 on
3 info 20 node1/crm: status change startup => wait_for_quorum
4 info 20 node1/lrm: status change startup => wait_for_agent_lock
5 info 20 cmdlist: execute power node2 on
6 info 20 node2/crm: status change startup => wait_for_quorum
7 info 20 node2/lrm: status change startup => wait_for_agent_lock
8 info 20 cmdlist: execute power node3 on
9 info 20 node3/crm: status change startup => wait_for_quorum
10 info 20 node3/lrm: status change startup => wait_for_agent_lock
11 info 20 cmdlist: execute power node5 on
12 info 20 node5/crm: status change startup => wait_for_quorum
13 info 20 node5/lrm: status change startup => wait_for_agent_lock
14 info 20 node1/crm: got lock 'ha_manager_lock'
15 info 20 node1/crm: status change wait_for_quorum => master
16 info 20 node1/crm: using scheduler mode 'static'
17 info 20 node1/crm: node 'node1': state changed from 'unknown' => 'online'
18 info 20 node1/crm: node 'node2': state changed from 'unknown' => 'online'
19 info 20 node1/crm: node 'node3': state changed from 'unknown' => 'online'
20 info 20 node1/crm: node 'node5': state changed from 'unknown' => 'online'
21 info 20 node1/crm: adding new service 'vm:102' on node 'node1'
22 info 21 node1/lrm: got lock 'ha_agent_node1_lock'
23 info 21 node1/lrm: status change wait_for_agent_lock => active
24 info 21 node1/lrm: starting service vm:102
25 info 21 node1/lrm: service status vm:102 started
26 info 22 node2/crm: status change wait_for_quorum => slave
27 info 24 node3/crm: status change wait_for_quorum => slave
28 info 26 node5/crm: status change wait_for_quorum => slave
29 info 120 cmdlist: execute power node1 off
30 info 120 node1/crm: killed by poweroff
31 info 120 node1/lrm: killed by poweroff
32 info 220 cmdlist: execute delay 300
33 info 222 node3/crm: got lock 'ha_manager_lock'
34 info 222 node3/crm: status change slave => master
35 info 222 node3/crm: using scheduler mode 'static'
36 info 222 node3/crm: node 'node1': state changed from 'online' => 'unknown'
37 info 282 node3/crm: service 'vm:102': state changed from 'started' to 'fence'
38 info 282 node3/crm: node 'node1': state changed from 'unknown' => 'fence'
39 emai 282 node3/crm: FENCE: Try to fence node 'node1'
40 info 282 node3/crm: got lock 'ha_agent_node1_lock'
41 info 282 node3/crm: fencing: acknowledged - got agent lock for node 'node1'
42 info 282 node3/crm: node 'node1': state changed from 'fence' => 'unknown'
43 emai 282 node3/crm: SUCCEED: fencing: acknowledged - got agent lock for node 'node1'
44 info 282 node3/crm: service 'vm:102': state changed from 'fence' to 'recovery'
45 info 282 node3/crm: recover service 'vm:102' from fenced node 'node1' to node 'node3'
46 info 282 node3/crm: service 'vm:102': state changed from 'recovery' to 'started' (node = node3)
47 info 283 node3/lrm: got lock 'ha_agent_node3_lock'
48 info 283 node3/lrm: status change wait_for_agent_lock => active
49 info 283 node3/lrm: starting service vm:102
50 info 283 node3/lrm: service status vm:102 started
51 info 600 cmdlist: execute power node1 on
52 info 600 node1/crm: status change startup => wait_for_quorum
53 info 600 node1/lrm: status change startup => wait_for_agent_lock
54 info 600 node1/crm: status change wait_for_quorum => slave
55 info 604 node3/crm: node 'node1': state changed from 'unknown' => 'online'
56 info 604 node3/crm: migrate service 'vm:102' to node 'node1' (running)
57 info 604 node3/crm: service 'vm:102': state changed from 'started' to 'migrate' (node = node3, target = node1)
58 info 605 node3/lrm: service vm:102 - start migrate to node 'node1'
59 info 605 node3/lrm: service vm:102 - end migrate to node 'node1'
60 info 624 node3/crm: service 'vm:102': state changed from 'migrate' to 'started' (node = node1)
61 info 641 node1/lrm: got lock 'ha_agent_node1_lock'
62 info 641 node1/lrm: status change wait_for_agent_lock => active
63 info 641 node1/lrm: starting service vm:102
64 info 641 node1/lrm: service status vm:102 started
65 info 700 cmdlist: execute delay 300
66 info 1080 cmdlist: execute power node4 on
67 info 1080 node4/crm: status change startup => wait_for_quorum
68 info 1080 node4/lrm: status change startup => wait_for_agent_lock
69 info 1084 node3/crm: node 'node4': state changed from 'unknown' => 'online'
70 info 1086 node4/crm: status change wait_for_quorum => slave
71 info 1180 cmdlist: execute power node1 off
72 info 1180 node1/crm: killed by poweroff
73 info 1180 node1/lrm: killed by poweroff
74 info 1182 node3/crm: node 'node1': state changed from 'online' => 'unknown'
75 info 1242 node3/crm: service 'vm:102': state changed from 'started' to 'fence'
76 info 1242 node3/crm: node 'node1': state changed from 'unknown' => 'fence'
77 emai 1242 node3/crm: FENCE: Try to fence node 'node1'
78 info 1280 cmdlist: execute delay 300
79 info 1282 node3/crm: got lock 'ha_agent_node1_lock'
80 info 1282 node3/crm: fencing: acknowledged - got agent lock for node 'node1'
81 info 1282 node3/crm: node 'node1': state changed from 'fence' => 'unknown'
82 emai 1282 node3/crm: SUCCEED: fencing: acknowledged - got agent lock for node 'node1'
83 info 1282 node3/crm: service 'vm:102': state changed from 'fence' to 'recovery'
84 info 1282 node3/crm: recover service 'vm:102' from fenced node 'node1' to node 'node4'
85 info 1282 node3/crm: service 'vm:102': state changed from 'recovery' to 'started' (node = node4)
86 info 1285 node4/lrm: got lock 'ha_agent_node4_lock'
87 info 1285 node4/lrm: status change wait_for_agent_lock => active
88 info 1285 node4/lrm: starting service vm:102
89 info 1285 node4/lrm: service status vm:102 started
90 info 1660 cmdlist: execute power node1 on
91 info 1660 node1/crm: status change startup => wait_for_quorum
92 info 1660 node1/lrm: status change startup => wait_for_agent_lock
93 info 1660 node1/crm: status change wait_for_quorum => slave
94 info 1664 node3/crm: node 'node1': state changed from 'unknown' => 'online'
95 info 1664 node3/crm: migrate service 'vm:102' to node 'node1' (running)
96 info 1664 node3/crm: service 'vm:102': state changed from 'started' to 'migrate' (node = node4, target = node1)
97 info 1667 node4/lrm: service vm:102 - start migrate to node 'node1'
98 info 1667 node4/lrm: service vm:102 - end migrate to node 'node1'
99 info 1684 node3/crm: service 'vm:102': state changed from 'migrate' to 'started' (node = node1)
100 info 1701 node1/lrm: got lock 'ha_agent_node1_lock'
101 info 1701 node1/lrm: status change wait_for_agent_lock => active
102 info 1701 node1/lrm: starting service vm:102
103 info 1701 node1/lrm: service status vm:102 started
104 info 1760 cmdlist: execute delay 300
105 info 1825 node3/lrm: node had no service configured for 60 rounds, going idle.
106 info 1825 node3/lrm: status change active => wait_for_agent_lock
107 info 2140 cmdlist: execute power node2 off
108 info 2140 node2/crm: killed by poweroff
109 info 2140 node2/lrm: killed by poweroff
110 info 2142 node3/crm: node 'node2': state changed from 'online' => 'unknown'
111 info 2240 cmdlist: execute power node1 off
112 info 2240 node1/crm: killed by poweroff
113 info 2240 node1/lrm: killed by poweroff
114 info 2240 node3/crm: node 'node1': state changed from 'online' => 'unknown'
115 info 2300 node3/crm: service 'vm:102': state changed from 'started' to 'fence'
116 info 2300 node3/crm: node 'node1': state changed from 'unknown' => 'fence'
117 emai 2300 node3/crm: FENCE: Try to fence node 'node1'
118 info 2340 cmdlist: execute delay 300
119 info 2360 node3/crm: got lock 'ha_agent_node1_lock'
120 info 2360 node3/crm: fencing: acknowledged - got agent lock for node 'node1'
121 info 2360 node3/crm: node 'node1': state changed from 'fence' => 'unknown'
122 emai 2360 node3/crm: SUCCEED: fencing: acknowledged - got agent lock for node 'node1'
123 info 2360 node3/crm: service 'vm:102': state changed from 'fence' to 'recovery'
124 info 2360 node3/crm: recover service 'vm:102' from fenced node 'node1' to node 'node4'
125 info 2360 node3/crm: service 'vm:102': state changed from 'recovery' to 'started' (node = node4)
126 info 2363 node4/lrm: starting service vm:102
127 info 2363 node4/lrm: service status vm:102 started
128 info 2720 cmdlist: execute power node1 on
129 info 2720 node1/crm: status change startup => wait_for_quorum
130 info 2720 node1/lrm: status change startup => wait_for_agent_lock
131 info 2720 node1/crm: status change wait_for_quorum => slave
132 info 2722 node3/crm: node 'node1': state changed from 'unknown' => 'online'
133 info 2722 node3/crm: migrate service 'vm:102' to node 'node1' (running)
134 info 2722 node3/crm: service 'vm:102': state changed from 'started' to 'migrate' (node = node4, target = node1)
135 info 2725 node4/lrm: service vm:102 - start migrate to node 'node1'
136 info 2725 node4/lrm: service vm:102 - end migrate to node 'node1'
137 info 2742 node3/crm: service 'vm:102': state changed from 'migrate' to 'started' (node = node1)
138 info 2761 node1/lrm: got lock 'ha_agent_node1_lock'
139 info 2761 node1/lrm: status change wait_for_agent_lock => active
140 info 2761 node1/lrm: starting service vm:102
141 info 2761 node1/lrm: service status vm:102 started
142 info 2820 cmdlist: execute delay 300
143 info 3200 cmdlist: execute power node2 on
144 info 3200 node2/crm: status change startup => wait_for_quorum
145 info 3200 node2/lrm: status change startup => wait_for_agent_lock
146 info 3202 node2/crm: status change wait_for_quorum => slave
147 info 3204 node3/crm: node 'node2': state changed from 'unknown' => 'online'
148 info 3300 cmdlist: execute power node3 off
149 info 3300 node3/crm: killed by poweroff
150 info 3300 node3/lrm: killed by poweroff
151 info 3400 cmdlist: execute power node1 off
152 info 3400 node1/crm: killed by poweroff
153 info 3400 node1/lrm: killed by poweroff
154 info 3420 node2/crm: got lock 'ha_manager_lock'
155 info 3420 node2/crm: status change slave => master
156 info 3420 node2/crm: using scheduler mode 'static'
157 info 3420 node2/crm: node 'node1': state changed from 'online' => 'unknown'
158 info 3420 node2/crm: node 'node3': state changed from 'online' => 'unknown'
159 info 3480 node2/crm: service 'vm:102': state changed from 'started' to 'fence'
160 info 3480 node2/crm: node 'node1': state changed from 'unknown' => 'fence'
161 emai 3480 node2/crm: FENCE: Try to fence node 'node1'
162 info 3520 node2/crm: got lock 'ha_agent_node1_lock'
163 info 3520 node2/crm: fencing: acknowledged - got agent lock for node 'node1'
164 info 3520 node2/crm: node 'node1': state changed from 'fence' => 'unknown'
165 emai 3520 node2/crm: SUCCEED: fencing: acknowledged - got agent lock for node 'node1'
166 info 3520 node2/crm: service 'vm:102': state changed from 'fence' to 'recovery'
167 info 3520 node2/crm: recover service 'vm:102' from fenced node 'node1' to node 'node4'
168 info 3520 node2/crm: service 'vm:102': state changed from 'recovery' to 'started' (node = node4)
169 info 3523 node4/lrm: starting service vm:102
170 info 3523 node4/lrm: service status vm:102 started
171 info 4000 hardware: exit simulation - done
projects / pve-ha-manager.git / blob