Jan Friesse [Fri, 30 Jul 2021 12:40:15 +0000 (14:40 +0200)]
totemconfig: Do not process totem.nodeid
totem.nodeid is relict from times when nodelist was not required and
totemsrp was sending whole membership with ip addresses.
With Corosync 3 ip addresses are no longer sent so
it is not possible to find "next" node ip address where to send token
(because only nodeid is sent) without having information about all of
the nodes stored locally.
When totem.nodeid was configured it was partly used and other parts
(most notably totemudpu_token_target_set) were using autogenerated
nodeid. Together it was not possible to create even single node
membership.
Solution is to ignore totem.nodeid completely (and display warning when
it is set).
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Currently if there is a gap in the links (eg link0 is missing)
corosync-cfgtool -s will still display the links as 0,1,2,3...
even if they are 1,2,5,6...
Also display the KNET transport type with the link in
corosync-cfgtool -s & -n
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Mon, 3 May 2021 13:29:04 +0000 (15:29 +0200)]
main: Add support for cgroup v2 and auto mode
Support for cgroup v2 is very similar to cgroup v1 just checking (and
writing) different file.
Because of all the problems described later with cgroup v2 new "auto"
mode (new default) is added. This mode first tries to set rr scheduling
and moves Corosync to root cgroup only if it fails.
Testing this feature is a bit harder than with cgroup v1 so it's
probably worh noting in this commit message.
1. Copy some service file (I've used httpd service) and set
CPUQuota=30% in the [service] section.
2. Check /sys/fs/cgroup/cgroup.subtree_control - there should be no
"cpu"
3. Start modified service
4. Check /sys/fs/cgroup/cgroup.subtree_control - there should be "cpu"
5. Start corosync - It should be able to get rt priority
When move_to_root_cgroup is disabled (applies only for kernels
with CONFIG_RT_GROUP_SCHED enabled), behavior differs:
- If corosync is started before modified service, so
there is no "cpu" in /sys/fs/cgroup/cgroup.subtree_control
corosync starts without problem and gets rt priority.
Starting modified service later will never add "cpu" into
/sys/fs/cgroup/cgroup.subtree_control (because corosync is holding
rt priority and it is placed in the non-root cgroup by systemd).
- When corosync is started after modified service, so "cpu"
is in /sys/fs/cgroup/cgroup.subtree_control, corosync is not
able to get RT priority.
It's worth noting problems when cgroup v2 is used together with systemd
logging described in corosync.conf(5) man page.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
The libqb map API leaves 'ownership' of the data with the caller
but does its own lifetime management, so it can easily happen that
map_rm() is called and the data deleted by the caller.
But if an iterator is running over that item then the map entry
will not get removed (leaving dangling pointers) until later.
libqb has a hack-y callback that tells the owner when it is safe to
delete the allocated memory, so we hook into that. icmap is already
using this.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Wed, 2 Jun 2021 10:29:31 +0000 (12:29 +0200)]
man: Add note about single node configuration
Internally knet is using just one link for localhost so for single node
configuration knet_link_get_link_list returns only one entry. This is
propagated to `corosync-cfgtool -s`.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Mon, 3 May 2021 13:29:04 +0000 (15:29 +0200)]
main: Add support for cgroup v2
Support for cgroup v2 is very similar to cgroup v1 just checking (and
writing) different file.
Testing this feature is a bit harder than with cgroup v1 so it's
probably worh noting in this commit message.
1. Copy some service file (I've used httpd service) and set
CPUQuota=30% in the [service] section.
2. Check /sys/fs/cgroup/cgroup.subtree_control - there should be no
"cpu"
3. Start modified service
4. Check /sys/fs/cgroup/cgroup.subtree_control - there should be "cpu"
5. Start corosync - It should be able to get rt priority
When move_to_root_cgroup is disabled, behavior differs:
- If corosync is started before modified service, so
there is no "cpu" in /sys/fs/cgroup/cgroup.subtree_control
corosync starts without problem and gets rt priority.
Starting modified service later will never add "cpu" into
/sys/fs/cgroup/cgroup.subtree_control (because corosync is holding
rt priority and it is placed in the non-root cgroup by systemd).
- When corosync is started after modified service, so "cpu"
is in /sys/fs/cgroup/cgroup.subtree_control, corosync is not
able to get RT priority.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Fri, 9 Apr 2021 10:21:15 +0000 (12:21 +0200)]
config: Properly check crypto and compress models
Use knet_get_crypto_list to find knet supported crypto models and use
them instead of hardcoded list.
Also fix compression handling. Previously knet_compression_model
value was not checked at all and was directly passed to knet.
Use knet_get_compress_list to find knet supported compress models and
use them to check validity of config file and for more informative
error message.
Lastly enhance corosync version display with information
about available crypto/compression models.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
totemknet_configure_compression was using knet_context
just to gather the knet handle / instance.
On first time config knet_contex is not initialized till
much later in the code, passing some random garbage pointers
to knet_handle_compress, that would crash later trying
to acquire a mutex lock.
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Dan Streetman [Fri, 12 Mar 2021 12:31:47 +0000 (07:31 -0500)]
totemknet: retry knet_handle_new if it fails
Retry knet_handle_new without privileged operations if it fails
knet_handle_new can fail with ENAMETOOLONG if its privileged operations
fail, which can happen if we're running as a user process or in an
unprivileged container.
This adds a cmap key 'allow_knet_handle_fallback' that defaults to no,
which is the current behavior of exiting with error if the knet_handle
can't be created with privileged operations. If the new cmap key is set
to 'yes' and the knet_handle creation fails, fallback to creating the
handle using unprivileged operations is tried.
Signed-off-by: Dan Streetman <ddstreet@canonical.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Dan Streetman [Wed, 10 Mar 2021 17:48:01 +0000 (12:48 -0500)]
main: Check memlock rlimit
Don't lock all current and future memory if can't
increase memlock rlimit.
If we fail to increase our RLIMIT_MEMLOCK, then locking all our current
and future memory is extremely dangerous; once our memory use reaches
our RLIMIT_MEMLOCK, memory allocations will start failing, very likely
leading to our entire process crashing.
This can happen if we aren't a privileged process, for example if
running as non-root user, or inside an unprivileged container.
Signed-off-by: Dan Streetman <ddstreet@canonical.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
CFG tracking was removed in 815375411e80131f31b172d7c43625769ee8b53d,
probably as a mistake, as part of the tidy up of cfg and the removal of
dynamic loading. This means that shutdown tracking (using
cfg_try_shutdown()) stopped working.
This patch restores the trackstart & trackstop API calls (renamed to be
more consistent with the exiting libraries) so that shutdown tracking
can be used again.
Change cfg.shutdown_timeout to be in milliseconds rather than seconds
nd use libqb macros for conversion.
Add --force option to corosync-cfgtool -H
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Tue, 24 Nov 2020 11:20:25 +0000 (12:20 +0100)]
cfg: Improve nodestatusget versioning
Patch tries to make nodestatusget really extendable. Following changes
are implemented:
- corosync_cfg_node_status_version_t is added with (for now) single
value CFG_NODE_STATUS_V1
- corosync_knet_node_status renamed to corosync_cfg_node_status_v1 (it
isn't really knet because it works as well for udp(u()
- struct res_lib_cfg_nodestatusget_version is added which holds only ipc
result header and version on same position as for
corosync_cfg_node_status_v1
- corosync_cfg_node_status_get requires version and pointer to one of
corosync_cfg_node_status_v structures
- request is handled in case switches to make adding new version easier
Also fix following bugs:
- totempg_nodestatus_get error was retyped to cs_error_t without any
meaning.
- header.error was not checked at all in the library
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Tue, 10 Nov 2020 17:10:17 +0000 (18:10 +0100)]
totemknet: Check both cipher and hash for crypto
Previously only crypto cipher was used as a way to find out if crypto is
enabled or disabled.
This usually works ok until cipher is set to none and hash to some other
value (like sha1). Such config is perfectly valid and it was not
supported correctly.
As a solution, check both cipher and hash.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Ferenc Wágner [Sun, 8 Nov 2020 19:49:15 +0000 (20:49 +0100)]
The ring id file needn't be executable
At the same time simplify the overwrite logic and stop clearing the
umask (which is unexpected and quite pointless here, as applications
can't really protect the users from their own pathological settings).
Signed-off-by: Ferenc Wágner <wferi@debian.org> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Mon, 12 Oct 2020 12:26:10 +0000 (14:26 +0200)]
config: Increase default token timeout to 3000 ms
Default token timeout of 1000 ms was often changed by users because of
other workloads on machine which may make corosync responding a bit
later than needed and resulting in token loss.
3000 ms was chosen as a compromise between token timeout increase
and allow live cluster upgrade (other nodes should receive token
by node with new default on time).
It doesn't affect token token_coefficient so final token timeout still
depends on number of configured nodes (just base is higher).
This change slows down failover a bit so for clusters where failover
times are important, please change the token timeout in configuration
file corosync.conf as a:
totem {
version: 2
token: 1000
...
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Tue, 29 Sep 2020 16:44:44 +0000 (18:44 +0200)]
tests: Use CS_DISPATCH_BLOCKING instead of cycle
Some tests were using dispatch function in CS_DISPATCH_ALL mode
without poll/select on fd. This leads to busywait cycle, because
CS_DISPATCH_ALL masks CS_ERR_TRY_AGAIN error.
Simpliest solution is to use CS_DISPATCH_BLOCKING instead and remove
while cycle, because CS_DISPATCH_BLOCKING handles CS_ERR_TRY_AGAIN
correctly.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Wed, 9 Sep 2020 12:16:55 +0000 (14:16 +0200)]
quorum: Add support for nodelist callback
Current quorum callback contains only actual view list and there is no
way how to find out joined/left nodes. This cannot be emulated by user
app, because when corosync restarts before other nodes notices then view
list is unchanged (ring id is changed tho).
Solution is to implement similar callback as for cpg which contains ring
id, member list, joined list and left list.
To implement such callback and keep backwards compatibility,
quorum_model_initialize is introduced. Its behavior is similar to
cpg_model_initialize. This allows passing model v1, which contains
enhanced quorum (full ring id is passed instead of just seq number)
and nodelist callbacks.
To find out which events should be sent by corosync daemon, new message
MESSAGE_REQ_QUORUM_MODEL_GETTYPE is used. Quorum library on init was
sending MESSAGE_REQ_QUORUM_GETTYPE. Whem model v1 is requested the
MESSAGE_REQ_QUORUM_MODEL_GETTYPE is used, which contains model number
so corosync knows that client is using model v1 and can send enhanced
quorum and nodelist events.
Nodelist event is (for now) send both in case of change of membership
and also when requested, also when CS_TRACK_CURRENT is requested, but
then left_list and joined_list is left empty, because they don't make
too much sense there.
New test application testquorummodel is added as an example of new API
usage.
Also during patch developement, I found few bugs here and there, which
are also fixed:
- quorum_initialize was never returning error code returned by
MESSAGE_REQ_QUORUM_GETTYPE call (always returned CS_OK)
- Allocated memory in send_library_notification was based
on sizeof(unsigned int) instead of mar_uint32_t. That's not wrong,
but it make more sense to use sizeof(mar_uint32_t) instead
(big thanks to Chrissie for englishify the man pages)
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
liangxin1300 [Fri, 7 Aug 2020 16:50:29 +0000 (00:50 +0800)]
tools: use util_strtonum for options checking
Function atoi is not safe since miss validation;
Function strtol is better but need to consider empty string and overflows
Function util_strtonum is a safer wrapper of strtoll
Use util_strtonum to check nodeid option and strict checking condition.
Signed-off-by: liangxin1300 <XLiang@suse.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Thu, 16 Jul 2020 13:50:42 +0000 (15:50 +0200)]
build: Use git-version-gen during specfile build
Instead of copying parts of git-version-gen for spec target use
git-version-gen directly and parse final version into components
(rpmver, alphatag, numcomm) and use them.
Main reason is to simplify code a bit (sed scripts are a bit repetitive
tho), reuse the code and also allow building of RPM from dist tarball
generated from non-tagged commit or dirty git (not very useful).
The code relies on fact, that hyphen is never used in tagged release
name.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
If it's not available, then fall back to the old
API and forbid changing crypto while running.
To avoid us being dependant on the leader node, each
node sends its own crypto_reconfig_phase messages so
we can guarantee that the reconfiguration always completes
on each node.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
The basic plan here is to fix reloads to be more stable
using read/parse/verify/build/commit stages, so that any errors
will not leave corosync in an unstable state. This should
also make the code more maintainable as currently the verify/commit
stages are horribly intertwined.
Also:
- Fix local_node_pos not being updated in the new map during validation
(broke adding and removing new nodes in the middle of the list).
- Fix reconfiguration so that nodes are indexed by nodeid and not their
position in the list. This is an old bug that's just been carried
over
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Wed, 22 Apr 2020 11:28:57 +0000 (13:28 +0200)]
Revert "totemip: Add support for sin6_scope_id"
This reverts commit 934c47ed4384daf2819c26306bebba3225807499 which is
causing protocol incompatibility in needle. Master seems to be not
affected, but it needs more checking.
Jan Friesse [Tue, 10 Mar 2020 16:49:27 +0000 (17:49 +0100)]
votequorum: set wfa status only on startup
Previously reload of configuration with enabled wait_for_all result in
set of wait_for_all_status which set cluster_is_quorate to 0 but didn't
inform the quorum service so votequorum and quorum information may get
out of sync.
Example is 1 node cluster, which is extended to 3 nodes. Quorum service
reports cluster as a quorate (incorrect) and votequorum as not-quorate
(correct). Similar behavior happens when extending cluster in general,
but some configurations are less incorrect (3->4).
Discussed solution was to inform quorum service but that would mean
every reload would cause loss of quorum until all nodes would be seen
again.
Such behaviour is consistent but seems to be a bit too strict.
Proposed solution sets wait_for_all_status only on startup and
doesn't touch it during reload.
This solution fulfills requirement of "cluster will be quorate for
the first time only after all nodes have been visible at least
once at the same time." because node clears wait_for_all_status only
after it sees all other nodes or joins cluster which is quorate. It also
solves problem with extending cluster, because when cluster becomes
unquorate (1->3) wait_for_all_status is set.
Added assert is only for ensure that I haven't missed any case when
quorate cluster may become unquorate.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Wed, 4 Mar 2020 10:42:15 +0000 (11:42 +0100)]
votequorum: Change check of expected_votes
Previously value of new expected_votes was checked so newly computed
quorum value was in the interval <total_votes / 2, total_votes>. The
upper range prevented the cluster to become unquorate, but bottom check
was almost useless because it allowed to change expected_votes so it is
smaller than total_votes.
Solution is to check if expected_votes is bigger or equal to total_votes
and for quorate cluster only check if cluster doesn't become unquorate
(for unquorate cluster one can set upper range freely - as it is
perfectly possible when using config file)
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Tue, 25 Feb 2020 14:17:05 +0000 (15:17 +0100)]
man: Enhance link_mode priority description
Some users found description of priority for passive link_mode
confusing (probably because "priority" word is too
overloaded) so add some redundancy to make description
unambiguous.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
liangxin1300 [Thu, 20 Feb 2020 05:38:49 +0000 (13:38 +0800)]
totemip: compare sin6_scope_id and interface_num
When user configure a specific interface like vlan
with the same IPv6 link-local address, Corosync should
compare sin6_scope_id with interface_num, to make sure got
the right interface to bind
Signed-off-by: liangxin1300 <XLiang@suse.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Fri, 14 Feb 2020 08:42:08 +0000 (09:42 +0100)]
totemip: Add support for sin6_scope_id
sin6_scope_id was not present in totemip structure making impossible to
use link-local ipv6 address.
Patch adds sin6_scope_id and changes convert/copy functions to use it
(formally also comparator functions should be changed, but it seems to
cause more harm and it is not really needed).
This makes corosync work with link-local addresses fine for both UDPU
and Knet transport as long as interface specification is used (so
fe80::xxxx:xxxx:xxxx:xxxx%eth0).
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Mon, 10 Feb 2020 15:13:42 +0000 (16:13 +0100)]
cfgtool: Improve link status display
Totemknet is enhanced to use 'n' character for localhost and not adding
status, because it is safe to expect that localhost link is always
connectd. corosync-cfgtool is enhanced to properly decode 'n', '?' and
'd' characters and display its meaning for extended status. Special
characters are also documented in man page.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
This patch add a stats.schedmiss.* set of entries that
are a record of the last 10 times corosync was not scheduled
in time.
These entries are keypt in reverse order (so stats.schedmiss.0.* is
always the latest one kept) and the values, including the timestamp,
are in milliseconds.
It's also possible to use a cmap tracker to follow these events, which
might be useful.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Thu, 16 Jan 2020 14:43:59 +0000 (15:43 +0100)]
votequorum: Reflect runtime change of 2Node to WFA
When 2Node mode is set, WFA is also set unless WFA is configured
explicitly. This behavior was not reflected on runtime change, so
restarted corosync behavior was different (WFA not set). Also when
cluster is reduced from 3 nodes to 2 nodes during runtime, WFA was not
set, what may result in two quorate partitions.
Solution is to set WFA depending on 2Node when WFA
is not explicitly configured.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
projects / mirror_corosync.git / log