- rename generic.in and notifyd.in to corosync.in and corosync-notifyd.in
(makes build simpler)
- fix sysvinit corosync.in sleep time to include a check for when IPC
are ready and drop cman bits (there is no cman with corosync 2.0)
- corosync-notifyd.service should always start after corosync.service
- corosync.service should always start after network
- corosync.service uses init script wrapper
- install/ship sysvinit as wrappers for systemd in /usr/share/corosync
when necessary
- change the build system to deal with all of the above
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
clean up a lot of allocated blocks at exit.
those changes has no runtime effects, but it makes valgrind
output a bit more useful by dropping over 700 errors/warnings to skip
over every single run.
there are still a few icmap related valgrind errors but those need
some more complex and timeconsuming investigation.
pre patch:
==21844== HEAP SUMMARY:
==21844== in use at exit: 1,229,321 bytes in 1,516 blocks
==21844== total heap usage: 7,191 allocs, 5,675 frees, 3,819,853 bytes allocated
==21844== LEAK SUMMARY:
==21844== definitely lost: 3,617 bytes in 11 blocks
==21844== indirectly lost: 21,960 bytes in 11 blocks
==21844== possibly lost: 1,080,101 bytes in 131 blocks
==21844== still reachable: 123,643 bytes in 1,363 blocks
==21844== suppressed: 0 bytes in 0 blocks
==21844== ERROR SUMMARY: 136 errors from 136 contexts (suppressed: 0 from 0)
post patch:
==25793== HEAP SUMMARY:
==25793== in use at exit: 1,185,870 bytes in 808 blocks
==25793== total heap usage: 9,427 allocs, 8,619 frees, 4,156,841 bytes allocated
==25793== LEAK SUMMARY:
==25793== definitely lost: 3,697 bytes in 12 blocks
==25793== indirectly lost: 22,248 bytes in 13 blocks
==25793== possibly lost: 1,079,655 bytes in 113 blocks
==25793== still reachable: 80,270 bytes in 670 blocks
==25793== suppressed: 0 bytes in 0 blocks
==25793== ERROR SUMMARY: 119 errors from 119 contexts (suppressed: 0 from 0)
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
ipcs: allow connections only after all services are ready
this fixes a rather annoying race condition at startup where a client
connects to corosync "too fast" before the service is ready to operate
and client gets some random data during initialization phase.
With this fix, we allow connections to ipc only after the main engine
is operational and configured (and after the first totem transition).
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Angus Salkeld <asalkeld@redhat.com>
Only call qb_ipcc_disconnect when the instance is fully dereferenced.
Sometimes calling xyz_finilize() within a dispatch would
cause a crash because the qb_ipcc_disconnect actually
disconnects immediatly and frees it't memory. whereas
the corosync structure is reference counted. So this
makes use of the reference counting to only call
qb_ipcc_disconnect when it is fully dereferenced.
Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com> Signed-off-by: Angus Salkeld <asalkeld@redhat.com>
totemcrypto: major code cleanup (no functional or onwire changes)
- cleanup include list
- reorder code and functions (crypto then hash)
- split crypt/decrypt/hash functions
- some micro optimizations by dropping a few memcpy
- make the code more readable (better var names and buffers mapping)
- improve exit paths on error (return codes and free)
- store crypto header size instead of recalculating it per packet
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Mon, 26 Mar 2012 08:30:20 +0000 (10:30 +0200)]
Make ifaces_get work with dynamic no_rings
Commit which added number of addresses to srp_address structure didn't
count with totemsrp_ifaces_get where whole structure was copied instead
of addresses only. This is now fixed.
Also to make API totempg forward compatible, size of interfaces array
must be passed to ifaces_get like functions to prevent memory overwrite.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
while it might seem a waste of space by using 2 extra bytes in
the crypto_config_header, it actually gives us the option
to grow "unknown at this time" features without hopefully
breaking onwire compat
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
crypto: change network packets and add dynamic crypto header/data
The new network packet will look:
struct crypto_config_header * that provides info on crypto/hashing
hash_block[size based on hashing function] (if hash is selected)
salt[SALT_SIZE] (if crypto is selected)
...data...
and we kill the concept of crypto_security_header completely since
values are now dynamic for hash_block_size.
the reason why hash_block needs to be there, is because we do
hash salt in case both hashing and crypto are selected.
the crypto_config_header is totally transparent to totem
and to any underlaying crypto functions.
as we go cleaning, also use HASH_BLOCK_SIZE to generate hash_block.
the input buffer and output buffer size are dependent on the algo
used to hash.
we can now determine the real header size and adjust net_mtu properly
at startup. This will allow in future to use any algorithm since
size is dynamic.
some part of the code still needs some polishing to make it more
readable (specially the mapping of pointers into the packet
is still a bit obscure).
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
crypto: mask the crypto operations from totem packet size management
totem doesn't need to understand what crypto does.
totem needs to be able to tell crypto: "those are data, play with them"
and crypto needs to return: "here are your scrambled data and the new size"
similar to decrypt/verify.
this way we add enough dynamic within crypto to change header size and all
at any given time (for different hash algorithm for example) without
affecting on wire compat.
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
1) evs is only partially implemented with no plans to complete it
typedef enum {
EVS_TYPE_UNORDERED, /* not implemented */
EVS_TYPE_FIFO, /* same as agreed */
EVS_TYPE_AGREED,
EVS_TYPE_SAFE /* not implemented */
} evs_guarantee_t;
2) evs has no users in any upstream distribution and no search
engine can find any other upstream using it.
3) the only reason (I was told) to carry around evs was that evs
receives the full ring_id struct from totem. This is only
partially correct because while the structures are prepared
to carry around those data, they are never transmitted from
corosync engine down the IPC line to the user.
CPG ring_id contains the exact same information and it's
actually less buggy (due to prototying of the info).
worst case scenario where a user really absolutely need libevs,
it can be easily reimplemented as libcpg wrapper and avoid
lots of code duplication.
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Steven Dake <sdake@redhat.com>
stop using malloc for each new node, because we cannot free the memory
easily. Move to a static allocated buffer that can contain
PROCESSOR_MAX + qdevice cluster_node instead.
We can never have more than PROCESSOR_MAX nodes anyway and the memory
footprint is small enough compared to memory leaks (those can
effectively happen only in very dynamic clusters with tons of different
nodes joining/leaveing with different nodeids).
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
votequorum: rename leave_remove to allow_downscale
pointed out that leave_remove can be easily confused with the old
cman leave_remove behavior. The two are substantially different
and we need to avoid confusion both for users and our support team.
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
votequorum: disallow wait_for_all and qdevice operations
The problem here is that user expectations, when using both modes
at the same time, have not been set yet. There are 2/3 options
that need investigation.
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
when reading static config us->flags does not exists yet and therefor
setting it will cause a segfault.
Move the settings after cluster_node *us is created, with the long
term plan to simply kill the whole _static readconfig bits
in favour of dynamic (runtime changeable) bits.
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
votequorum: major rework to fix qdevice API and integration with core
qdevice is a very special node in the cluster and it adds a certain
amount of complexity and special cases across the code.
most of the qdevice data are shared across the cluster (name/votes)
but effectively each node has a different view of the qdevice
(registered/unregistered/voting/etc.)
with this change, we align the qdevice view across the node,
exchanging more data between nodes and we fix how qdevice behaves
and it is configured.
The only side effect is that the amount of data transmitted on wire
is slightly higher.
The qdevice API is still disabled by default. This means that
the amount of real changes in current code are a lot smaller
than it appears by this patch.
TODO: documentation/man pages needs to be updated once
this change is in (and behavior finalized).
User visible changes:
- configuration (coroparse, exec/votequorum):
the quorum device section is now standalone within the quorum.
the keyword "model:" is mandatory to enable qdevice in configuration
and should express the name of the script/daemon that will provide
the qdevice. Looking into the future, an init script or systemd
service will look for that name in /path/to/be/decided/name
and start/stop qdevice.
timeout: defines the maximum interval the qdevice implementation
has available between poll (see votequorum_qdevice_poll.3) before
the device is considered dead and votes discarded
votes: is now a configuration parameter and not an API call.
quorum devices don't care what they need to vote.
votes is autocalculated when a nodelist is available and all
nodes in the list vote 1. Otherwise this parameter is mandatory.
- configuration (exec/votequorum):
startup and runtime configuration changes have been improved.
errors at startup are considered fatal. errors at runtime
have different exit paths.
startup:
* quorum.two_node and qdevice are incompatible.
* quorum.expected_votes requires quorum.device.votes.
* quorum.expected_votes - quorum.device.votes cannot be lower
than 2.
* qdevice and last_man_standing are mutually exclusive.
* qdevice and auto_tie_breaker are mutually exclusive.
runtime config changes:
* quorum.two_node and qdevice are incompatible:
if quorum device is alive, two_node is disabled.
if quorum device is not alive and node count is 2, two_node is
enabled, and quorum device cannot be registered
* if either last_man_standing or auto_tie_breaker were enabled
at startup, and at runtime quorum device is configured,
quorum device registration will be blocked.
* if quorum.expected_votes is configured but not quorum.device.votes,
quorum device registration will be blocked.
* if quorum.device.votes is not configured and we cannot
automatically calculate it, quorum device registration will be blocked.
* An error in configuring quorum.expected_votes and quorum.device.votes
will block quorum device registration.
blocking quorum device registation, also means dropping the votes.
quorum.device.votes (either set or automatically calculated) is now
used to determine current expected_votes in the cluster.
- logging (exec/votequorum):
all errors from configuration are treated as WARNING/CRITICAL.
lots of extra DEBUG output is added (see internal changes too).
* added option to forcefully kick out a quorum device from the local
node. This is for emergency recovery only and it is only
available when qdevice API is built-in.
* Improved status output, specifically add node state and qdevice
information
[root@fedora-master-node2 coro]# corosync-quorumtool -s
Version: 1.99.4.12-9c7d-dirty
Quorum type: corosync_votequorum
Nodes: 2
Ring ID: 132
Quorate: Yes
Node votes: 1
Node state: Member
Expected votes: 3
Highest expected: 3
Total votes: 3
Quorum: 2
Flags: Quorate Qdevice
Nodeid Votes Name
1 1 fedora-master-node1.int.fabbione.net
2 1 fedora-master-node2.int.fabbione.net
0 1 QDEVICE (Voting)
* allow to print status for any node in the cluster known to
local node.
[root@fedora-master-node1 coro]# corosync-quorumtool -s
Version: 1.99.4.12-9c7d-dirty
Quorum type: corosync_votequorum
Nodes: 2
Ring ID: 144
Quorate: Yes
Node votes: 1
Node state: Member
Expected votes: 3
Highest expected: 3
Total votes: 2
Quorum: 2
Flags: Quorate
Nodeid Votes Name
1 1 fedora-master-node1.int.fabbione.net
2 1 fedora-master-node2.int.fabbione.net
- change qdevice timer to not run all time, but only when necessary.
- change votequorum_nodeinfo on wire data to use flags instead of uint8_t
and add QDEVICE status.
- allocate nodeid 0 to qdevice since it's the only real
nodeid that be reserved.
- change send_nodeinfo to allow to send nodeinfo for any node
so that we can share qdevice info across the cluster
(and this might be useful in future if we need to sync
internal cluster view).
- add votequorum api call to update qdevice name
- add runtime data if quorum device has been forcefully disabled
by config error
- add qdevice votes to expected_votes calculation (this
is probably the biggest difference vs cman)
- change votequorum_read_nodelist_configuration so that
we can autocalculate votes for qdevice (we need the nodecount
vs votes).
- add all checks for startup/runtime config (see above).
- do not make qdevice part of the membership_list received from
totem. None of our users care about it and it is not a real node.
- change onwire message handlers to deal with "data for this node from any node"
case and undersand nodeid 0 for qdevice info
- always allocate qdevice at startup. this simplifies code a lot.
- dispatch qdevice nodeinfo on membership changes.
- inform libvotequorum users when a qdevice is registered
- improve substantially qdevice api and add a simple
barrier based on qdevice name.
- add qdevice API barrier at cluster level. This feature allow
only one qdevice name to be active in the cluster at any time.
- qdevice getinfo can now report status for qdevice on any node.
- change slightly the way the qdevice API is built-in/out:
only the libvotequorum calls are #ifdef'out now. Doing so in
the core is too complex and would make the code unreadable
with the risk of missing a bit or two effectively introducing
an on-wire incompatibility if we will ever turn the API on.
- probably added some bugs on the way...
TODO: update qdevice_* API once the above is settled and test
qdevice integration with other features.
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com> Reviewed-by: Steven Dake <sdake@redhat.com> (only second part)
build: fix fallout from swithing to common shared lib
when building corosync on a clean system or for the very first
time, corosync_common needs to be visible both via -L for link
and for the LD_PATH, otherwise the linker cannot resolve
normal library dependencies.
This issue does NOT affect corosync users, but it's confined
to internal corosync only.
Signed-off-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Tue, 21 Feb 2012 13:30:35 +0000 (14:30 +0100)]
iba: Use configured node id
Corosync was ignoring nodeid for iba transport and always used
autogenerated one.
Original patch by: Jason Dillaman <jdillama@redhat.com> Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
Jan Friesse [Wed, 15 Feb 2012 16:04:25 +0000 (17:04 +0100)]
Allow autoconfiguration of interface section
Thanks to totemip_getifaddrs infrastructure it's now possible to use
nodelist informations to autoconfigure interface bindnetaddr. Together
with cluster_name, interface section can be completely omitted.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Steven Dake <sdake@redhat.com>
Jan Friesse [Wed, 15 Feb 2012 16:00:25 +0000 (17:00 +0100)]
totemconfig: ensure suffix for ringX_addr
Patch makes sure, that ringX_addr key has really _addr suffix.
Previously, it was possible to enter ringXanything and it was
interpreted as ringX_addr.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Steven Dake <sdake@redhat.com>
Jan Friesse [Wed, 15 Feb 2012 12:45:53 +0000 (13:45 +0100)]
Create solaris specific getifaddrs
This not only makes possible to use generic totemip_iface_check, but
also fixes some problems with previous implementation (fixed mask, not
very well supported ipv6, ...)
Tested on OpenIndiana 151a
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Steven Dake <sdake@redhat.com>
projects / mirror_corosync.git / log