Quentin Young [Fri, 12 Jan 2018 21:00:24 +0000 (16:00 -0500)]
doc: make conf.py know about variable substitutions
The old Texi docs had some Makefile-style variable substitutions they
used to pull in information like release date, configured install path,
etc. Sphinx has this ability as well, we start making use of it here.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Fri, 8 Dec 2017 23:22:26 +0000 (18:22 -0500)]
doc: more organizing & updating
* Add chapter on BGPD
* Add diagram for git workflow
* Convert next-hop tracking documents to ReST
* Update & organize workflow document
* Move ldpd docs back up to the parent directory
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Fri, 8 Dec 2017 16:47:49 +0000 (11:47 -0500)]
doc: begin developer's manual
* Move doc/code/ -> doc/developer/
* Move disparate code documentation to doc/developer/
* Convert Markdown docs to reStructuredText
* Organize docs into Sphinx tree
* Move build docs -> doc/developer/
* Change build doc titles to fit under Building subsection of manual
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Chirag Shah [Mon, 4 Dec 2017 22:08:23 +0000 (14:08 -0800)]
ospfd: fix crash no router ospf/show running
no router ospf removes default ospf instance,
if there are other non-default vrf instance present
with interface level configuration. Lookup ospf instance
for ifp->vrf_id, if ospf instnace present use that
to access 'instance id'.
Ticket: CM-19078
Testing Done:
run no router ospf and show running config along with other
non-default vrf aware ospf configurations.
Signed-off-by: Chirag Shah <chirag@cumulusnetworks.com>
Chirag Shah [Mon, 27 Nov 2017 21:59:19 +0000 (13:59 -0800)]
ospfd: Display all vrf aware interface config
OSPF interface specific configuration can be done independent
of router ospf [vrf x] global config.
In cases where ospf interface non default vrf configuration
is done prior to 'router ospf vrf x', show running-config
would not display such configuration.
To display configuration now walk all vrfs and interface list
and only display where OSPF configure params are set.
Ticket:CM-18952
Testing Done:
Tried ospf interface specific configuration with VRF,
where router ospf vrf x is not present.
Signed-off-by: Chirag Shah <chirag@cumulusnetworks.com>
Donald Sharp [Fri, 1 Dec 2017 16:49:13 +0000 (11:49 -0500)]
bgpd: Allow Address-Family activation to work in certain states
If we are in OpenSent or OpenConfirm peer state and we receive a new
address-family activation, we would end up ignoring the new activation
and not tell our peer about it. You could notice this by seeing
the fact that a 'show bgp neighbor' command returns a 'Not in
any update group' for a particular family.
This modifies the code such that we now notice that we are in
either OpenSent or OpenConfirm state and reset the peer to
allow us to send them the new capability.
Ticket: CM-19021 Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Quentin Young [Thu, 30 Nov 2017 19:11:12 +0000 (14:11 -0500)]
bgpd: intelligently adjust coalesce timer
The subgroup coalesce timer controls how long updates to a particular
subgroup are delayed in order to allow additional peers to join the
subgroup. Presently the timer value is 200 ms. Increase it to 1 second
and adjust up as peers are configured, with an upper cap at 10s.
This cuts convergence time by a factor of 3 at large scale (300+ peers,
1000+ prefixes per peer).
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 13 Nov 2017 22:59:04 +0000 (17:59 -0500)]
bgpd: turn off keepalives when sending NOTIFY
This is necessary because otherwise between the time we wipe the output
buffer and the time we push the NOTIFY onto it, the KA generation thread
could have pushed a KEEPALIVE in the middle.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 13 Nov 2017 08:18:49 +0000 (03:18 -0500)]
bgpd: yield more when generating UPDATEs
In the same vein as the round-robin input commit, this re-adds logic for
limiting the amount of time spent generating UPDATEs per generation
cycle. Missed this when shifting around wpkt_quanta; prior to MT it
limited both calls to write() as well as UPDATE generation.
Quentin Young [Fri, 10 Nov 2017 21:42:49 +0000 (16:42 -0500)]
bgpd: restore packet input limit
Unfortunately, batching input processing severely impacts BGP initial
convergence times. As a consequence of the way update-groups were
implemented, advancing the state of the routing table based on prefixes
learned from one peer prior to all (or at least most) peers establishing
connections will cause us to start generating outbound UPDATEs, which is
a very expensive operation at present. This intensive processing starves
out bgp_accept(), delaying connection of additional peers. When
additional peers do connect the problem gets worse and worse, yielding
approximately exponential growth in convergence time dependent on both
peering and prefix counts. This behavior is present pre-multithreading
as well, but batched input exacerbates it.
Round-robin input processing marginally harms convergence times for
small topologies but should allow much larger topologies to function
within reasonable performance thresholds.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Tue, 7 Nov 2017 07:49:54 +0000 (02:49 -0500)]
bgpd: schedule process packet as timer
Different places scheduling the same thread should use the same
semantics and thread type. Additionally providing the back reference
here makes sure we only schedule the job once and avoids flooding the
event queue with jobs to process an empty buffer.
Quentin Young [Mon, 6 Nov 2017 19:15:36 +0000 (14:15 -0500)]
bgpd: re-add write trigger logic
Apparently I didn't fully understand how subgroup packets make their way
out to individual peers. Turns out (on the base branch) we just busy
poll while waiting for packets to make their way onto subgroup queues.
While this needs to be fixed in the future, for now readding this logic
fixes performance issues with convergence.
Quentin Young [Mon, 6 Nov 2017 06:41:27 +0000 (01:41 -0500)]
bgpd: properly set peer->last_update
Instead of checking whether the post-write number of updates sent was
greater than the pre-write number of updates sent, it was comparing post
to zero. In effect this meant every time we wrote a packet it was
counted as an update for route advertisement timer purposes.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 6 Nov 2017 05:33:46 +0000 (00:33 -0500)]
bgpd: schedule packet job after connection xfer
During initial session establishment, bgpd performs a "connection
transfer" to a new peer struct if the connection was initiated passively
(i.e. by the remote peer). With the addition of buffered input and a
reorganized packet processor, the following race condition manifests:
1. Remote peer initiates a connection. After exchanging OPEN messages,
we send them a KEEPALIVE. They send us a KEEPALIVE followed by
10,000 UPDATE messages. The I/O thread pushes these onto our local
peer's input buffer and schedules a packet processing job on the
main thread.
2. The packet job runs and processes the KEEPALIVE, which completes the
handshake on our end. As part of transferring to ESTABLISHED we
transfer all peer state to a new struct, as mentioned. Upon returning
from the KEEPALIVE processing routing, the peer context we had has
now been destroyed. We notice this and stop processing. Meanwhile
10k UPDATE messages are sitting on the input buffer.
3. N seconds later, the remote peer sends us a KEEPALIVE. The I/O thread
schedules another process job, which finds 10k UPDATEs waiting for
it. Convergence is achieved, but has been delayed by the value of the
KEEPALIVE timer.
The racey part is that if the remote peer takes a little bit of time to
send UPDATEs after KEEPALIVEs -- somewhere on the order of a few hundred
milliseconds -- we complete the transfer successfully and the packet
processing job is scheduled on the new peer upon arrival of the UPDATE
messages. Yuck.
The solution is to schedule a packet processing job on the new peer
struct after transferring state.
Lengthy commit message in case someone has to debug similar problems in
the future...
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Fri, 3 Nov 2017 18:47:56 +0000 (14:47 -0400)]
bgpd: transfer raw input buffer to new peer
During initial session establishment, bgpd performs a "connection
transfer" to a new peer struct if the connection was initiated passively
(i.e. by the remote peer). With the addition of buffered input, I forgot
to transfer the raw input buffer to the new peer. This resulted in
infrequent failures during session handshaking whereby half of a packet
would be thrown away in the middle of a read causing us to send a NOTIFY
for an unsynchronized header. Usually the transfer coincided with a
clean input buffer, hence why it only showed up once in a while.
Quentin Young [Mon, 25 Sep 2017 02:18:15 +0000 (22:18 -0400)]
bgpd: fix bgp active open
At some point when rearranging FSM code, bgpd lost the ability to
perform active opens because it was only paying attention to POLLIN and
not POLLOUT, when the latter is used to signify a successful connection
in the active case.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 12 Jun 2017 21:16:40 +0000 (21:16 +0000)]
bgpd: be more promiscuous with updgrp packets
Slightly incorrect trigger for generating update group packets. In order
to match semantics of previous bgp_write() we need to trigger
update-group packet generation after every write operation, even if no
packets were written. Of course if we're tearing down the session we can
still skip this operation.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 12 Jun 2017 20:20:50 +0000 (20:20 +0000)]
bgpd: re-add update-group write triggers
Removed in earlier version where the I/O pthread busy-waited for packets
to be posted to an output queue. Now that it's poll()-based, it's
necessary once again. Although this time we can say what we're actually
doing instead of a side effect of a write job.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 12 Jun 2017 02:53:42 +0000 (02:53 +0000)]
bgpd: misc fsm fixes
* Keepalive on/off calls are necessary in certain cases due to screwy
fsm flow not turning them on after transferring a passive peer
connection in peer_xfer_conn
* Missed a case bgp_event_update() that resulted in a return code of -1
instead of BGP_Stop, which confuses the packet processing routine
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Sat, 10 Jun 2017 01:01:56 +0000 (01:01 +0000)]
bgpd: fix bgp_packet.c / bgp_fsm.c organization
Despaghettification of bgp_packet.c and bgp_fsm.c
Sometimes we call bgp_event_update() inline packet parsing.
Sometimes we post events instead.
Sometimes we increment packet counters in the FSM.
Sometimes we do it in packet routines.
Sometimes we update EOR's in FSM.
Sometimes we do it in packet routines.
Fix the madness.
bgp_process_packet() is now the centralized place to:
- Update message counters
- Execute FSM events in response to incoming packets
FSM events are now executed directly from this function instead of being
queued on the thread_master. This is to ensure that the FSM contains the
proper state after each packet is parsed. Otherwise there could be race
conditions where two packets are parsed in succession without the
appropriate FSM update in between, leading to session closure due to
receiving inappropriate messages for the current FSM state.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Thu, 8 Jun 2017 21:14:18 +0000 (21:14 +0000)]
bgpd: small i/o threading improvements
* Start bit flags at 1, not 2
* Make run-flags atomic for i/o thread
* Remove work_cond mutex, it should no longer be necessary
* Add asserts to ensure proper ordering in bgp_connect()
* Use true/false with booleans, not 1/0
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 5 Jun 2017 20:14:47 +0000 (20:14 +0000)]
bgpd: atomize write-quanta, add read-quanta
bgpd supports setting a write-quanta that serves as a hint on how many
packets to write per I/O cycle. Now that input is buffered, it makes
sense to add the equivalent parameter for how many packets are processed
per cycle. This is *not* how many packets are read off the wire per I/O
cycle; rather it is how many packets are processed from the input buffer
in a given cycle after having been read off the wire and sanitized.
Since these values must be used from multiple threads, they have also
been made atomic.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Fri, 2 Jun 2017 01:52:39 +0000 (01:52 +0000)]
bgpd: batched i/o
Instead of reading a packet header and the rest of the packet in two
separate i/o cycles, instead read a chunk of data at one time and then
parse as many packets as possible out of the chunk.
Also changes bgp_packet.c to batch process packets.
To avoid thrashing on useless mutex locks, the scheduling call for
bgp_process_packet has been changed to always succeed at the cost of no
longer being cancel-able. In this case this is acceptable; following the
pattern of other event-based callbacks, an additional check in
bgp_process_packet to ignore stray events is sufficient. Before deleting
the peer all events are cleared which provides the requisite ordering.
XXX: chunk hardcoded to 5, should use something similar to wpkt_quanta
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Tue, 2 May 2017 00:37:45 +0000 (00:37 +0000)]
bgpd: implement buffered reads
* Move and modify all network input related code to bgp_io.c
* Add a real input buffer to `struct peer`
* Move connection initialization to its own thread.c task instead of
piggybacking off of bgp_read()
* Tons of little fixups
Primary changes are in bgp_packet.[ch], bgp_io.[ch], bgp_fsm.[ch].
Changes made elsewhere are almost exclusively refactoring peer->ibuf to
peer->curr since peer->ibuf is now the true FIFO packet input buffer
while peer->curr represents the packet currently being processed by the
main pthread.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Tue, 18 Apr 2017 18:11:43 +0000 (18:11 +0000)]
bgpd: move bgp i/o to a separate source file
After implement threading, bgp_packet.c was serving the double purpose
of consolidating packet parsing functionality and handling actual I/O
operations. This is somewhat messy and difficult to understand. I've
thus moved all code and data structures for handling threaded packet
writes to bgp_io.[ch].
Although bgp_io.[ch] only handles writes at the moment to keep the noise
on this commit series down, for organization purposes, it's probably
best to move bgp_read() and its trappings into here as well and
restructure that code so that read()'s happen in the pthread and packet
processing happens on the main thread.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Thu, 6 Apr 2017 23:45:57 +0000 (23:45 +0000)]
bgpd: correctly schedule select() at session startup
On TCP connection failure during session setup, bgp_stop() checks
whether peer->t_read is non-null to know whether or not to unschedule
select() on peer->fd before calling close() on it. Using the API exposed
by thread.c instead of bgpd's wrapper macro BGP_READ_ON() results in
this thread value never being set, which causes bgp_stop() to skip the
cancellation of select() before calling close(). Subsequent calls to
select() on that fd crash the daemon.
Use the macro instead.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
projects / mirror_frr.git / log