bgpd: restore packet input limit

Unfortunately, batching input processing severely impacts BGP initial
convergence times. As a consequence of the way update-groups were
implemented, advancing the state of the routing table based on prefixes
learned from one peer prior to all (or at least most) peers establishing
connections will cause us to start generating outbound UPDATEs, which is
a very expensive operation at present. This intensive processing starves
out bgp_accept(), delaying connection of additional peers. When
additional peers do connect the problem gets worse and worse, yielding
approximately exponential growth in convergence time dependent on both
peering and prefix counts. This behavior is present pre-multithreading
as well, but batched input exacerbates it.

Round-robin input processing marginally harms convergence times for
small topologies but should allow much larger topologies to function
within reasonable performance thresholds.

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>

diff --git a/bgpd/bgp_packet.c b/bgpd/bgp_packet.c
index 42ca87679f169a0d1058edff9a1514e3f926349c..d814d6eb99c447fab35389e1972cb10f612044b2 100644 (file)
--- a/bgpd/bgp_packet.c
+++ b/bgpd/bgp_packet.c
@@ -2142,8 +2142,22 @@ int bgp_process_packet(struct thread *thread)
        int mprc;                 // message processing return code
 
        peer = THREAD_ARG(thread);
+/*
+ * This functionality is presently disabled. Unfortunately due to the
+ * way bgpd is structured, reading more than one packet per input cycle
+ * severely impacts convergence time. This is because advancing the
+ * state of the routing table based on prefixes learned from one peer
+ * prior to all (or at least most) peers being established and placed
+ * into an update-group will make UPDATE generation starve
+ * bgp_accept(), delaying convergence. This is a deficiency that needs
+ * to be fixed elsewhere in the codebase, but for now our hand is
+ * forced.
+ */
+#if 0
        rpkt_quanta_old = atomic_load_explicit(&peer->bgp->rpkt_quanta,
                                               memory_order_relaxed);
+#endif
+       rpkt_quanta_old = 1;
        fsm_update_result = 0;
 
        /* Guard against scheduled events that occur after peer deletion. */