]> git.proxmox.com Git - mirror_ubuntu-jammy-kernel.git/commit
rcu: Don't deboost before reporting expedited quiescent state
authorPaul E. McKenney <paulmck@kernel.org>
Fri, 21 Jan 2022 20:40:08 +0000 (12:40 -0800)
committerStefan Bader <stefan.bader@canonical.com>
Fri, 20 May 2022 12:37:12 +0000 (14:37 +0200)
commitf6e619197ca4d3a2de8cf8d523eaf2a3cf3e1407
tree766a0db51dacb27747027755b6fc7b5fd17eb4aa
parent9e0577f9e0b0d9a635cf95180b6e62ca4d1c1c92
rcu: Don't deboost before reporting expedited quiescent state

BugLink: https://bugs.launchpad.net/bugs/1969106
commit 10c535787436d62ea28156a4b91365fd89b5a432 upstream.

Currently rcu_preempt_deferred_qs_irqrestore() releases rnp->boost_mtx
before reporting the expedited quiescent state.  Under heavy real-time
load, this can result in this function being preempted before the
quiescent state is reported, which can in turn prevent the expedited grace
period from completing.  Tim Murray reports that the resulting expedited
grace periods can take hundreds of milliseconds and even more than one
second, when they should normally complete in less than a millisecond.

This was fine given that there were no particular response-time
constraints for synchronize_rcu_expedited(), as it was designed
for throughput rather than latency.  However, some users now need
sub-100-millisecond response-time constratints.

This patch therefore follows Neeraj's suggestion (seconded by Tim and
by Uladzislau Rezki) of simply reversing the two operations.

Reported-by: Tim Murray <timmurray@google.com>
Reported-by: Joel Fernandes <joelaf@google.com>
Reported-by: Neeraj Upadhyay <quic_neeraju@quicinc.com>
Reviewed-by: Neeraj Upadhyay <quic_neeraju@quicinc.com>
Reviewed-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Tested-by: Tim Murray <timmurray@google.com>
Cc: Todd Kjos <tkjos@google.com>
Cc: Sandeep Patil <sspatil@google.com>
Cc: <stable@vger.kernel.org> # 5.4.x
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 058d62a03e7d057d5eeec0db800117765ff23e6c)
Signed-off-by: Paolo Pisati <paolo.pisati@canonical.com>
kernel/rcu/tree_plugin.h