Techwizz/kernel-fxtec-pro1x
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

rcu: Allow only one expedited GP to run concurrently with wakeups

Browse source 
The current expedited RCU grace-period code expects that a task
requesting an expedited grace period cannot awaken until that grace
period has reached the wakeup phase.  However, it is possible for a long
preemption to result in the waiting task never sleeping.  For example,
consider the following sequence of events:

1.      Task A starts an expedited grace period by invoking
        synchronize_rcu_expedited().  It proceeds normally up to the
        wait_event() near the end of that function, and is then preempted
        (or interrupted or whatever).

2.      The expedited grace period completes, and a kworker task starts
        the awaken phase, having incremented the counter and acquired
        the rcu_state structure's .exp_wake_mutex.  This kworker task
        is then preempted or interrupted or whatever.

3.      Task A resumes and enters wait_event(), which notes that the
        expedited grace period has completed, and thus doesn't sleep.

4.      Task B starts an expedited grace period exactly as did Task A,
        complete with the preemption (or whatever delay) just before
        the call to wait_event().

5.      The expedited grace period completes, and another kworker
        task starts the awaken phase, having incremented the counter.
        However, it blocks when attempting to acquire the rcu_state
        structure's .exp_wake_mutex because step 2's kworker task has
        not yet released it.

6.      Steps 4 and 5 repeat, resulting in overflow of the rcu_node
        structure's ->exp_wq[] array.

In theory, this is harmless.  Tasks waiting on the various ->exp_wq[]
array will just be spuriously awakened, but they will just sleep again
on noting that the rcu_state structure's ->expedited_sequence value has
not advanced far enough.

In practice, this wastes CPU time and is an accident waiting to happen.
This commit therefore moves the rcu_exp_gp_seq_end() call that officially
ends the expedited grace period (along with associate tracing) until
after the ->exp_wake_mutex has been acquired.  This prevents Task A from
awakening prematurely, thus preventing more than one expedited grace
period from being in flight during a previous expedited grace period's
wakeup phase.

Fixes: 3b5f668e71 ("rcu: Overlap wakeups with next expedited grace period")
Change-Id: I9bd12a8d639deec801b4e3546a1cf4729fe3f26c
Signed-off-by: Neeraj Upadhyay <neeraju@codeaurora.org>
[ paulmck: Added updated comment. ]
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Git-commit: 3dcffb240ef452fd97aa30d9c88efd4f4b030a68
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
[prsood@codeaurora.org: improved comment style]
Signed-off-by: Prateek Sood <prsood@codeaurora.org>
This commit is contained in:
Neeraj Upadhyay 2019-12-04 15:16:54 +05:30 committed by Prateek Sood
parent 87988027ae
commit beb2ec8fe8
1 changed files with 5 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
kernel/rcu/tree_exp.h
View file

@ -595,14 +595,14 @@ static void rcu_exp_wait_wake(struct rcu_state *rsp, unsigned long s)
struct rcu_node *rnp;
synchronize_sched_expedited_wait(rsp);
rcu_exp_gp_seq_end(rsp);
trace_rcu_exp_grace_period(rsp->name, s, TPS("end"));
/*
* Switch over to wakeup mode, allowing the next GP, but -only- the
* next GP, to proceed.
/* Switch over to wakeup mode, allowing the next GP to proceed.
* End the previous grace period only after acquiring the mutex
* to ensure that only one GP runs concurrently with wakeups.
*/
mutex_lock(&rsp->exp_wake_mutex);
rcu_exp_gp_seq_end(rsp);
trace_rcu_exp_grace_period(rcu_state.name, s, TPS("end"));
rcu_for_each_node_breadth_first(rsp, rnp) {
if (ULONG_CMP_LT(READ_ONCE(rnp->exp_seq_rq), s)) {