Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitf97bb52

Browse files
author
Peter Zijlstra
committed
sched: Fix data-race in wakeup
Mel reported that on some ARM64 platforms loadavg goes bananas andWill tracked it down to the following race: CPU0CPU1 schedule() prev->sched_contributes_to_load = X; deactivate_task(prev);try_to_wake_up() if (p->on_rq &&) // false if (smp_load_acquire(&p->on_cpu) && // true ttwu_queue_wakelist()) p->sched_remote_wakeup = Y; smp_store_release(prev->on_cpu, 0);where both p->sched_contributes_to_load and p->sched_remote_wakeup arein the same word, and thus the stores X and Y race (and can clobberone another's data).Whereas prior to commitc6e7bd7 ("sched/core: Optimize ttwu()spinning on p->on_cpu") the p->on_cpu handoff serialized access top->sched_remote_wakeup (just as it still does withp->sched_contributes_to_load) that commit broke that by callingttwu_queue_wakelist() with p->on_cpu != 0.However, due to p->XXX = Xttwu() schedule() if (p->on_rq && ...) // false smp_mb__after_spinlock() if (smp_load_acquire(&p->on_cpu) && deactivate_task() ttwu_queue_wakelist()) p->on_rq = 0; p->sched_remote_wakeup = Y;We can be sure any 'current' store is complete and 'current' isguaranteed asleep. Therefore we can move p->sched_remote_wakeup intothe current flags word.Note: while the observed failure was loadavg accounting gone wrong dueto ttwu() cobbering p->sched_contributes_to_load, the reverse problemis also possible where schedule() clobbers p->sched_remote_wakeup,this could result in enqueue_entity() wrecking ->vruntime and causingscheduling artifacts.Fixes:c6e7bd7 ("sched/core: Optimize ttwu() spinning on p->on_cpu")Reported-by: Mel Gorman <mgorman@techsingularity.net>Debugged-by: Will Deacon <will@kernel.org>Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>Link:https://lkml.kernel.org/r/20201117083016.GK3121392@hirez.programming.kicks-ass.net
1 parent8e1ac42 commitf97bb52

File tree

1 file changed

+15
-1
lines changed

1 file changed

+15
-1
lines changed

‎include/linux/sched.h‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -769,7 +769,6 @@ struct task_struct {
769769
unsignedsched_reset_on_fork:1;
770770
unsignedsched_contributes_to_load:1;
771771
unsignedsched_migrated:1;
772-
unsignedsched_remote_wakeup:1;
773772
#ifdefCONFIG_PSI
774773
unsignedsched_psi_wake_requeue:1;
775774
#endif
@@ -779,6 +778,21 @@ struct task_struct {
779778

780779
/* Unserialized, strictly 'current' */
781780

781+
/*
782+
* This field must not be in the scheduler word above due to wakelist
783+
* queueing no longer being serialized by p->on_cpu. However:
784+
*
785+
* p->XXX = X;ttwu()
786+
* schedule() if (p->on_rq && ..) // false
787+
* smp_mb__after_spinlock(); if (smp_load_acquire(&p->on_cpu) && //true
788+
* deactivate_task() ttwu_queue_wakelist())
789+
* p->on_rq = 0;p->sched_remote_wakeup = Y;
790+
*
791+
* guarantees all stores of 'current' are visible before
792+
* ->sched_remote_wakeup gets used, so it can be in this word.
793+
*/
794+
unsignedsched_remote_wakeup:1;
795+
782796
/* Bit to tell LSMs we're in execve(): */
783797
unsignedin_execve:1;
784798
unsignedin_iowait:1;

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp