Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit765cc3a

Browse files
derklingIngo Molnar
authored and
Ingo Molnar
committed
sched/core: Optimize sched_feat() for !CONFIG_SCHED_DEBUG builds
When the kernel is compiled with !CONFIG_SCHED_DEBUG support, we expect thatall SCHED_FEAT are turned into compile time constants being propagatedto support compiler optimizations.Specifically, we expect that code blocks like this: if (sched_feat(FEATURE_NAME) [&& <other_conditions>]) {/* FEATURE CODE */ }are turned into dead-code in case FEATURE_NAME defaults to FALSE, and thusbeing removed by the compiler from the finale image.For this mechanism to properly work it's required for the compiler tohave full access, from each translation unit, to whatever is the valuedefined by the sched_feat macro. This macro is defined as: #define sched_feat(x) (sysctl_sched_features & (1UL << __SCHED_FEAT_##x))and thus, the compiler can optimize that code only if the value ofsysctl_sched_features is visible within each translation unit.Since:029632f ("sched: Make separate sched*.c translation units")the scheduler code has been split into separate translation unitshowever the definition of sysctl_sched_features is part ofkernel/sched/core.c while, for all the other scheduler modules, it isvisible only via kernel/sched/sched.h as an: extern const_debug unsigned int sysctl_sched_featuresUnfortunately, an extern reference does not allow the compiler to applyconstants propagation. Thus, on !CONFIG_SCHED_DEBUG kernel we still end upwith code to load a memory reference and (eventually) doing an unconditionaljump of a chunk of code.This mechanism is unavoidable when sched_features can be turned on and off atrun-time. However, this is not the case for "production" kernels compiled with!CONFIG_SCHED_DEBUG. In this case, sysctl_sched_features is just a constant valuewhich cannot be changed at run-time and thus memory loads and jumps can beavoided altogether.This patch fixes the case of !CONFIG_SCHED_DEBUG kernel by declaring a local versionof the sysctl_sched_features constant for each translation unit. This willultimately allow the compiler to perform constants propagation and dead-codepruning.Tests have been done, with !CONFIG_SCHED_DEBUG on a v4.14-rc8 with and withoutthe patch, by running 30 iterations of: perf bench sched messaging --pipe --thread --group 4 --loop 50000on a 40 cores Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz using thepowersave governor to rule out variations due to frequency scaling.Statistics on the reported completion time: count mean std min 99% max v4.14-rc8 30.0 15.7831 0.176032 15.442 16.01226 16.014 v4.14-rc8+patch 30.0 15.5033 0.189681 15.232 15.93938 15.962... show a 1.8% speedup on average completion time and 0.5% speedup in the99 percentile.Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>Signed-off-by: Chris Redpath <chris.redpath@arm.com>Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>Reviewed-by: Brendan Jackman <brendan.jackman@arm.com>Acked-by: Peter Zijlstra <peterz@infradead.org>Cc: Juri Lelli <juri.lelli@redhat.com>Cc: Linus Torvalds <torvalds@linux-foundation.org>Cc: Morten Rasmussen <morten.rasmussen@arm.com>Cc: Thomas Gleixner <tglx@linutronix.de>Cc: Vincent Guittot <vincent.guittot@linaro.org>Link:http://lkml.kernel.org/r/20171108184101.16006-1-patrick.bellasi@arm.comSigned-off-by: Ingo Molnar <mingo@kernel.org>
1 parent8a103df commit765cc3a

File tree

2 files changed

+28
-6
lines changed

2 files changed

+28
-6
lines changed

‎kernel/sched/core.c‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -43,18 +43,21 @@
4343

4444
DEFINE_PER_CPU_SHARED_ALIGNED(structrq,runqueues);
4545

46+
#if defined(CONFIG_SCHED_DEBUG)&& defined(HAVE_JUMP_LABEL)
4647
/*
4748
* Debugging: various feature bits
49+
*
50+
* If SCHED_DEBUG is disabled, each compilation unit has its own copy of
51+
* sysctl_sched_features, defined in sched.h, to allow constants propagation
52+
* at compile time and compiler optimization based on features default.
4853
*/
49-
5054
#defineSCHED_FEAT(name,enabled)\
5155
(1UL << __SCHED_FEAT_##name) * enabled |
52-
5356
const_debug unsignedintsysctl_sched_features=
5457
#include"features.h"
5558
0;
56-
5759
#undef SCHED_FEAT
60+
#endif
5861

5962
/*
6063
* Number of tasks to iterate in a single balance run.

‎kernel/sched/sched.h‎

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1233,8 +1233,6 @@ static inline void __set_task_cpu(struct task_struct *p, unsigned int cpu)
12331233
# defineconst_debug const
12341234
#endif
12351235

1236-
externconst_debug unsignedintsysctl_sched_features;
1237-
12381236
#defineSCHED_FEAT(name,enabled)\
12391237
__SCHED_FEAT_##name ,
12401238

@@ -1246,20 +1244,41 @@ enum {
12461244
#undef SCHED_FEAT
12471245

12481246
#if defined(CONFIG_SCHED_DEBUG)&& defined(HAVE_JUMP_LABEL)
1247+
1248+
/*
1249+
* To support run-time toggling of sched features, all the translation units
1250+
* (but core.c) reference the sysctl_sched_features defined in core.c.
1251+
*/
1252+
externconst_debug unsignedintsysctl_sched_features;
1253+
12491254
#defineSCHED_FEAT(name,enabled)\
12501255
static __always_inline bool static_branch_##name(struct static_key *key) \
12511256
{\
12521257
return static_key_##enabled(key);\
12531258
}
12541259

12551260
#include"features.h"
1256-
12571261
#undef SCHED_FEAT
12581262

12591263
externstructstatic_keysched_feat_keys[__SCHED_FEAT_NR];
12601264
#definesched_feat(x) (static_branch_##x(&sched_feat_keys[__SCHED_FEAT_##x]))
1265+
12611266
#else/* !(SCHED_DEBUG && HAVE_JUMP_LABEL) */
1267+
1268+
/*
1269+
* Each translation unit has its own copy of sysctl_sched_features to allow
1270+
* constants propagation at compile time and compiler optimization based on
1271+
* features default.
1272+
*/
1273+
#defineSCHED_FEAT(name,enabled)\
1274+
(1UL << __SCHED_FEAT_##name) * enabled |
1275+
staticconst_debug__maybe_unusedunsignedintsysctl_sched_features=
1276+
#include"features.h"
1277+
0;
1278+
#undef SCHED_FEAT
1279+
12621280
#definesched_feat(x) (sysctl_sched_features & (1UL << __SCHED_FEAT_##x))
1281+
12631282
#endif/* SCHED_DEBUG && HAVE_JUMP_LABEL */
12641283

12651284
externstructstatic_key_falsesched_numa_balancing;

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp