Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit525392d

Browse files
committed
Don't lock partitions pruned by initial pruning
Before executing a cached generic plan, AcquireExecutorLocks() inplancache.c locks all relations in a plan's range table to ensure theplan is safe for execution. However, this locks runtime-prunablerelations that will later be pruned during "initial" runtime pruning,introducing unnecessary overhead.This commit defers locking for such relations to executor startup andensures that if the CachedPlan is invalidated due to concurrent DDLduring this window, replanning is triggered. Deferring these locksavoids unnecessary locking overhead for pruned partitions, resultingin significant speedup, particularly when many partitions are prunedduring initial runtime pruning.* Changes to locking when executing generic plans:AcquireExecutorLocks() now locks only unprunable relations, that is,those found in PlannedStmt.unprunableRelids (introduced in commitcbc1279), to avoid locking runtime-prunable partitionsunnecessarily. The remaining locks are taken byExecDoInitialPruning(), which acquires them only for partitions thatsurvive pruning.This deferral does not affect the locks required for permissionchecking in InitPlan(), which takes place before initial pruning.ExecCheckPermissions() now includes an Assert to verify that allrelations undergoing permission checks, none of which can be in theset of runtime-prunable relations, are properly locked.* Plan invalidation handling:Deferring locks introduces a window where prunable relations may bealtered by concurrent DDL, invalidating the plan. A new function,ExecutorStartCachedPlan(), wraps ExecutorStart() to detect and handleinvalidation caused by deferred locking. If invalidation occurs,ExecutorStartCachedPlan() updates CachedPlan using the newUpdateCachedPlan() function and retries execution with the updatedplan. To ensure all code paths that may be affected by this handleinvalidation properly, all callers of ExecutorStart that may execute aPlannedStmt from a CachedPlan have been updated to useExecutorStartCachedPlan() instead.UpdateCachedPlan() replaces stale plans in CachedPlan.stmt_list. A newCachedPlan.stmt_context, created as a child of CachedPlan.context,allows freeing old PlannedStmts while preserving the CachedPlanstructure and its statement list. This ensures that loops overstatements in upstream callers of ExecutorStartCachedPlan() remainintact.ExecutorStart() and ExecutorStart_hook implementations now return aboolean value indicating whether plan initialization succeeded with avalid PlanState tree in QueryDesc.planstate, or false otherwise, inwhich case QueryDesc.planstate is NULL. Hook implementations arerequired to call standard_ExecutorStart() at the beginning, and if itreturns false, they should do the same without proceeding.* Testing:To verify these changes, the delay_execution module tests scenarioswhere cached plans become invalid due to changes in prunable relationsafter deferred locks.* Note to extension authors:ExecutorStart_hook implementations must verify plan validity aftercalling standard_ExecutorStart(), as explained earlier. For example: if (prev_ExecutorStart) plan_valid = prev_ExecutorStart(queryDesc, eflags); else plan_valid = standard_ExecutorStart(queryDesc, eflags); if (!plan_valid) return false; <extension-code> return true;Extensions accessing child relations, especially prunable partitions,via ExecGetRangeTableRelation() must now ensure their RT indexes arepresent in es_unpruned_relids (introduced in commitcbc1279), orthey will encounter an error. This is a strict requirement after thischange, as only relations in that set are locked.The idea of deferring some locks to executor startup, allowing locksfor prunable partitions to be skipped, was first proposed by Tom Lane.Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions)Reviewed-by: David Rowley <dgrowleyml@gmail.com> (earlier versions)Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (earlier versions)Reviewed-by: Tomas Vondra <tomas@vondra.me>Reviewed-by: Junwang Zhao <zhjwpku@gmail.com>Discussion:https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
1 parent4aa6fa3 commit525392d

33 files changed

+1014
-95
lines changed

‎contrib/auto_explain/auto_explain.c

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ static ExecutorRun_hook_type prev_ExecutorRun = NULL;
7676
staticExecutorFinish_hook_typeprev_ExecutorFinish=NULL;
7777
staticExecutorEnd_hook_typeprev_ExecutorEnd=NULL;
7878

79-
staticvoidexplain_ExecutorStart(QueryDesc*queryDesc,inteflags);
79+
staticboolexplain_ExecutorStart(QueryDesc*queryDesc,inteflags);
8080
staticvoidexplain_ExecutorRun(QueryDesc*queryDesc,
8181
ScanDirectiondirection,
8282
uint64count);
@@ -256,9 +256,11 @@ _PG_init(void)
256256
/*
257257
* ExecutorStart hook: start up logging if needed
258258
*/
259-
staticvoid
259+
staticbool
260260
explain_ExecutorStart(QueryDesc*queryDesc,inteflags)
261261
{
262+
boolplan_valid;
263+
262264
/*
263265
* At the beginning of each top-level statement, decide whether we'll
264266
* sample this statement. If nested-statement explaining is enabled,
@@ -294,9 +296,13 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
294296
}
295297

296298
if (prev_ExecutorStart)
297-
prev_ExecutorStart(queryDesc,eflags);
299+
plan_valid=prev_ExecutorStart(queryDesc,eflags);
298300
else
299-
standard_ExecutorStart(queryDesc,eflags);
301+
plan_valid=standard_ExecutorStart(queryDesc,eflags);
302+
303+
/* The plan may have become invalid during standard_ExecutorStart() */
304+
if (!plan_valid)
305+
return false;
300306

301307
if (auto_explain_enabled())
302308
{
@@ -314,6 +320,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
314320
MemoryContextSwitchTo(oldcxt);
315321
}
316322
}
323+
324+
return true;
317325
}
318326

319327
/*

‎contrib/pg_stat_statements/pg_stat_statements.c

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -333,7 +333,7 @@ static PlannedStmt *pgss_planner(Query *parse,
333333
constchar*query_string,
334334
intcursorOptions,
335335
ParamListInfoboundParams);
336-
staticvoidpgss_ExecutorStart(QueryDesc*queryDesc,inteflags);
336+
staticboolpgss_ExecutorStart(QueryDesc*queryDesc,inteflags);
337337
staticvoidpgss_ExecutorRun(QueryDesc*queryDesc,
338338
ScanDirectiondirection,
339339
uint64count);
@@ -987,13 +987,19 @@ pgss_planner(Query *parse,
987987
/*
988988
* ExecutorStart hook: start up tracking if needed
989989
*/
990-
staticvoid
990+
staticbool
991991
pgss_ExecutorStart(QueryDesc*queryDesc,inteflags)
992992
{
993+
boolplan_valid;
994+
993995
if (prev_ExecutorStart)
994-
prev_ExecutorStart(queryDesc,eflags);
996+
plan_valid=prev_ExecutorStart(queryDesc,eflags);
995997
else
996-
standard_ExecutorStart(queryDesc,eflags);
998+
plan_valid=standard_ExecutorStart(queryDesc,eflags);
999+
1000+
/* The plan may have become invalid during standard_ExecutorStart() */
1001+
if (!plan_valid)
1002+
return false;
9971003

9981004
/*
9991005
* If query has queryId zero, don't track it. This prevents double
@@ -1016,6 +1022,8 @@ pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
10161022
MemoryContextSwitchTo(oldcxt);
10171023
}
10181024
}
1025+
1026+
return true;
10191027
}
10201028

10211029
/*

‎src/backend/commands/copyto.c

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -556,7 +556,7 @@ BeginCopyTo(ParseState *pstate,
556556
((DR_copy*)dest)->cstate=cstate;
557557

558558
/* Create a QueryDesc requesting no output */
559-
cstate->queryDesc=CreateQueryDesc(plan,pstate->p_sourcetext,
559+
cstate->queryDesc=CreateQueryDesc(plan,NULL,pstate->p_sourcetext,
560560
GetActiveSnapshot(),
561561
InvalidSnapshot,
562562
dest,NULL,NULL,0);
@@ -566,7 +566,8 @@ BeginCopyTo(ParseState *pstate,
566566
*
567567
* ExecutorStart computes a result tupdesc for us
568568
*/
569-
ExecutorStart(cstate->queryDesc,0);
569+
if (!ExecutorStart(cstate->queryDesc,0))
570+
elog(ERROR,"ExecutorStart() failed unexpectedly");
570571

571572
tupDesc=cstate->queryDesc->tupDesc;
572573
}

‎src/backend/commands/createas.c

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -332,12 +332,13 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
332332
UpdateActiveSnapshotCommandId();
333333

334334
/* Create a QueryDesc, redirecting output to our tuple receiver */
335-
queryDesc=CreateQueryDesc(plan,pstate->p_sourcetext,
335+
queryDesc=CreateQueryDesc(plan,NULL,pstate->p_sourcetext,
336336
GetActiveSnapshot(),InvalidSnapshot,
337337
dest,params,queryEnv,0);
338338

339339
/* call ExecutorStart to prepare the plan for execution */
340-
ExecutorStart(queryDesc,GetIntoRelEFlags(into));
340+
if (!ExecutorStart(queryDesc,GetIntoRelEFlags(into)))
341+
elog(ERROR,"ExecutorStart() failed unexpectedly");
341342

342343
/* run the plan to completion */
343344
ExecutorRun(queryDesc,ForwardScanDirection,0);

‎src/backend/commands/explain.c

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -519,7 +519,8 @@ standard_ExplainOneQuery(Query *query, int cursorOptions,
519519
}
520520

521521
/* run it (if needed) and produce output */
522-
ExplainOnePlan(plan,into,es,queryString,params,queryEnv,
522+
ExplainOnePlan(plan,NULL,NULL,-1,into,es,queryString,params,
523+
queryEnv,
523524
&planduration, (es->buffers ?&bufusage :NULL),
524525
es->memory ?&mem_counters :NULL);
525526
}
@@ -641,7 +642,9 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
641642
* to call it.
642643
*/
643644
void
644-
ExplainOnePlan(PlannedStmt*plannedstmt,IntoClause*into,ExplainState*es,
645+
ExplainOnePlan(PlannedStmt*plannedstmt,CachedPlan*cplan,
646+
CachedPlanSource*plansource,intquery_index,
647+
IntoClause*into,ExplainState*es,
645648
constchar*queryString,ParamListInfoparams,
646649
QueryEnvironment*queryEnv,constinstr_time*planduration,
647650
constBufferUsage*bufusage,
@@ -697,7 +700,7 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
697700
dest=None_Receiver;
698701

699702
/* Create a QueryDesc for the query */
700-
queryDesc=CreateQueryDesc(plannedstmt,queryString,
703+
queryDesc=CreateQueryDesc(plannedstmt,cplan,queryString,
701704
GetActiveSnapshot(),InvalidSnapshot,
702705
dest,params,queryEnv,instrument_option);
703706

@@ -711,8 +714,17 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
711714
if (into)
712715
eflags |=GetIntoRelEFlags(into);
713716

714-
/* call ExecutorStart to prepare the plan for execution */
715-
ExecutorStart(queryDesc,eflags);
717+
/* Prepare the plan for execution. */
718+
if (queryDesc->cplan)
719+
{
720+
ExecutorStartCachedPlan(queryDesc,eflags,plansource,query_index);
721+
Assert(queryDesc->planstate);
722+
}
723+
else
724+
{
725+
if (!ExecutorStart(queryDesc,eflags))
726+
elog(ERROR,"ExecutorStart() failed unexpectedly");
727+
}
716728

717729
/* Execute the plan for statistics if asked for */
718730
if (es->analyze)

‎src/backend/commands/extension.c

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -907,11 +907,13 @@ execute_sql_string(const char *sql, const char *filename)
907907
QueryDesc*qdesc;
908908

909909
qdesc=CreateQueryDesc(stmt,
910+
NULL,
910911
sql,
911912
GetActiveSnapshot(),NULL,
912913
dest,NULL,NULL,0);
913914

914-
ExecutorStart(qdesc,0);
915+
if (!ExecutorStart(qdesc,0))
916+
elog(ERROR,"ExecutorStart() failed unexpectedly");
915917
ExecutorRun(qdesc,ForwardScanDirection,0);
916918
ExecutorFinish(qdesc);
917919
ExecutorEnd(qdesc);

‎src/backend/commands/matview.c

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -438,12 +438,13 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
438438
UpdateActiveSnapshotCommandId();
439439

440440
/* Create a QueryDesc, redirecting output to our tuple receiver */
441-
queryDesc=CreateQueryDesc(plan,queryString,
441+
queryDesc=CreateQueryDesc(plan,NULL,queryString,
442442
GetActiveSnapshot(),InvalidSnapshot,
443443
dest,NULL,NULL,0);
444444

445445
/* call ExecutorStart to prepare the plan for execution */
446-
ExecutorStart(queryDesc,0);
446+
if (!ExecutorStart(queryDesc,0))
447+
elog(ERROR,"ExecutorStart() failed unexpectedly");
447448

448449
/* run the plan */
449450
ExecutorRun(queryDesc,ForwardScanDirection,0);

‎src/backend/commands/portalcmds.c

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,7 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
117117
queryString,
118118
CMDTAG_SELECT,/* cursor's query is always a SELECT */
119119
list_make1(plan),
120+
NULL,
120121
NULL);
121122

122123
/*----------

‎src/backend/commands/prepare.c

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -202,7 +202,8 @@ ExecuteQuery(ParseState *pstate,
202202
query_string,
203203
entry->plansource->commandTag,
204204
plan_list,
205-
cplan);
205+
cplan,
206+
entry->plansource);
206207

207208
/*
208209
* For CREATE TABLE ... AS EXECUTE, we must verify that the prepared
@@ -582,6 +583,7 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
582583
MemoryContextCountersmem_counters;
583584
MemoryContextplanner_ctx=NULL;
584585
MemoryContextsaved_ctx=NULL;
586+
intquery_index=0;
585587

586588
if (es->memory)
587589
{
@@ -654,7 +656,8 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
654656
PlannedStmt*pstmt=lfirst_node(PlannedStmt,p);
655657

656658
if (pstmt->commandType!=CMD_UTILITY)
657-
ExplainOnePlan(pstmt,into,es,query_string,paramLI,pstate->p_queryEnv,
659+
ExplainOnePlan(pstmt,cplan,entry->plansource,query_index,
660+
into,es,query_string,paramLI,pstate->p_queryEnv,
658661
&planduration, (es->buffers ?&bufusage :NULL),
659662
es->memory ?&mem_counters :NULL);
660663
else
@@ -665,6 +668,8 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
665668
/* Separate plans with an appropriate separator */
666669
if (lnext(plan_list,p)!=NULL)
667670
ExplainSeparatePlans(es);
671+
672+
query_index++;
668673
}
669674

670675
if (estate)

‎src/backend/commands/trigger.c

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5057,6 +5057,21 @@ AfterTriggerBeginQuery(void)
50575057
}
50585058

50595059

5060+
/* ----------
5061+
* AfterTriggerAbortQuery()
5062+
*
5063+
* Called by standard_ExecutorEnd() if the query execution was aborted due to
5064+
* the plan becoming invalid during initialization.
5065+
* ----------
5066+
*/
5067+
void
5068+
AfterTriggerAbortQuery(void)
5069+
{
5070+
/* Revert the actions of AfterTriggerBeginQuery(). */
5071+
afterTriggers.query_depth--;
5072+
}
5073+
5074+
50605075
/* ----------
50615076
* AfterTriggerEndQuery()
50625077
*

‎src/backend/executor/README

Lines changed: 32 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -280,6 +280,28 @@ are typically reset to empty once per tuple. Per-tuple contexts are usually
280280
associated with ExprContexts, and commonly each PlanState node has its own
281281
ExprContext to evaluate its qual and targetlist expressions in.
282282

283+
Relation Locking
284+
----------------
285+
286+
When the executor initializes a plan tree for execution, it doesn't lock
287+
non-index relations if the plan tree is freshly generated and not derived
288+
from a CachedPlan. This is because such locks have already been established
289+
during the query's parsing, rewriting, and planning phases. However, with a
290+
cached plan tree, some relations may remain unlocked. The function
291+
AcquireExecutorLocks() only locks unprunable relations in the plan, deferring
292+
the locking of prunable ones to executor initialization. This avoids
293+
unnecessary locking of relations that will be pruned during "initial" runtime
294+
pruning in ExecDoInitialPruning().
295+
296+
This approach creates a window where a cached plan tree with child tables
297+
could become outdated if another backend modifies these tables before
298+
ExecDoInitialPruning() locks them. As a result, the executor has the added duty
299+
to verify the plan tree's validity whenever it locks a child table after
300+
doing initial pruning. This validation is done by checking the CachedPlan.is_valid
301+
flag. If the plan tree is outdated (is_valid = false), the executor stops
302+
further initialization, cleans up anything in EState that would have been
303+
allocated up to that point, and retries execution after recreating the
304+
invalid plan in the CachedPlan. See ExecutorStartCachedPlan().
283305

284306
Query Processing Control Flow
285307
-----------------------------
@@ -288,11 +310,13 @@ This is a sketch of control flow for full query processing:
288310

289311
CreateQueryDesc
290312

291-
ExecutorStart
313+
ExecutorStart or ExecutorStartCachedPlan
292314
CreateExecutorState
293315
creates per-query context
294-
switch to per-query context to run ExecInitNode
316+
switch to per-query context to runExecDoInitialPruning andExecInitNode
295317
AfterTriggerBeginQuery
318+
ExecDoInitialPruning
319+
does initial pruning and locks surviving partitions if needed
296320
ExecInitNode --- recursively scans plan tree
297321
ExecInitNode
298322
recurse into subsidiary nodes
@@ -316,7 +340,12 @@ This is a sketch of control flow for full query processing:
316340

317341
FreeQueryDesc
318342

319-
Per above comments, it's not really critical for ExecEndNode to free any
343+
As mentioned in the "Relation Locking" section, if the plan tree is found to
344+
be stale after locking partitions in ExecDoInitialPruning(), the control is
345+
immediately returned to ExecutorStartCachedPlan(), which will create a new plan
346+
tree and perform the steps starting from CreateExecutorState() again.
347+
348+
Per above comments, it's not really critical for ExecEndPlan to free any
320349
memory; it'll all go away in FreeExecutorState anyway. However, we do need to
321350
be careful to close relations, drop buffer pins, etc, so we do need to scan
322351
the plan state tree to find these sorts of resources.

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp