Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit27162a6

Browse files
committed
Fix partition pruning setup during DETACH CONCURRENTLY
When detaching partition in concurrent mode, it's possible for partitiondescriptors to not match the set that was recently seen when the planwas made, causing an assertion failure or (in production builds) failureto construct a working plan. The case that was reported involvesprepared statements, but I think it may be possible to hit this bugwithout that too.The problem is that CreatePartitionPruneState is constructing aPartitionPruneState under the assumption that new partitions can beadded, but never removed, but it turns out that this isn't true: aprepared statement gets replanned when the DETACH CONCURRENTLY sessionsends out its invalidation message, but if the invalidation messagearrives after ExecInitAppend started, we would build a partitiondescriptor without the partition, and then CreatePartitionPruneStatewould refuse to work with it.CreatePartitionPruneState already contains code to deal with the newdescriptor having more partitions than before (and behaving for theextra partitions as if they had been pruned), but doesn't have code todeal with less partitions than before, and it is naïve about the casewhere the number of partitions is the same. We could simply add that anew stanza for less partitions than before, and in simple testing itworks to do that; but it's possible to press the test scripts evenfurther and hit the case where one partition is added and a partition isremoved quickly enough that we see the same number of partitions, butthey don't actually match, causing hangs during execution.To cope with both these problems, we now memcmp() the arrays ofpartition OIDs, and do a more elaborate mapping (relying on the factthat both OID arrays are in partition-bounds order) if they're notidentical.Backpatch to 14, where DETACH CONCURRENTLY appeared.Reported-by: yajun Hu <1026592243@qq.com>Reviewed-by: Tender Wang <tndrwang@gmail.com>Discussion:https://postgr.es/m/18377-e0324601cfebdfe5@postgresql.org
1 parentf7f4e7e commit27162a6

File tree

1 file changed

+60
-50
lines changed

1 file changed

+60
-50
lines changed

‎src/backend/executor/execPartition.c

Lines changed: 60 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -1942,67 +1942,58 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
19421942
/*
19431943
* Initialize the subplan_map and subpart_map.
19441944
*
1945-
* Because we request detached partitions to be included, and
1946-
* detaching waits for old transactions, it is safe to assume that
1947-
* no partitions have disappeared since this query was planned.
1945+
* The set of partitions that exist now might not be the same that
1946+
* existed when the plan was made. The normal case is that it is;
1947+
* optimize for that case with a quick comparison, and just copy
1948+
* the subplan_map and make subpart_map point to the one in
1949+
* PruneInfo.
19481950
*
1949-
* However, new partitions may have been added.
1951+
* For the case where they aren't identical, we could have more
1952+
* partitions on either side; or even exactly the same number of
1953+
* them on both but the set of OIDs doesn't match fully. Handle
1954+
* this by creating new subplan_map and subpart_map arrays that
1955+
* corresponds to the ones in the PruneInfo where the new
1956+
* partition descriptor's OIDs match. Any that don't match can be
1957+
* set to -1, as if they were pruned. Both arrays must be in
1958+
* numerical OID order.
19501959
*/
1951-
Assert(partdesc->nparts >=pinfo->nparts);
19521960
pprune->nparts=partdesc->nparts;
19531961
pprune->subplan_map=palloc(sizeof(int)*partdesc->nparts);
1954-
if (partdesc->nparts==pinfo->nparts)
1962+
1963+
if (partdesc->nparts==pinfo->nparts&&
1964+
memcmp(partdesc->oids,pinfo->relid_map,
1965+
sizeof(int)*partdesc->nparts)==0)
19551966
{
1956-
/*
1957-
* There are no new partitions, so this is simple. We can
1958-
* simply point to the subpart_map from the plan, but we must
1959-
* copy the subplan_map since we may change it later.
1960-
*/
19611967
pprune->subpart_map=pinfo->subpart_map;
19621968
memcpy(pprune->subplan_map,pinfo->subplan_map,
19631969
sizeof(int)*pinfo->nparts);
1964-
1965-
/*
1966-
* Double-check that the list of unpruned relations has not
1967-
* changed. (Pruned partitions are not in relid_map[].)
1968-
*/
1969-
#ifdefUSE_ASSERT_CHECKING
1970-
for (intk=0;k<pinfo->nparts;k++)
1971-
{
1972-
Assert(partdesc->oids[k]==pinfo->relid_map[k]||
1973-
pinfo->subplan_map[k]==-1);
1974-
}
1975-
#endif
19761970
}
19771971
else
19781972
{
19791973
intpd_idx=0;
19801974
intpp_idx;
19811975

19821976
/*
1983-
* Some new partitions have appeared since plan time, and
1984-
* those are reflected in our PartitionDesc but were not
1985-
* present in the one used to construct subplan_map and
1986-
* subpart_map. So we must construct new and longer arrays
1987-
* where the partitions that were originally present map to
1988-
* the same sub-structures, and any added partitions map to
1989-
* -1, as if the new partitions had been pruned.
1977+
* When the partition arrays are not identical, there could be
1978+
* some new ones but it's also possible that one was removed;
1979+
* we cope with both situations by walking the arrays and
1980+
* discarding those that don't match.
19901981
*
1991-
* Note: pinfo->relid_map[] may contain InvalidOid entries for
1992-
* partitions pruned by the planner. We cannot tell exactly
1993-
* which of the partdesc entries these correspond to, but we
1994-
* don't have to; just skip over them. The non-pruned
1995-
* relid_map entries, however, had better be a subset of the
1996-
* partdesc entries and in the same order.
1982+
* If the number of partitions on both sides match, it's still
1983+
* possible that one partition has been detached and another
1984+
* attached. Cope with that by creating a map that skips any
1985+
* mismatches.
19971986
*/
19981987
pprune->subpart_map=palloc(sizeof(int)*partdesc->nparts);
1988+
19991989
for (pp_idx=0;pp_idx<partdesc->nparts;pp_idx++)
20001990
{
20011991
/* Skip any InvalidOid relid_map entries */
20021992
while (pd_idx<pinfo->nparts&&
20031993
!OidIsValid(pinfo->relid_map[pd_idx]))
20041994
pd_idx++;
20051995

1996+
recheck:
20061997
if (pd_idx<pinfo->nparts&&
20071998
pinfo->relid_map[pd_idx]==partdesc->oids[pp_idx])
20081999
{
@@ -2012,24 +2003,43 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
20122003
pprune->subpart_map[pp_idx]=
20132004
pinfo->subpart_map[pd_idx];
20142005
pd_idx++;
2006+
continue;
20152007
}
2016-
else
2008+
2009+
/*
2010+
* There isn't an exact match in the corresponding
2011+
* positions of both arrays. Peek ahead in
2012+
* pinfo->relid_map to see if we have a match for the
2013+
* current partition in partdesc. Normally if a match
2014+
* exists it's just one element ahead, and it means the
2015+
* planner saw one extra partition that we no longer see
2016+
* now (its concurrent detach finished just in between);
2017+
* so we skip that one by updating pd_idx to the new
2018+
* location and jumping above. We can then continue to
2019+
* match the rest of the elements after skipping the OID
2020+
* with no match; no future matches are tried for the
2021+
* element that was skipped, because we know the arrays to
2022+
* be in the same order.
2023+
*
2024+
* If we don't see a match anywhere in the rest of the
2025+
* pinfo->relid_map array, that means we see an element
2026+
* now that the planner didn't see, so mark that one as
2027+
* pruned and move on.
2028+
*/
2029+
for (intpd_idx2=pd_idx+1;pd_idx2<pinfo->nparts;pd_idx2++)
20172030
{
2018-
/* this partdesc entry is not in the plan */
2019-
pprune->subplan_map[pp_idx]=-1;
2020-
pprune->subpart_map[pp_idx]=-1;
2031+
if (pd_idx2 >=pinfo->nparts)
2032+
break;
2033+
if (pinfo->relid_map[pd_idx2]==partdesc->oids[pp_idx])
2034+
{
2035+
pd_idx=pd_idx2;
2036+
gotorecheck;
2037+
}
20212038
}
2022-
}
20232039

2024-
/*
2025-
* It might seem that we need to skip any trailing InvalidOid
2026-
* entries in pinfo->relid_map before checking that we scanned
2027-
* all of the relid_map. But we will have skipped them above,
2028-
* because they must correspond to some partdesc->oids
2029-
* entries; we just couldn't tell which.
2030-
*/
2031-
if (pd_idx!=pinfo->nparts)
2032-
elog(ERROR,"could not match partition child tables to plan elements");
2040+
pprune->subpart_map[pp_idx]=-1;
2041+
pprune->subplan_map[pp_idx]=-1;
2042+
}
20332043
}
20342044

20352045
/* present_parts is also subject to later modification */

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp