Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Commit66e569f

Browse files
committed
Fix partition pruning setup during DETACH CONCURRENTLY
When detaching partition in concurrent mode, it's possible for partitiondescriptors to not match the set that was recently seen when the planwas made, causing an assertion failure or (in production builds) failureto construct a working plan. The case that was reported involvesprepared statements, but I think it may be possible to hit this bugwithout that too.The problem is that CreatePartitionPruneState is constructing aPartitionPruneState under the assumption that new partitions can beadded, but never removed, but it turns out that this isn't true: aprepared statement gets replanned when the DETACH CONCURRENTLY sessionsends out its invalidation message, but if the invalidation messagearrives after ExecInitAppend started, we would build a partitiondescriptor without the partition, and then CreatePartitionPruneStatewould refuse to work with it.CreatePartitionPruneState already contains code to deal with the newdescriptor having more partitions than before (and behaving for theextra partitions as if they had been pruned), but doesn't have code todeal with less partitions than before, and it is naïve about the casewhere the number of partitions is the same. We could simply add that anew stanza for less partitions than before, and in simple testing itworks to do that; but it's possible to press the test scripts evenfurther and hit the case where one partition is added and a partition isremoved quickly enough that we see the same number of partitions, butthey don't actually match, causing hangs during execution.To cope with both these problems, we now memcmp() the arrays ofpartition OIDs, and do a more elaborate mapping (relying on the factthat both OID arrays are in partition-bounds order) if they're notidentical.Backpatch to 14, where DETACH CONCURRENTLY appeared.Reported-by: yajun Hu <1026592243@qq.com>Reviewed-by: Tender Wang <tndrwang@gmail.com>Discussion:https://postgr.es/m/18377-e0324601cfebdfe5@postgresql.org
1 parentac9615d commit66e569f

File tree

1 file changed

+60
-50
lines changed

1 file changed

+60
-50
lines changed

‎src/backend/executor/execPartition.c

Lines changed: 60 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -1608,67 +1608,58 @@ ExecCreatePartitionPruneState(PlanState *planstate,
16081608
/*
16091609
* Initialize the subplan_map and subpart_map.
16101610
*
1611-
* Because we request detached partitions to be included, and
1612-
* detaching waits for old transactions, it is safe to assume that
1613-
* no partitions have disappeared since this query was planned.
1611+
* The set of partitions that exist now might not be the same that
1612+
* existed when the plan was made. The normal case is that it is;
1613+
* optimize for that case with a quick comparison, and just copy
1614+
* the subplan_map and make subpart_map point to the one in
1615+
* PruneInfo.
16141616
*
1615-
* However, new partitions may have been added.
1617+
* For the case where they aren't identical, we could have more
1618+
* partitions on either side; or even exactly the same number of
1619+
* them on both but the set of OIDs doesn't match fully. Handle
1620+
* this by creating new subplan_map and subpart_map arrays that
1621+
* corresponds to the ones in the PruneInfo where the new
1622+
* partition descriptor's OIDs match. Any that don't match can be
1623+
* set to -1, as if they were pruned. Both arrays must be in
1624+
* numerical OID order.
16161625
*/
1617-
Assert(partdesc->nparts >=pinfo->nparts);
16181626
pprune->nparts=partdesc->nparts;
16191627
pprune->subplan_map=palloc(sizeof(int)*partdesc->nparts);
1620-
if (partdesc->nparts==pinfo->nparts)
1628+
1629+
if (partdesc->nparts==pinfo->nparts&&
1630+
memcmp(partdesc->oids,pinfo->relid_map,
1631+
sizeof(int)*partdesc->nparts)==0)
16211632
{
1622-
/*
1623-
* There are no new partitions, so this is simple. We can
1624-
* simply point to the subpart_map from the plan, but we must
1625-
* copy the subplan_map since we may change it later.
1626-
*/
16271633
pprune->subpart_map=pinfo->subpart_map;
16281634
memcpy(pprune->subplan_map,pinfo->subplan_map,
16291635
sizeof(int)*pinfo->nparts);
1630-
1631-
/*
1632-
* Double-check that the list of unpruned relations has not
1633-
* changed. (Pruned partitions are not in relid_map[].)
1634-
*/
1635-
#ifdefUSE_ASSERT_CHECKING
1636-
for (intk=0;k<pinfo->nparts;k++)
1637-
{
1638-
Assert(partdesc->oids[k]==pinfo->relid_map[k]||
1639-
pinfo->subplan_map[k]==-1);
1640-
}
1641-
#endif
16421636
}
16431637
else
16441638
{
16451639
intpd_idx=0;
16461640
intpp_idx;
16471641

16481642
/*
1649-
* Some new partitions have appeared since plan time, and
1650-
* those are reflected in our PartitionDesc but were not
1651-
* present in the one used to construct subplan_map and
1652-
* subpart_map. So we must construct new and longer arrays
1653-
* where the partitions that were originally present map to
1654-
* the same sub-structures, and any added partitions map to
1655-
* -1, as if the new partitions had been pruned.
1643+
* When the partition arrays are not identical, there could be
1644+
* some new ones but it's also possible that one was removed;
1645+
* we cope with both situations by walking the arrays and
1646+
* discarding those that don't match.
16561647
*
1657-
* Note: pinfo->relid_map[] may contain InvalidOid entries for
1658-
* partitions pruned by the planner. We cannot tell exactly
1659-
* which of the partdesc entries these correspond to, but we
1660-
* don't have to; just skip over them. The non-pruned
1661-
* relid_map entries, however, had better be a subset of the
1662-
* partdesc entries and in the same order.
1648+
* If the number of partitions on both sides match, it's still
1649+
* possible that one partition has been detached and another
1650+
* attached. Cope with that by creating a map that skips any
1651+
* mismatches.
16631652
*/
16641653
pprune->subpart_map=palloc(sizeof(int)*partdesc->nparts);
1654+
16651655
for (pp_idx=0;pp_idx<partdesc->nparts;pp_idx++)
16661656
{
16671657
/* Skip any InvalidOid relid_map entries */
16681658
while (pd_idx<pinfo->nparts&&
16691659
!OidIsValid(pinfo->relid_map[pd_idx]))
16701660
pd_idx++;
16711661

1662+
recheck:
16721663
if (pd_idx<pinfo->nparts&&
16731664
pinfo->relid_map[pd_idx]==partdesc->oids[pp_idx])
16741665
{
@@ -1678,24 +1669,43 @@ ExecCreatePartitionPruneState(PlanState *planstate,
16781669
pprune->subpart_map[pp_idx]=
16791670
pinfo->subpart_map[pd_idx];
16801671
pd_idx++;
1672+
continue;
16811673
}
1682-
else
1674+
1675+
/*
1676+
* There isn't an exact match in the corresponding
1677+
* positions of both arrays. Peek ahead in
1678+
* pinfo->relid_map to see if we have a match for the
1679+
* current partition in partdesc. Normally if a match
1680+
* exists it's just one element ahead, and it means the
1681+
* planner saw one extra partition that we no longer see
1682+
* now (its concurrent detach finished just in between);
1683+
* so we skip that one by updating pd_idx to the new
1684+
* location and jumping above. We can then continue to
1685+
* match the rest of the elements after skipping the OID
1686+
* with no match; no future matches are tried for the
1687+
* element that was skipped, because we know the arrays to
1688+
* be in the same order.
1689+
*
1690+
* If we don't see a match anywhere in the rest of the
1691+
* pinfo->relid_map array, that means we see an element
1692+
* now that the planner didn't see, so mark that one as
1693+
* pruned and move on.
1694+
*/
1695+
for (intpd_idx2=pd_idx+1;pd_idx2<pinfo->nparts;pd_idx2++)
16831696
{
1684-
/* this partdesc entry is not in the plan */
1685-
pprune->subplan_map[pp_idx]=-1;
1686-
pprune->subpart_map[pp_idx]=-1;
1697+
if (pd_idx2 >=pinfo->nparts)
1698+
break;
1699+
if (pinfo->relid_map[pd_idx2]==partdesc->oids[pp_idx])
1700+
{
1701+
pd_idx=pd_idx2;
1702+
gotorecheck;
1703+
}
16871704
}
1688-
}
16891705

1690-
/*
1691-
* It might seem that we need to skip any trailing InvalidOid
1692-
* entries in pinfo->relid_map before checking that we scanned
1693-
* all of the relid_map. But we will have skipped them above,
1694-
* because they must correspond to some partdesc->oids
1695-
* entries; we just couldn't tell which.
1696-
*/
1697-
if (pd_idx!=pinfo->nparts)
1698-
elog(ERROR,"could not match partition child tables to plan elements");
1706+
pprune->subpart_map[pp_idx]=-1;
1707+
pprune->subplan_map[pp_idx]=-1;
1708+
}
16991709
}
17001710

17011711
/* present_parts is also subject to later modification */

0 commit comments

Comments
 (0)

[8]ページ先頭

©2009-2025 Movatter.jp