- Notifications
You must be signed in to change notification settings - Fork28
Commit898e5e3
committed
Allow ATTACH PARTITION with only ShareUpdateExclusiveLock.
We still require AccessExclusiveLock on the partition itself, becauseotherwise an insert that violates the newly-imposed partitionconstraint could be in progress at the same time that we're changingthat constraint; only the lock level on the parent relation isweakened.To make this safe, we have to cope with (at least) three separateproblems. First, relevant DDL might commit while we're in the processof building a PartitionDesc. If so, find_inheritance_children() mightsee a new partition while the RELOID system cache still has the oldpartition bound cached, and even before invalidation messages havebeen queued. To fix that, if we see that the pg_class tuple seems tobe missing or to have a null relpartbound, refetch the value directlyfrom the table. We can't get the wrong value, because DETACH PARTITIONstill requires AccessExclusiveLock throughout; if we ever want tochange that, this will need more thought. In testing, I found it quitedifficult to hit even the null-relpartbound case; the race conditionis extremely tight, but the theoretical risk is there.Second, successive calls to RelationGetPartitionDesc might not returnthe same answer. The query planner will get confused if lookup up thePartitionDesc for a particular relation does not return a consistentanswer for the entire duration of query planning. Likewise, queryexecution will get confused if the same relation seems to have adifferent PartitionDesc at different times. Invent a newPartitionDirectory concept and use it to ensure consistency. Thisensures that a single invocation of either the planner or the executorsees the same view of the PartitionDesc from beginning to end, but itdoes not guarantee that the planner and the executor see the sameview. Since this allows pointers to old PartitionDesc entries tosurvive even after a relcache rebuild, also postpone removing the oldPartitionDesc entry until we're certain no one is using it.For the most part, it seems to be OK for the planner and executor tohave different views of the PartitionDesc, because the executor willjust ignore any concurrently added partitions which were unknown atplan time; those partitions won't be part of the inheritanceexpansion, but invalidation messages will trigger replanning at somepoint. Normally, this happens by the time the very next command isexecuted, but if the next command acquires no locks and executes aprepared query, it can manage not to notice until a new transaction isstarted. We might want to tighten that up, but it's material for aseparate patch. There would still be a small window where a querythat started just after an ATTACH PARTITION command committed mightfail to notice its results -- but only if the command starts beforethe commit has been acknowledged to the user. All in all, the wartshere around serializability seem small enough to be worth acceptingfor the considerable advantage of being able to add partitions withouta full table lock.Although in general the consequences of new partitions showing upbetween planning and execution are limited to the query not noticingthe new partitions, run-time partition pruning will get confused inthat case, so that's the third problem that this patch fixes.Run-time partition pruning assumes that indexes into the PartitionDescare stable between planning and execution. So, add code so that ifnew partitions are added between plan time and execution time, theindexes stored in the subplan_map[] and subpart_map[] arrays withinthe plan's PartitionedRelPruneInfo get adjusted accordingly. Theredoes not seem to be a simple way to generalize this scheme to copewith partitions that are removed, mostly because they could then getadded back again with different bounds, but it works OK for addedpartitions.This code does not try to ensure that every backend participating ina parallel query sees the same view of the PartitionDesc. Thatcurrently doesn't matter, because we never pass PartitionDescindexes between backends. Each backend will ignore the concurrentlyadded partitions which it notices, and it doesn't matter if differentbackends are ignoring different sets of concurrently added partitions.If in the future that matters, for example because we allow writes inparallel query and want all participants to do tuple routing to the sameset of partitions, the PartitionDirectory concept could be improved toshare PartitionDescs across backends. There is a draft patch toserialize and restore PartitionDescs on the thread where this patchwas discussed, which may be a useful place to start.Patch by me. Thanks to Alvaro Herrera, David Rowley, Simon Riggs,Amit Langote, and Michael Paquier for discussion, and to AlvaroHerrera for some review.Discussion:http://postgr.es/m/CA+Tgmobt2upbSocvvDej3yzokd7AkiT+PvgFH+a9-5VV1oJNSQ@mail.gmail.comDiscussion:http://postgr.es/m/CA+TgmoZE0r9-cyA-aY6f8WFEROaDLLL7Vf81kZ8MtFCkxpeQSw@mail.gmail.comDiscussion:http://postgr.es/m/CA+TgmoY13KQZF-=HNTrt9UYWYx3_oYOQpu9ioNT49jGgiDpUEA@mail.gmail.com1 parentec51727 commit898e5e3
File tree
21 files changed
+314
-45
lines changed- doc/src/sgml
- src
- backend
- commands
- executor
- nodes
- optimizer
- plan
- util
- partitioning
- utils/cache
- include
- executor
- nodes
- partitioning
21 files changed
+314
-45
lines changedLines changed: 2 additions & 1 deletion
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
3827 | 3827 |
| |
3828 | 3828 |
| |
3829 | 3829 |
| |
3830 |
| - | |
| 3830 | + | |
| 3831 | + | |
3831 | 3832 |
| |
3832 | 3833 |
| |
3833 | 3834 |
| |
|
Lines changed: 1 addition & 1 deletion
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2556 | 2556 |
| |
2557 | 2557 |
| |
2558 | 2558 |
| |
2559 |
| - | |
| 2559 | + | |
2560 | 2560 |
| |
2561 | 2561 |
| |
2562 | 2562 |
| |
|
Lines changed: 3 additions & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
3692 | 3692 |
| |
3693 | 3693 |
| |
3694 | 3694 |
| |
| 3695 | + | |
| 3696 | + | |
| 3697 | + | |
3695 | 3698 |
| |
3696 | 3699 |
| |
3697 | 3700 |
| |
|
Lines changed: 77 additions & 19 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
167 | 167 |
| |
168 | 168 |
| |
169 | 169 |
| |
170 |
| - | |
| 170 | + | |
| 171 | + | |
171 | 172 |
| |
172 | 173 |
| |
173 | 174 |
| |
| |||
201 | 202 |
| |
202 | 203 |
| |
203 | 204 |
| |
204 |
| - | |
| 205 | + | |
| 206 | + | |
205 | 207 |
| |
206 | 208 |
| |
207 | 209 |
| |
| |||
223 | 225 |
| |
224 | 226 |
| |
225 | 227 |
| |
226 |
| - | |
| 228 | + | |
| 229 | + | |
227 | 230 |
| |
228 | 231 |
| |
229 | 232 |
| |
| |||
424 | 427 |
| |
425 | 428 |
| |
426 | 429 |
| |
427 |
| - | |
| 430 | + | |
| 431 | + | |
428 | 432 |
| |
429 | 433 |
| |
430 | 434 |
| |
| |||
988 | 992 |
| |
989 | 993 |
| |
990 | 994 |
| |
991 |
| - | |
| 995 | + | |
| 996 | + | |
992 | 997 |
| |
993 | 998 |
| |
994 | 999 |
| |
| |||
997 | 1002 |
| |
998 | 1003 |
| |
999 | 1004 |
| |
| 1005 | + | |
| 1006 | + | |
| 1007 | + | |
| 1008 | + | |
1000 | 1009 |
| |
1001 | 1010 |
| |
1002 | 1011 |
| |
| |||
1008 | 1017 |
| |
1009 | 1018 |
| |
1010 | 1019 |
| |
1011 |
| - | |
| 1020 | + | |
1012 | 1021 |
| |
1013 | 1022 |
| |
1014 | 1023 |
| |
| |||
1554 | 1563 |
| |
1555 | 1564 |
| |
1556 | 1565 |
| |
| 1566 | + | |
| 1567 | + | |
| 1568 | + | |
| 1569 | + | |
1557 | 1570 |
| |
1558 | 1571 |
| |
1559 | 1572 |
| |
| |||
1610 | 1623 |
| |
1611 | 1624 |
| |
1612 | 1625 |
| |
1613 |
| - | |
1614 |
| - | |
1615 |
| - | |
1616 |
| - | |
1617 |
| - | |
1618 |
| - | |
1619 |
| - | |
1620 |
| - | |
1621 |
| - | |
1622 |
| - | |
1623 |
| - | |
1624 |
| - | |
1625 | 1626 |
| |
1626 | 1627 |
| |
1627 | 1628 |
| |
| |||
1633 | 1634 |
| |
1634 | 1635 |
| |
1635 | 1636 |
| |
1636 |
| - | |
| 1637 | + | |
| 1638 | + | |
| 1639 | + | |
| 1640 | + | |
| 1641 | + | |
| 1642 | + | |
| 1643 | + | |
| 1644 | + | |
| 1645 | + | |
| 1646 | + | |
| 1647 | + | |
| 1648 | + | |
| 1649 | + | |
| 1650 | + | |
| 1651 | + | |
| 1652 | + | |
| 1653 | + | |
| 1654 | + | |
| 1655 | + | |
| 1656 | + | |
| 1657 | + | |
| 1658 | + | |
| 1659 | + | |
| 1660 | + | |
| 1661 | + | |
| 1662 | + | |
| 1663 | + | |
| 1664 | + | |
| 1665 | + | |
| 1666 | + | |
| 1667 | + | |
| 1668 | + | |
| 1669 | + | |
| 1670 | + | |
| 1671 | + | |
| 1672 | + | |
| 1673 | + | |
| 1674 | + | |
| 1675 | + | |
| 1676 | + | |
| 1677 | + | |
| 1678 | + | |
| 1679 | + | |
| 1680 | + | |
| 1681 | + | |
| 1682 | + | |
| 1683 | + | |
| 1684 | + | |
| 1685 | + | |
| 1686 | + | |
| 1687 | + | |
| 1688 | + | |
| 1689 | + | |
| 1690 | + | |
| 1691 | + | |
| 1692 | + | |
| 1693 | + | |
| 1694 | + | |
1637 | 1695 |
| |
1638 | 1696 |
| |
1639 | 1697 |
| |
|
Lines changed: 8 additions & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
54 | 54 |
| |
55 | 55 |
| |
56 | 56 |
| |
| 57 | + | |
57 | 58 |
| |
58 | 59 |
| |
59 | 60 |
| |
| |||
214 | 215 |
| |
215 | 216 |
| |
216 | 217 |
| |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
217 | 225 |
| |
218 | 226 |
| |
219 | 227 |
| |
|
Lines changed: 1 addition & 1 deletion
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2186 | 2186 |
| |
2187 | 2187 |
| |
2188 | 2188 |
| |
2189 |
| - | |
| 2189 | + | |
2190 | 2190 |
| |
2191 | 2191 |
| |
2192 | 2192 |
| |
|
Lines changed: 1 addition & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
1197 | 1197 |
| |
1198 | 1198 |
| |
1199 | 1199 |
| |
| 1200 | + | |
1200 | 1201 |
| |
1201 | 1202 |
| |
1202 | 1203 |
| |
|
Lines changed: 1 addition & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
947 | 947 |
| |
948 | 948 |
| |
949 | 949 |
| |
| 950 | + | |
950 | 951 |
| |
951 | 952 |
| |
952 | 953 |
| |
|
Lines changed: 1 addition & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2386 | 2386 |
| |
2387 | 2387 |
| |
2388 | 2388 |
| |
| 2389 | + | |
2389 | 2390 |
| |
2390 | 2391 |
| |
2391 | 2392 |
| |
|
Lines changed: 4 additions & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
56 | 56 |
| |
57 | 57 |
| |
58 | 58 |
| |
| 59 | + | |
59 | 60 |
| |
60 | 61 |
| |
61 | 62 |
| |
| |||
567 | 568 |
| |
568 | 569 |
| |
569 | 570 |
| |
| 571 | + | |
| 572 | + | |
| 573 | + | |
570 | 574 |
| |
571 | 575 |
| |
572 | 576 |
| |
|
Lines changed: 8 additions & 1 deletion
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
147 | 147 |
| |
148 | 148 |
| |
149 | 149 |
| |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
150 | 154 |
| |
151 | 155 |
| |
152 | 156 |
| |
| |||
246 | 250 |
| |
247 | 251 |
| |
248 | 252 |
| |
249 |
| - | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
250 | 257 |
| |
251 | 258 |
| |
252 | 259 |
| |
|
Lines changed: 2 additions & 1 deletion
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2086 | 2086 |
| |
2087 | 2087 |
| |
2088 | 2088 |
| |
2089 |
| - | |
| 2089 | + | |
| 2090 | + | |
2090 | 2091 |
| |
2091 | 2092 |
| |
2092 | 2093 |
| |
|
0 commit comments
Comments
(0)