forked frompostgres/postgres
- Notifications
You must be signed in to change notification settings - Fork6
Commit3581cbd
committed
Fix handling of empty ranges and NULLs in BRIN
BRIN indexes did not properly distinguish between summaries for empty(no rows) and all-NULL ranges, treating them as essentially the samething. Summaries were initialized with allnulls=true, and opclassessimply reset allnulls to false when processing the first non-NULL value.This however produces incorrect results if the range starts with a NULLvalue (or a sequence of NULL values), in which case we forget the rangecontains NULL values when adding the first non-NULL value.This happens because the allnulls flag is used for two separatepurposes - to mark empty ranges (not representing any rows yet) andranges containing only NULL values.Opclasses don't know which of these cases it is, and so don't knowwhether to set hasnulls=true. Setting the flag in both cases would makeit correct, but it would also make BRIN indexes useless for queries withIS NULL clauses. All ranges start empty (and thus allnulls=true), so allranges would end up with either allnulls=true or hasnulls=true.The severity of the issue is somewhat reduced by the fact that it onlyhappens when adding values to an existing summary with allnulls=true.This can happen e.g. for small tables (because a summary for the firstrange exists for all BRIN indexes), or for tables with large fraction ofNULL values in the indexed columns.Bulk summarization (e.g. during CREATE INDEX or automatic summarization)that processes all values at once is not affected by this issue. In thiscase the flags were updated in a slightly different way, not forgettingthe NULL values.To identify empty ranges we use a new flag, stored in an unused bit inthe BRIN tuple header so the on-disk format remains the same. A matchingflag is added to BrinMemTuple, into a 3B gap after bt_placeholder.That means there's no risk of ABI breakage, although we don't actuallypass the BrinMemTuple to any public API.We could also skip storing index tuples for empty summaries, but thenwe'd have to always process such ranges - even if there are no rows inlarge parts of the table (e.g. after a bulk DELETE), it would stillrequire reading the pages etc. So we store them, but ignore them whenbuilding the bitmap.Backpatch to 11. The issue exists since BRIN indexes were introduced in9.5, but older releases are already EOL.Backpatch-through: 11Reviewed-by: Justin Pryzby, Matthias van de Meent, Alvaro HerreraDiscussion:https://postgr.es/m/402430e4-7d9d-6cf1-09ef-464d80afff3b@enterprisedb.com1 parent1158c8c commit3581cbd
File tree
5 files changed
+134
-8
lines changed- src
- backend/access/brin
- include/access
- test/modules/brin
- expected
- specs
5 files changed
+134
-8
lines changedLines changed: 112 additions & 1 deletion
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
592 | 592 |
| |
593 | 593 |
| |
594 | 594 |
| |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
595 | 606 |
| |
596 | 607 |
| |
597 | 608 |
| |
| |||
1606 | 1617 |
| |
1607 | 1618 |
| |
1608 | 1619 |
| |
| 1620 | + | |
| 1621 | + | |
| 1622 | + | |
| 1623 | + | |
| 1624 | + | |
| 1625 | + | |
| 1626 | + | |
| 1627 | + | |
| 1628 | + | |
| 1629 | + | |
| 1630 | + | |
| 1631 | + | |
| 1632 | + | |
| 1633 | + | |
| 1634 | + | |
| 1635 | + | |
| 1636 | + | |
| 1637 | + | |
| 1638 | + | |
| 1639 | + | |
| 1640 | + | |
| 1641 | + | |
| 1642 | + | |
| 1643 | + | |
| 1644 | + | |
| 1645 | + | |
| 1646 | + | |
| 1647 | + | |
| 1648 | + | |
| 1649 | + | |
| 1650 | + | |
| 1651 | + | |
| 1652 | + | |
| 1653 | + | |
| 1654 | + | |
| 1655 | + | |
| 1656 | + | |
| 1657 | + | |
| 1658 | + | |
| 1659 | + | |
| 1660 | + | |
| 1661 | + | |
| 1662 | + | |
| 1663 | + | |
| 1664 | + | |
| 1665 | + | |
| 1666 | + | |
| 1667 | + | |
| 1668 | + | |
| 1669 | + | |
| 1670 | + | |
| 1671 | + | |
| 1672 | + | |
| 1673 | + | |
| 1674 | + | |
| 1675 | + | |
| 1676 | + | |
| 1677 | + | |
1609 | 1678 |
| |
1610 | 1679 |
| |
1611 | 1680 |
| |
| |||
1711 | 1780 |
| |
1712 | 1781 |
| |
1713 | 1782 |
| |
1714 |
| - | |
| 1783 | + | |
| 1784 | + | |
| 1785 | + | |
1715 | 1786 |
| |
1716 | 1787 |
| |
1717 | 1788 |
| |
| |||
1725 | 1796 |
| |
1726 | 1797 |
| |
1727 | 1798 |
| |
| 1799 | + | |
1728 | 1800 |
| |
1729 | 1801 |
| |
1730 | 1802 |
| |
| 1803 | + | |
| 1804 | + | |
| 1805 | + | |
| 1806 | + | |
| 1807 | + | |
| 1808 | + | |
| 1809 | + | |
| 1810 | + | |
| 1811 | + | |
| 1812 | + | |
| 1813 | + | |
| 1814 | + | |
| 1815 | + | |
| 1816 | + | |
1731 | 1817 |
| |
1732 | 1818 |
| |
1733 | 1819 |
| |
| |||
1753 | 1839 |
| |
1754 | 1840 |
| |
1755 | 1841 |
| |
| 1842 | + | |
| 1843 | + | |
| 1844 | + | |
| 1845 | + | |
| 1846 | + | |
| 1847 | + | |
| 1848 | + | |
| 1849 | + | |
| 1850 | + | |
| 1851 | + | |
| 1852 | + | |
| 1853 | + | |
| 1854 | + | |
| 1855 | + | |
| 1856 | + | |
1756 | 1857 |
| |
1757 | 1858 |
| |
| 1859 | + | |
| 1860 | + | |
| 1861 | + | |
| 1862 | + | |
| 1863 | + | |
| 1864 | + | |
| 1865 | + | |
| 1866 | + | |
| 1867 | + | |
| 1868 | + | |
1758 | 1869 |
| |
1759 | 1870 |
| |
1760 | 1871 |
| |
|
Lines changed: 13 additions & 1 deletion
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
372 | 372 |
| |
373 | 373 |
| |
374 | 374 |
| |
| 375 | + | |
| 376 | + | |
| 377 | + | |
375 | 378 |
| |
376 | 379 |
| |
377 | 380 |
| |
| |||
399 | 402 |
| |
400 | 403 |
| |
401 | 404 |
| |
402 |
| - | |
| 405 | + | |
403 | 406 |
| |
404 | 407 |
| |
405 | 408 |
| |
| |||
489 | 492 |
| |
490 | 493 |
| |
491 | 494 |
| |
| 495 | + | |
| 496 | + | |
492 | 497 |
| |
493 | 498 |
| |
494 | 499 |
| |
| |||
527 | 532 |
| |
528 | 533 |
| |
529 | 534 |
| |
| 535 | + | |
| 536 | + | |
530 | 537 |
| |
531 | 538 |
| |
532 | 539 |
| |
| |||
560 | 567 |
| |
561 | 568 |
| |
562 | 569 |
| |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
563 | 575 |
| |
564 | 576 |
| |
565 | 577 |
| |
|
Lines changed: 4 additions & 2 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
44 | 44 |
| |
45 | 45 |
| |
46 | 46 |
| |
| 47 | + | |
47 | 48 |
| |
48 | 49 |
| |
49 | 50 |
| |
| |||
69 | 70 |
| |
70 | 71 |
| |
71 | 72 |
| |
72 |
| - | |
| 73 | + | |
73 | 74 |
| |
74 | 75 |
| |
75 | 76 |
| |
| |||
82 | 83 |
| |
83 | 84 |
| |
84 | 85 |
| |
85 |
| - | |
| 86 | + | |
86 | 87 |
| |
87 | 88 |
| |
88 | 89 |
| |
89 | 90 |
| |
90 | 91 |
| |
91 | 92 |
| |
| 93 | + | |
92 | 94 |
| |
93 | 95 |
| |
94 | 96 |
| |
|
Lines changed: 4 additions & 4 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
4 | 4 |
| |
5 | 5 |
| |
6 | 6 |
| |
7 |
| - | |
| 7 | + | |
8 | 8 |
| |
9 | 9 |
| |
10 | 10 |
| |
| |||
26 | 26 |
| |
27 | 27 |
| |
28 | 28 |
| |
29 |
| - | |
| 29 | + | |
30 | 30 |
| |
31 | 31 |
| |
32 | 32 |
| |
| |||
35 | 35 |
| |
36 | 36 |
| |
37 | 37 |
| |
38 |
| - | |
| 38 | + | |
39 | 39 |
| |
40 | 40 |
| |
41 | 41 |
| |
| |||
45 | 45 |
| |
46 | 46 |
| |
47 | 47 |
| |
48 |
| - | |
| 48 | + | |
49 | 49 |
| |
50 | 50 |
| |
51 | 51 |
|
Lines changed: 1 addition & 0 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
9 | 9 |
| |
10 | 10 |
| |
11 | 11 |
| |
| 12 | + | |
12 | 13 |
| |
13 | 14 |
| |
14 | 15 |
| |
|
0 commit comments
Comments
(0)