forked frompostgres/postgres
- Notifications
You must be signed in to change notification settings - Fork6
Commit98c7c71
committed
Fix extreme skew detection in Parallel Hash Join.
After repartitioning the inner side of a hash join that would haveexceeded the allowed size, we check if all the tuples from a parentpartition moved to one child partition. That is evidence that itcontains duplicate keys and later attempts to repartition will alsofail, so we should give up trying to limit memory (for lack of a betterfallback strategy).A thinko prevented the check from working correctly in partition 0 (theone that is partially loaded into memory already). Afterrepartitioning, we should check for extreme skew if the *parent*partition's space_exhausted flag was set, not the child partition's.The consequence was repeated futile repartitioning until per-partitiondata exceeded various limits including "ERROR: invalid DSA memory allocrequest size 1811939328", OS allocation failure, or temporary disk spaceerrors. (We could also do something about some of those symptoms, butthat's material for separate patches.)This problem only became likely when PostgreSQL 16 introduced supportfor Parallel Hash Right/Full Join, allowing NULL keys into the hashtable. Repartitioning always leaves NULL in partition 0, no matter howmany times you do it, because the hash value is all zero bits. That'sunlikely for other hashed values, but they might still have causedwasted extra effort before giving up.Back-patch to all supported releases.Reported-by: Craig Milhiser <craig@milhiser.com>Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>Discussion:https://postgr.es/m/CA%2BwnhO1OfgXbmXgC4fv_uu%3DOxcDQuHvfoQ4k0DFeB0Qqd-X-rQ%40mail.gmail.com1 parentd893a29 commit98c7c71
1 file changed
+12
-5
lines changedLines changed: 12 additions & 5 deletions
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
1228 | 1228 |
| |
1229 | 1229 |
| |
1230 | 1230 |
| |
| 1231 | + | |
1231 | 1232 |
| |
1232 | 1233 |
| |
1233 | 1234 |
| |
1234 | 1235 |
| |
1235 | 1236 |
| |
1236 | 1237 |
| |
1237 | 1238 |
| |
| 1239 | + | |
| 1240 | + | |
1238 | 1241 |
| |
1239 | 1242 |
| |
1240 | 1243 |
| |
1241 |
| - | |
| 1244 | + | |
| 1245 | + | |
| 1246 | + | |
1242 | 1247 |
| |
| 1248 | + | |
1243 | 1249 |
| |
1244 | 1250 |
| |
1245 |
| - | |
1246 |
| - | |
1247 |
| - | |
1248 | 1251 |
| |
1249 | 1252 |
| |
| 1253 | + | |
| 1254 | + | |
| 1255 | + | |
| 1256 | + | |
| 1257 | + | |
1250 | 1258 |
| |
1251 | 1259 |
| |
1252 | 1260 |
| |
1253 | 1261 |
| |
1254 | 1262 |
| |
1255 | 1263 |
| |
1256 |
| - | |
1257 | 1264 |
| |
1258 | 1265 |
| |
1259 | 1266 |
| |
|
0 commit comments
Comments
(0)