Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commite3f005d

Browse files
committed
Limit the number of index clauses considered in choose_bitmap_and().
classify_index_clause_usage() is O(N^2) in the number of distinct indexqual clauses it considers, because of its use of a simple search list tostore them. For nearly all queries, that's fine because only a few clauseswill be considered. But Alexander Kuzmenkov reported a machine-generatedquery with 80000 (!) index qual clauses, which caused this code to takeforever. Somewhat remarkably, this is the only O(N^2) behavior we nowhave for such a query, so let's fix it.We can get rid of the O(N^2) runtime for cases like this without muchdamage to the functionality of choose_bitmap_and() by separating outpaths with "too many" qual or pred clauses, and deeming them to alwaysbe nonredundant with other paths. Then their clauses needn't go intothe search list, so it doesn't get too long, but we don't lose theability to consider bitmap AND plans altogether. I set the thresholdfor "too many" to be 100 clauses per path, which should be plenty toensure no change in planning behavior for normal queries.There are other things we could do to make this go faster, but it's notclear that it's worth any additional effort. 80000 qual clauses requirea whole lot of work in many other places, too.The code's been like this for a long time, so back-patch to all supportedbranches. The troublesome query only works back to 9.5 (in 9.4 it failswith stack overflow in the parser); so I'm not sure that fixing this in9.4 has any real-world benefit, but perhaps it does.Discussion:https://postgr.es/m/90c5bdfa-d633-dabe-9889-3cf3e1acd443@postgrespro.ru
1 parentffb6898 commite3f005d

File tree

1 file changed

+31
-1
lines changed

1 file changed

+31
-1
lines changed

‎src/backend/optimizer/path/indxpath.c

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ typedef struct
6767
List*quals;/* the WHERE clauses it uses */
6868
List*preds;/* predicates of its partial index(es) */
6969
Bitmapset*clauseids;/* quals+preds represented as a bitmapset */
70+
boolunclassifiable;/* has too many quals+preds to process? */
7071
}PathClauseUsage;
7172

7273
/* Callback argument for ec_member_matches_indexcol */
@@ -1447,9 +1448,18 @@ choose_bitmap_and(PlannerInfo *root, RelOptInfo *rel, List *paths)
14471448
Path*ipath= (Path*)lfirst(l);
14481449

14491450
pathinfo=classify_index_clause_usage(ipath,&clauselist);
1451+
1452+
/* If it's unclassifiable, treat it as distinct from all others */
1453+
if (pathinfo->unclassifiable)
1454+
{
1455+
pathinfoarray[npaths++]=pathinfo;
1456+
continue;
1457+
}
1458+
14501459
for (i=0;i<npaths;i++)
14511460
{
1452-
if (bms_equal(pathinfo->clauseids,pathinfoarray[i]->clauseids))
1461+
if (!pathinfoarray[i]->unclassifiable&&
1462+
bms_equal(pathinfo->clauseids,pathinfoarray[i]->clauseids))
14531463
break;
14541464
}
14551465
if (i<npaths)
@@ -1484,6 +1494,10 @@ choose_bitmap_and(PlannerInfo *root, RelOptInfo *rel, List *paths)
14841494
* For each surviving index, consider it as an "AND group leader", and see
14851495
* whether adding on any of the later indexes results in an AND path with
14861496
* cheaper total cost than before. Then take the cheapest AND group.
1497+
*
1498+
* Note: paths that are either clauseless or unclassifiable will have
1499+
* empty clauseids, so that they will not be rejected by the clauseids
1500+
* filter here, nor will they cause later paths to be rejected by it.
14871501
*/
14881502
for (i=0;i<npaths;i++)
14891503
{
@@ -1711,6 +1725,21 @@ classify_index_clause_usage(Path *path, List **clauselist)
17111725
result->preds=NIL;
17121726
find_indexpath_quals(path,&result->quals,&result->preds);
17131727

1728+
/*
1729+
* Some machine-generated queries have outlandish numbers of qual clauses.
1730+
* To avoid getting into O(N^2) behavior even in this preliminary
1731+
* classification step, we want to limit the number of entries we can
1732+
* accumulate in *clauselist. Treat any path with more than 100 quals +
1733+
* preds as unclassifiable, which will cause calling code to consider it
1734+
* distinct from all other paths.
1735+
*/
1736+
if (list_length(result->quals)+list_length(result->preds)>100)
1737+
{
1738+
result->clauseids=NULL;
1739+
result->unclassifiable= true;
1740+
returnresult;
1741+
}
1742+
17141743
/* Build up a bitmapset representing the quals and preds */
17151744
clauseids=NULL;
17161745
foreach(lc,result->quals)
@@ -1728,6 +1757,7 @@ classify_index_clause_usage(Path *path, List **clauselist)
17281757
find_list_position(node,clauselist));
17291758
}
17301759
result->clauseids=clauseids;
1760+
result->unclassifiable= false;
17311761

17321762
returnresult;
17331763
}

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp