NotificationsYou must be signed in to change notification settings
Fork28
Star151

Commit3d3bf62

committed

Omit null rows when setting the threshold for what's a most-common value.

As with the previous patch, large numbers of null rows could skew thiscalculation unfavorably, causing us to discard values that have alegitimate claim to be MCVs, since our definition of MCV is that it'smost common among the non-null population of the column. Hence, makethe numerator of avgcount be the number of non-null sample values notthe number of sample rows; likewise for maxmincount in thecompute_scalar_stats variant.Also, make the denominator be the number of distinct values actuallyobserved in the sample, rather than reversing it back out of the computedstadistinct. This avoids depending on the accuracy of the Haas-Stokesapproximation, and really it's what we want anyway; the threshold shoulddepend only on what we see in the sample, not on what we extrapolateabout the contents of the whole column.Alex Shulgin, reviewed by Tomas Vondra and myself

1 parent5cb8826 commit3d3bf62Copy full SHA for 3d3bf62

File tree

1 file changed

-11

lines changed

src/backend/commands
- analyze.c

1 file changed

-11

lines changed

`‎src/backend/commands/analyze.c`

Lines changed: 9 additions & 11 deletions

Original file line number	Diff line number	Diff line change
`@@ -2133,14 +2133,13 @@ compute_distinct_stats(VacAttrStatsP stats,`
`2133`	`2133`	`}`
`2134`	`2134`	`else`
`2135`	`2135`	`{`
`2136`		`-doublendistinct=stats->stadistinct;`
	`2136`	`+/* d here is the same as d in the Haas-Stokes formula */`
	`2137`	`+intd=nonnull_cnt-summultiple+nmultiple;`
`2137`	`2138`	`doubleavgcount,`
`2138`	`2139`	`mincount;`
`2139`	`2140`
`2140`		`-if (ndistinct<0)`
`2141`		`-ndistinct=-ndistinct*totalrows;`
`2142`		`-/* estimate # of occurrences in sample of a typical value */`
`2143`		`-avgcount= (double)samplerows /ndistinct;`
	`2141`	`+/* estimate # occurrences in sample of a typical nonnull value */`
	`2142`	`+avgcount= (double)nonnull_cnt / (double)d;`
`2144`	`2143`	`/* set minimum threshold count to store a value */`
`2145`	`2144`	`mincount=avgcount*1.25;`
`2146`	`2145`	`if (mincount<2)`
`@@ -2494,21 +2493,20 @@ compute_scalar_stats(VacAttrStatsP stats,`
`2494`	`2493`	`}`
`2495`	`2494`	`else`
`2496`	`2495`	`{`
`2497`		`-doublendistinct=stats->stadistinct;`
	`2496`	`+/* d here is the same as d in the Haas-Stokes formula */`
	`2497`	`+intd=ndistinct+toowide_cnt;`
`2498`	`2498`	`doubleavgcount,`
`2499`	`2499`	`mincount,`
`2500`	`2500`	`maxmincount;`
`2501`	`2501`
`2502`		`-if (ndistinct<0)`
`2503`		`-ndistinct=-ndistinct*totalrows;`
`2504`		`-/* estimate # of occurrences in sample of a typical value */`
`2505`		`-avgcount= (double)samplerows /ndistinct;`
	`2502`	`+/* estimate # occurrences in sample of a typical nonnull value */`
	`2503`	`+avgcount= (double)values_cnt / (double)d;`
`2506`	`2504`	`/* set minimum threshold count to store a value */`
`2507`	`2505`	`mincount=avgcount*1.25;`
`2508`	`2506`	`if (mincount<2)`
`2509`	`2507`	`mincount=2;`
`2510`	`2508`	`/* don't let threshold exceed 1/K, however */`
`2511`		`-maxmincount= (double)samplerows / (double)num_bins;`
	`2509`	`+maxmincount= (double)values_cnt / (double)num_bins;`
`2512`	`2510`	`if (mincount>maxmincount)`
`2513`	`2511`	`mincount=maxmincount;`
`2514`	`2512`	`if (num_mcv>track_cnt)`

0 commit comments

Comments

(0)

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Commit3d3bf62

File tree

1 file changed

1 file changed

`‎src/backend/commands/analyze.c`

0 commit comments