Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit98edd61

Browse files
committed
Fix datatype confusion with the new lossy GiST distance functions.
We can only support a lossy distance function when the distance function'sdatatype is comparable with the original ordering operator's datatype.The distance function always returns a float8, so we are limited to float8,and float4 (by a hard-coded cast of the float8 to float4).In light of this limitation, it seems like a good idea to have a separate'recheck' flag for the ORDER BY expressions, so that if you have a non-lossydistance function, it still works with lossy quals. There are cases likethat with the build-in or contrib opclasses, but it's plausible.There was a hidden assumption that the ORDER BY values returned by GiSTmatch the original ordering operator's return type, but there are plentyof examples where that's not true, e.g. in btree_gist and pg_trgm. As longas the distance function is not lossy, we can tolerate that and just notreturn the distance to the executor (or rather, always return NULL). Theexecutor doesn't need the distances if there are no lossy results.There was another little bug: the recheck variable was not initializedbefore calling the distance function. That revealed the bigger issue,as the executor tried to reorder tuples that didn't need reordering, andthat failed because of the datatype mismatch.
1 parenta868931 commit98edd61

File tree

6 files changed

+78
-16
lines changed

6 files changed

+78
-16
lines changed

‎doc/src/sgml/gist.sgml

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -818,10 +818,15 @@ my_distance(PG_FUNCTION_ARGS)
818818
</para>
819819

820820
<para>
821-
The result value can be any finite <type>float8</> value. (Infinity and
822-
minus infinity are used internally to handle cases such as nulls, so it
823-
is not recommended that <function>distance</> functions return these
824-
values.)
821+
If the distance function returns *recheck=true for a leaf node, the
822+
original ordering operator's return type must be float8 or float4, and
823+
the distance function's return value must be comparable with the actual
824+
distance operator. Otherwise, the distance function's return type can
825+
be any finit <type>float8</> value, as long as the relative order of
826+
the returned values matches the order returned by the ordering operator.
827+
(Infinity and minus infinity are used internally to handle cases such as
828+
nulls, so it is not recommended that <function>distance</> functions
829+
return these values.)
825830
</para>
826831

827832
</listitem>

‎src/backend/access/gist/gistget.c

Lines changed: 44 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616

1717
#include"access/gist_private.h"
1818
#include"access/relscan.h"
19+
#include"catalog/pg_type.h"
1920
#include"miscadmin.h"
2021
#include"pgstat.h"
2122
#include"lib/pairingheap.h"
@@ -31,9 +32,11 @@
3132
* depending on whether the containing page is a leaf page or not.
3233
*
3334
* On success return for a heap tuple, *recheck_p is set to indicate whether
34-
*recheck is needed. We recheck if any of the consistent() or distance()
35+
*the quals need to be rechecked. We recheck if any of the consistent()
3536
* functions request it. recheck is not interesting when examining a non-leaf
3637
* entry, since we must visit the lower index page if there's any doubt.
38+
* Similarly, *recheck_distances_p is set to indicate whether the distances
39+
* need to be rechecked, and it is also ignored for non-leaf entries.
3740
*
3841
* If we are doing an ordered scan, so->distances[] is filled with distance
3942
* data from the distance() functions before returning success.
@@ -50,7 +53,8 @@ gistindex_keytest(IndexScanDesc scan,
5053
IndexTupletuple,
5154
Pagepage,
5255
OffsetNumberoffset,
53-
bool*recheck_p)
56+
bool*recheck_p,
57+
bool*recheck_distances_p)
5458
{
5559
GISTScanOpaqueso= (GISTScanOpaque)scan->opaque;
5660
GISTSTATE*giststate=so->giststate;
@@ -60,6 +64,7 @@ gistindex_keytest(IndexScanDesc scan,
6064
Relationr=scan->indexRelation;
6165

6266
*recheck_p= false;
67+
*recheck_distances_p= false;
6368

6469
/*
6570
* If it's a leftover invalid tuple from pre-9.1, treat it as a match with
@@ -194,21 +199,22 @@ gistindex_keytest(IndexScanDesc scan,
194199
* use.)
195200
*
196201
* Distance functions get a recheck argument as well. In this
197-
* case the returned distance is the lower bound of distance
198-
* and needs to be rechecked. We return single recheck flag
199-
* which means that both quals and distances are to be
200-
* rechecked.
202+
* case the returned distance is the lower bound of distance and
203+
* needs to be rechecked. We return single recheck flag which
204+
* means that both quals and distances are to be rechecked. We
205+
* initialize the flag to 'false'. The flag was added in version
206+
* 9.5 and the distance operators written before that won't know
207+
* about the flag, and are never lossy.
201208
*/
209+
recheck= false;
202210
dist=FunctionCall5Coll(&key->sk_func,
203211
key->sk_collation,
204212
PointerGetDatum(&de),
205213
key->sk_argument,
206214
Int32GetDatum(key->sk_strategy),
207215
ObjectIdGetDatum(key->sk_subtype),
208216
PointerGetDatum(&recheck));
209-
210-
*recheck_p |=recheck;
211-
217+
*recheck_distances_p |=recheck;
212218
*distance_p=DatumGetFloat8(dist);
213219
}
214220

@@ -310,14 +316,16 @@ gistScanPage(IndexScanDesc scan, GISTSearchItem *pageItem, double *myDistances,
310316
IndexTupleit= (IndexTuple)PageGetItem(page,PageGetItemId(page,i));
311317
boolmatch;
312318
boolrecheck;
319+
boolrecheck_distances;
313320

314321
/*
315322
* Must call gistindex_keytest in tempCxt, and clean up any leftover
316323
* junk afterward.
317324
*/
318325
oldcxt=MemoryContextSwitchTo(so->giststate->tempCxt);
319326

320-
match=gistindex_keytest(scan,it,page,i,&recheck);
327+
match=gistindex_keytest(scan,it,page,i,
328+
&recheck,&recheck_distances);
321329

322330
MemoryContextSwitchTo(oldcxt);
323331
MemoryContextReset(so->giststate->tempCxt);
@@ -375,6 +383,7 @@ gistScanPage(IndexScanDesc scan, GISTSearchItem *pageItem, double *myDistances,
375383
item->blkno=InvalidBlockNumber;
376384
item->data.heap.heapPtr=it->t_tid;
377385
item->data.heap.recheck=recheck;
386+
item->data.heap.recheckDistances=recheck_distances;
378387

379388
/*
380389
* In an index-only scan, also fetch the data from the tuple.
@@ -461,10 +470,33 @@ getNextNearest(IndexScanDesc scan)
461470
/* found a heap item at currently minimal distance */
462471
scan->xs_ctup.t_self=item->data.heap.heapPtr;
463472
scan->xs_recheck=item->data.heap.recheck;
473+
scan->xs_recheckorderby=item->data.heap.recheckDistances;
464474
for (i=0;i<scan->numberOfOrderBys;i++)
465475
{
466-
scan->xs_orderbyvals[i]=Float8GetDatum(item->distances[i]);
467-
scan->xs_orderbynulls[i]= false;
476+
if (so->orderByTypes[i]==FLOAT8OID)
477+
{
478+
scan->xs_orderbyvals[i]=Float8GetDatum(item->distances[i]);
479+
scan->xs_orderbynulls[i]= false;
480+
}
481+
elseif (so->orderByTypes[i]==FLOAT4OID)
482+
{
483+
scan->xs_orderbyvals[i]=Float4GetDatum((float4)item->distances[i]);
484+
scan->xs_orderbynulls[i]= false;
485+
}
486+
else
487+
{
488+
/*
489+
* If the ordering operator's return value is anything
490+
* else, we don't know how to convert the float8 bound
491+
* calculated by the distance function to that. The
492+
* executor won't actually need the order by values we
493+
* return here, if there are no lossy results, so only
494+
* insist on the datatype if the *recheck is set.
495+
*/
496+
if (scan->xs_recheckorderby)
497+
elog(ERROR,"GiST operator family's FOR ORDER BY operator must return float8 or float4 if the distance function is lossy");
498+
scan->xs_orderbynulls[i]= true;
499+
}
468500
}
469501

470502
/* in an index-only scan, also return the reconstructed tuple. */

‎src/backend/access/gist/gistscan.c

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
#include"access/gist_private.h"
1818
#include"access/gistscan.h"
1919
#include"access/relscan.h"
20+
#include"utils/lsyscache.h"
2021
#include"utils/memutils.h"
2122
#include"utils/rel.h"
2223

@@ -263,6 +264,8 @@ gistrescan(PG_FUNCTION_ARGS)
263264
memmove(scan->orderByData,orderbys,
264265
scan->numberOfOrderBys*sizeof(ScanKeyData));
265266

267+
so->orderByTypes= (Oid*)palloc(scan->numberOfOrderBys*sizeof(Oid));
268+
266269
/*
267270
* Modify the order-by key so that the Distance method is called for
268271
* all comparisons. The original operator is passed to the Distance
@@ -281,6 +284,19 @@ gistrescan(PG_FUNCTION_ARGS)
281284
GIST_DISTANCE_PROC,skey->sk_attno,
282285
RelationGetRelationName(scan->indexRelation));
283286

287+
/*
288+
* Look up the datatype returned by the original ordering operator.
289+
* GiST always uses a float8 for the distance function, but the
290+
* ordering operator could be anything else.
291+
*
292+
* XXX: The distance function is only allowed to be lossy if the
293+
* ordering operator's result type is float4 or float8. Otherwise
294+
* we don't know how to return the distance to the executor. But
295+
* we cannot check that here, as we won't know if the distance
296+
* function is lossy until it returns *recheck = true for the
297+
* first time.
298+
*/
299+
so->orderByTypes[i]=get_func_rettype(skey->sk_func.fn_oid);
284300
fmgr_info_copy(&(skey->sk_func),finfo,so->giststate->scanCxt);
285301

286302
/* Restore prior fn_extra pointers, if not first time */

‎src/backend/executor/nodeIndexscan.c

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,7 +236,12 @@ IndexNextWithReorder(IndexScanState *node)
236236
InstrCountFiltered2(node,1);
237237
gotonext_indextuple;
238238
}
239+
}
239240

241+
if (scandesc->xs_recheckorderby)
242+
{
243+
econtext->ecxt_scantuple=slot;
244+
ResetExprContext(econtext);
240245
EvalOrderByExpressions(node,econtext);
241246

242247
/*

‎src/include/access/gist_private.h

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,7 @@ typedef struct GISTSearchHeapItem
121121
{
122122
ItemPointerDataheapPtr;
123123
boolrecheck;/* T if quals must be rechecked */
124+
boolrecheckDistances;/* T if distances must be rechecked */
124125
IndexTupleftup;/* data fetched back from the index, used in
125126
* index-only scans */
126127
}GISTSearchHeapItem;
@@ -150,6 +151,8 @@ typedef struct GISTSearchItem
150151
typedefstructGISTScanOpaqueData
151152
{
152153
GISTSTATE*giststate;/* index information, see above */
154+
Oid*orderByTypes;/* datatypes of ORDER BY expressions */
155+
153156
pairingheap*queue;/* queue of unvisited items */
154157
MemoryContextqueueCxt;/* context holding the queue */
155158
boolqual_ok;/* false if qual can never be satisfied */

‎src/include/access/relscan.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,7 @@ typedef struct IndexScanDescData
9999
*/
100100
Datum*xs_orderbyvals;
101101
bool*xs_orderbynulls;
102+
boolxs_recheckorderby;/* T means ORDER BY exprs must be rechecked */
102103

103104
/* state data for traversing HOT chains in index_getnext */
104105
boolxs_continue_hot;/* T if must keep walking HOT chain */

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp