Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit0f57382

Browse files
committed
Use ExprStates for hashing in GROUP BY and SubPlans
This speeds up obtaining hash values for GROUP BY and hashed SubPlans byusing the ExprState support for hashing, thus allowing JIT compilation forobtaining hash values for these operations.This, even without JIT compilation, has been shown to improve HashAggregate performance in some cases by around 15% and hashed NOT INqueries in one case by over 30%, however, real-world cases are likely tosee smaller gains as the test cases used were purposefully designed tohave high hashing overheads by keeping the hash table small to preventadditional memory overheads that would be a factor when working with largehash tables.In passing, fix a hypothetical bug in ExecBuildHash32Expr() so that theinitial value is stored directly in the ExprState's result field ifthere are no expressions to hash. None of the current users of thisfunction use an initial value, so the bug is only hypothetical.Reviewed-by: Andrei Lepikhov <lepihov@gmail.com>Discussion:https://postgr.es/m/CAApHDvpYSO3kc9UryMevWqthTBrxgfd9djiAjKHMPUSQeX9vdQ@mail.gmail.com
1 parenta435674 commit0f57382

File tree

5 files changed

+224
-66
lines changed

5 files changed

+224
-66
lines changed

‎src/backend/executor/execExpr.c

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3978,6 +3978,161 @@ ExecBuildAggTransCall(ExprState *state, AggState *aggstate,
39783978
}
39793979
}
39803980

3981+
/*
3982+
* Build an ExprState that calls the given hash function(s) on the attnums
3983+
* given by 'keyColIdx' . When numCols > 1, the hash values returned by each
3984+
* hash function are combined to produce a single hash value.
3985+
*
3986+
* desc: tuple descriptor for the to-be-hashed columns
3987+
* ops: TupleTableSlotOps to use for the give TupleDesc
3988+
* hashfunctions: FmgrInfos for each hash function to call, one per numCols.
3989+
* These are used directly in the returned ExprState so must remain allocated.
3990+
* collations: collation to use when calling the hash function.
3991+
* numCols: array length of hashfunctions, collations and keyColIdx.
3992+
* parent: PlanState node that the resulting ExprState will be evaluated at
3993+
* init_value: Normally 0, but can be set to other values to seed the hash
3994+
* with. Non-zero is marginally slower, so best to only use if it's provably
3995+
* worthwhile.
3996+
*/
3997+
ExprState*
3998+
ExecBuildHash32FromAttrs(TupleDescdesc,constTupleTableSlotOps*ops,
3999+
FmgrInfo*hashfunctions,Oid*collations,
4000+
intnumCols,AttrNumber*keyColIdx,
4001+
PlanState*parent,uint32init_value)
4002+
{
4003+
ExprState*state=makeNode(ExprState);
4004+
ExprEvalStepscratch= {0};
4005+
NullableDatum*iresult=NULL;
4006+
intptr_topcode;
4007+
AttrNumberlast_attnum=0;
4008+
4009+
Assert(numCols >=0);
4010+
4011+
state->parent=parent;
4012+
4013+
/*
4014+
* Make a place to store intermediate hash values between subsequent
4015+
* hashing of individual columns. We only need this if there is more than
4016+
* one column to hash or an initial value plus one column.
4017+
*/
4018+
if ((int64)numCols+ (init_value!=0)>1)
4019+
iresult=palloc(sizeof(NullableDatum));
4020+
4021+
/* find the highest attnum so we deform the tuple to that point */
4022+
for (inti=0;i<numCols;i++)
4023+
last_attnum=Max(last_attnum,keyColIdx[i]);
4024+
4025+
scratch.opcode=EEOP_INNER_FETCHSOME;
4026+
scratch.d.fetch.last_var=last_attnum;
4027+
scratch.d.fetch.fixed= false;
4028+
scratch.d.fetch.kind=ops;
4029+
scratch.d.fetch.known_desc=desc;
4030+
if (ExecComputeSlotInfo(state,&scratch))
4031+
ExprEvalPushStep(state,&scratch);
4032+
4033+
if (init_value==0)
4034+
{
4035+
/*
4036+
* No initial value, so we can assign the result of the hash function
4037+
* for the first attribute without having to concern ourselves with
4038+
* combining the result with any initial value.
4039+
*/
4040+
opcode=EEOP_HASHDATUM_FIRST;
4041+
}
4042+
else
4043+
{
4044+
/*
4045+
* Set up operation to set the initial value. Normally we store this
4046+
* in the intermediate hash value location, but if there are no
4047+
* columns to hash, store it in the ExprState's result field.
4048+
*/
4049+
scratch.opcode=EEOP_HASHDATUM_SET_INITVAL;
4050+
scratch.d.hashdatum_initvalue.init_value=UInt32GetDatum(init_value);
4051+
scratch.resvalue=numCols>0 ?&iresult->value :&state->resvalue;
4052+
scratch.resnull=numCols>0 ?&iresult->isnull :&state->resnull;
4053+
4054+
ExprEvalPushStep(state,&scratch);
4055+
4056+
/*
4057+
* When using an initial value use the NEXT32 ops as the FIRST ops
4058+
* would overwrite the stored initial value.
4059+
*/
4060+
opcode=EEOP_HASHDATUM_NEXT32;
4061+
}
4062+
4063+
for (inti=0;i<numCols;i++)
4064+
{
4065+
FmgrInfo*finfo;
4066+
FunctionCallInfofcinfo;
4067+
Oidinputcollid=collations[i];
4068+
AttrNumberattnum=keyColIdx[i]-1;
4069+
4070+
finfo=&hashfunctions[i];
4071+
fcinfo=palloc0(SizeForFunctionCallInfo(1));
4072+
4073+
/* Initialize function call parameter structure too */
4074+
InitFunctionCallInfoData(*fcinfo,finfo,1,inputcollid,NULL,NULL);
4075+
4076+
/*
4077+
* Fetch inner Var for this attnum and store it in the 1st arg of the
4078+
* hash func.
4079+
*/
4080+
scratch.opcode=EEOP_INNER_VAR;
4081+
scratch.resvalue=&fcinfo->args[0].value;
4082+
scratch.resnull=&fcinfo->args[0].isnull;
4083+
scratch.d.var.attnum=attnum;
4084+
scratch.d.var.vartype=TupleDescAttr(desc,attnum)->atttypid;
4085+
4086+
ExprEvalPushStep(state,&scratch);
4087+
4088+
/* Call the hash function */
4089+
scratch.opcode=opcode;
4090+
4091+
if (i==numCols-1)
4092+
{
4093+
/*
4094+
* The result for hashing the final column is stored in the
4095+
* ExprState.
4096+
*/
4097+
scratch.resvalue=&state->resvalue;
4098+
scratch.resnull=&state->resnull;
4099+
}
4100+
else
4101+
{
4102+
Assert(iresult!=NULL);
4103+
4104+
/* intermediate values are stored in an intermediate result */
4105+
scratch.resvalue=&iresult->value;
4106+
scratch.resnull=&iresult->isnull;
4107+
}
4108+
4109+
/*
4110+
* NEXT32 opcodes need to look at the intermediate result. We might
4111+
* as well just set this for all ops. FIRSTs won't look at it.
4112+
*/
4113+
scratch.d.hashdatum.iresult=iresult;
4114+
4115+
scratch.d.hashdatum.finfo=finfo;
4116+
scratch.d.hashdatum.fcinfo_data=fcinfo;
4117+
scratch.d.hashdatum.fn_addr=finfo->fn_addr;
4118+
scratch.d.hashdatum.jumpdone=-1;
4119+
4120+
ExprEvalPushStep(state,&scratch);
4121+
4122+
/* subsequent attnums must be combined with the previous */
4123+
opcode=EEOP_HASHDATUM_NEXT32;
4124+
}
4125+
4126+
scratch.resvalue=NULL;
4127+
scratch.resnull=NULL;
4128+
scratch.opcode=EEOP_DONE;
4129+
ExprEvalPushStep(state,&scratch);
4130+
4131+
ExecReadyExpr(state);
4132+
4133+
returnstate;
4134+
}
4135+
39814136
/*
39824137
* Build an ExprState that calls the given hash function(s) on the given
39834138
* 'hash_exprs'. When multiple expressions are present, the hash values

‎src/backend/executor/execGrouping.c

Lines changed: 34 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,7 @@ BuildTupleHashTableExt(PlanState *parent,
169169
Sizehash_mem_limit;
170170
MemoryContextoldcontext;
171171
boolallow_jit;
172+
uint32hash_iv=0;
172173

173174
Assert(nbuckets>0);
174175

@@ -183,14 +184,13 @@ BuildTupleHashTableExt(PlanState *parent,
183184

184185
hashtable->numCols=numCols;
185186
hashtable->keyColIdx=keyColIdx;
186-
hashtable->tab_hash_funcs=hashfunctions;
187187
hashtable->tab_collations=collations;
188188
hashtable->tablecxt=tablecxt;
189189
hashtable->tempcxt=tempcxt;
190190
hashtable->entrysize=entrysize;
191191
hashtable->tableslot=NULL;/* will be made on first lookup */
192192
hashtable->inputslot=NULL;
193-
hashtable->in_hash_funcs=NULL;
193+
hashtable->in_hash_expr=NULL;
194194
hashtable->cur_eq_func=NULL;
195195

196196
/*
@@ -202,9 +202,7 @@ BuildTupleHashTableExt(PlanState *parent,
202202
* underestimated.
203203
*/
204204
if (use_variable_hash_iv)
205-
hashtable->hash_iv=murmurhash32(ParallelWorkerNumber);
206-
else
207-
hashtable->hash_iv=0;
205+
hash_iv=murmurhash32(ParallelWorkerNumber);
208206

209207
hashtable->hashtab=tuplehash_create(metacxt,nbuckets,hashtable);
210208

@@ -225,6 +223,16 @@ BuildTupleHashTableExt(PlanState *parent,
225223
*/
226224
allow_jit=metacxt!=tablecxt;
227225

226+
/* build hash ExprState for all columns */
227+
hashtable->tab_hash_expr=ExecBuildHash32FromAttrs(inputDesc,
228+
&TTSOpsMinimalTuple,
229+
hashfunctions,
230+
collations,
231+
numCols,
232+
keyColIdx,
233+
allow_jit ?parent :NULL,
234+
hash_iv);
235+
228236
/* build comparator for all columns */
229237
/* XXX: should we support non-minimal tuples for the inputslot? */
230238
hashtable->tab_eq_func=ExecBuildGroupingEqual(inputDesc,inputDesc,
@@ -316,7 +324,7 @@ LookupTupleHashEntry(TupleHashTable hashtable, TupleTableSlot *slot,
316324

317325
/* set up data needed by hash and match functions */
318326
hashtable->inputslot=slot;
319-
hashtable->in_hash_funcs=hashtable->tab_hash_funcs;
327+
hashtable->in_hash_expr=hashtable->tab_hash_expr;
320328
hashtable->cur_eq_func=hashtable->tab_eq_func;
321329

322330
local_hash=TupleHashTableHash_internal(hashtable->hashtab,NULL);
@@ -342,7 +350,7 @@ TupleHashTableHash(TupleHashTable hashtable, TupleTableSlot *slot)
342350
uint32hash;
343351

344352
hashtable->inputslot=slot;
345-
hashtable->in_hash_funcs=hashtable->tab_hash_funcs;
353+
hashtable->in_hash_expr=hashtable->tab_hash_expr;
346354

347355
/* Need to run the hash functions in short-lived context */
348356
oldContext=MemoryContextSwitchTo(hashtable->tempcxt);
@@ -370,7 +378,7 @@ LookupTupleHashEntryHash(TupleHashTable hashtable, TupleTableSlot *slot,
370378

371379
/* set up data needed by hash and match functions */
372380
hashtable->inputslot=slot;
373-
hashtable->in_hash_funcs=hashtable->tab_hash_funcs;
381+
hashtable->in_hash_expr=hashtable->tab_hash_expr;
374382
hashtable->cur_eq_func=hashtable->tab_eq_func;
375383

376384
entry=LookupTupleHashEntry_internal(hashtable,slot,isnew,hash);
@@ -386,14 +394,14 @@ LookupTupleHashEntryHash(TupleHashTable hashtable, TupleTableSlot *slot,
386394
* created if there's not a match. This is similar to the non-creating
387395
* case of LookupTupleHashEntry, except that it supports cross-type
388396
* comparisons, in which the given tuple is not of the same type as the
389-
* table entries. The caller must provide the hashfunctions to use for
390-
* the input tuple, as well as the equalityfunctions, since these may be
397+
* table entries. The caller must provide the hashExprState to use for
398+
* the input tuple, as well as the equalityExprState, since these may be
391399
* different from the table's internal functions.
392400
*/
393401
TupleHashEntry
394402
FindTupleHashEntry(TupleHashTablehashtable,TupleTableSlot*slot,
395403
ExprState*eqcomp,
396-
FmgrInfo*hashfunctions)
404+
ExprState*hashexpr)
397405
{
398406
TupleHashEntryentry;
399407
MemoryContextoldContext;
@@ -404,7 +412,7 @@ FindTupleHashEntry(TupleHashTable hashtable, TupleTableSlot *slot,
404412

405413
/* Set up data needed by hash and match functions */
406414
hashtable->inputslot=slot;
407-
hashtable->in_hash_funcs=hashfunctions;
415+
hashtable->in_hash_expr=hashexpr;
408416
hashtable->cur_eq_func=eqcomp;
409417

410418
/* Search the hash table */
@@ -421,25 +429,24 @@ FindTupleHashEntry(TupleHashTable hashtable, TupleTableSlot *slot,
421429
* copied into the table.
422430
*
423431
* Also, the caller must select an appropriate memory context for running
424-
* the hash functions. (dynahash.c doesn't change CurrentMemoryContext.)
432+
* the hash functions.
425433
*/
426434
staticuint32
427435
TupleHashTableHash_internal(structtuplehash_hash*tb,
428436
constMinimalTupletuple)
429437
{
430438
TupleHashTablehashtable= (TupleHashTable)tb->private_data;
431-
intnumCols=hashtable->numCols;
432-
AttrNumber*keyColIdx=hashtable->keyColIdx;
433-
uint32hashkey=hashtable->hash_iv;
439+
uint32hashkey;
434440
TupleTableSlot*slot;
435-
FmgrInfo*hashfunctions;
436-
inti;
441+
boolisnull;
437442

438443
if (tuple==NULL)
439444
{
440445
/* Process the current input tuple for the table */
441-
slot=hashtable->inputslot;
442-
hashfunctions=hashtable->in_hash_funcs;
446+
hashtable->exprcontext->ecxt_innertuple=hashtable->inputslot;
447+
hashkey=DatumGetUInt32(ExecEvalExpr(hashtable->in_hash_expr,
448+
hashtable->exprcontext,
449+
&isnull));
443450
}
444451
else
445452
{
@@ -449,38 +456,17 @@ TupleHashTableHash_internal(struct tuplehash_hash *tb,
449456
* (this case never actually occurs due to the way simplehash.h is
450457
* used, as the hash-value is stored in the entries)
451458
*/
452-
slot=hashtable->tableslot;
459+
slot=hashtable->exprcontext->ecxt_innertuple=hashtable->tableslot;
453460
ExecStoreMinimalTuple(tuple,slot, false);
454-
hashfunctions=hashtable->tab_hash_funcs;
455-
}
456-
457-
for (i=0;i<numCols;i++)
458-
{
459-
AttrNumberatt=keyColIdx[i];
460-
Datumattr;
461-
boolisNull;
462-
463-
/* combine successive hashkeys by rotating */
464-
hashkey=pg_rotate_left32(hashkey,1);
465-
466-
attr=slot_getattr(slot,att,&isNull);
467-
468-
if (!isNull)/* treat nulls as having hash key 0 */
469-
{
470-
uint32hkey;
471-
472-
hkey=DatumGetUInt32(FunctionCall1Coll(&hashfunctions[i],
473-
hashtable->tab_collations[i],
474-
attr));
475-
hashkey ^=hkey;
476-
}
461+
hashkey=DatumGetUInt32(ExecEvalExpr(hashtable->tab_hash_expr,
462+
hashtable->exprcontext,
463+
&isnull));
477464
}
478465

479466
/*
480-
* The way hashes are combined above, among each other and with the IV,
481-
* doesn't lead to good bit perturbation. As the IV's goal is to lead to
482-
* achieve that, perform a round of hashing of the combined hash -
483-
* resulting in near perfect perturbation.
467+
* The hashing done above, even with an initial value, doesn't tend to
468+
* result in good hash perturbation. Running the value produced above
469+
* through murmurhash32 leads to near perfect hash perturbation.
484470
*/
485471
returnmurmurhash32(hashkey);
486472
}

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp