Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitb676ac4

Browse files
committed
Optimize creation of slots for FDW bulk inserts
Commitb663a41 introduced bulk inserts for FDW, but the handling oftuple slots turned out to be problematic for two reasons. Firstly, theslots were re-created for each individual batch. Secondly, all slotsreferenced the same tuple descriptor - with reasonably small batchesthis is not an issue, but with large batches this triggers O(N^2)behavior in the resource owner code.These two issues work against each other - to reduce the number of timesa slot has to be created/dropped, larger batches are needed. However,the larger the batch, the more expensive the resource owner gets. Forpractical batch sizes (100 - 1000) this would not be a big problem, asthe benefits (latency savings) greatly exceed the resource owner costs.But for extremely large batches it might be much worse, possibly evenlosing with non-batching mode.Fixed by initializing tuple slots only once (and reusing them acrossbatches) and by using a new tuple descriptor copy for each slot.Discussion:https://postgr.es/m/ebbbcc7d-4286-8c28-0272-61b4753af761%40enterprisedb.com
1 parent96540f8 commitb676ac4

File tree

2 files changed

+37
-16
lines changed

2 files changed

+37
-16
lines changed

‎src/backend/executor/nodeModifyTable.c‎

Lines changed: 36 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -703,16 +703,31 @@ ExecInsert(ModifyTableState *mtstate,
703703
resultRelInfo->ri_BatchSize);
704704
}
705705

706-
resultRelInfo->ri_Slots[resultRelInfo->ri_NumSlots]=
707-
MakeSingleTupleTableSlot(slot->tts_tupleDescriptor,
708-
slot->tts_ops);
709-
ExecCopySlot(resultRelInfo->ri_Slots[resultRelInfo->ri_NumSlots],
710-
slot);
711-
resultRelInfo->ri_PlanSlots[resultRelInfo->ri_NumSlots]=
712-
MakeSingleTupleTableSlot(planSlot->tts_tupleDescriptor,
713-
planSlot->tts_ops);
714-
ExecCopySlot(resultRelInfo->ri_PlanSlots[resultRelInfo->ri_NumSlots],
715-
planSlot);
706+
/*
707+
* Initialize the batch slots. We don't know how many slots will be
708+
* needed, so we initialize them as the batch grows, and we keep
709+
* them across batches. To mitigate an inefficiency in how resource
710+
* owner handles objects with many references (as with many slots
711+
* all referencing the same tuple descriptor) we copy the tuple
712+
* descriptor for each slot.
713+
*/
714+
if (resultRelInfo->ri_NumSlots >=resultRelInfo->ri_NumSlotsInitialized)
715+
{
716+
TupleDesctdesc=CreateTupleDescCopy(slot->tts_tupleDescriptor);
717+
718+
resultRelInfo->ri_Slots[resultRelInfo->ri_NumSlots]=
719+
MakeSingleTupleTableSlot(tdesc,slot->tts_ops);
720+
ExecCopySlot(resultRelInfo->ri_Slots[resultRelInfo->ri_NumSlots],
721+
slot);
722+
723+
resultRelInfo->ri_PlanSlots[resultRelInfo->ri_NumSlots]=
724+
MakeSingleTupleTableSlot(tdesc,planSlot->tts_ops);
725+
ExecCopySlot(resultRelInfo->ri_PlanSlots[resultRelInfo->ri_NumSlots],
726+
planSlot);
727+
728+
/* remember how many batch slots we initialized */
729+
resultRelInfo->ri_NumSlotsInitialized++;
730+
}
716731

717732
resultRelInfo->ri_NumSlots++;
718733

@@ -1034,12 +1049,6 @@ ExecBatchInsert(ModifyTableState *mtstate,
10341049

10351050
if (canSetTag&&numInserted>0)
10361051
estate->es_processed+=numInserted;
1037-
1038-
for (i=0;i<numSlots;i++)
1039-
{
1040-
ExecDropSingleTupleTableSlot(slots[i]);
1041-
ExecDropSingleTupleTableSlot(planSlots[i]);
1042-
}
10431052
}
10441053

10451054
/* ----------------------------------------------------------------
@@ -3162,13 +3171,24 @@ ExecEndModifyTable(ModifyTableState *node)
31623171
*/
31633172
for (i=0;i<node->mt_nrels;i++)
31643173
{
3174+
intj;
31653175
ResultRelInfo*resultRelInfo=node->resultRelInfo+i;
31663176

31673177
if (!resultRelInfo->ri_usesFdwDirectModify&&
31683178
resultRelInfo->ri_FdwRoutine!=NULL&&
31693179
resultRelInfo->ri_FdwRoutine->EndForeignModify!=NULL)
31703180
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
31713181
resultRelInfo);
3182+
3183+
/*
3184+
* Cleanup the initialized batch slots. This only matters for FDWs with
3185+
* batching, but the other cases will have ri_NumSlotsInitialized == 0.
3186+
*/
3187+
for (j=0;j<resultRelInfo->ri_NumSlotsInitialized;j++)
3188+
{
3189+
ExecDropSingleTupleTableSlot(resultRelInfo->ri_Slots[j]);
3190+
ExecDropSingleTupleTableSlot(resultRelInfo->ri_PlanSlots[j]);
3191+
}
31723192
}
31733193

31743194
/*

‎src/include/nodes/execnodes.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -462,6 +462,7 @@ typedef struct ResultRelInfo
462462

463463
/* batch insert stuff */
464464
intri_NumSlots;/* number of slots in the array */
465+
intri_NumSlotsInitialized;/* number of initialized slots */
465466
intri_BatchSize;/* max slots inserted in a single batch */
466467
TupleTableSlot**ri_Slots;/* input tuples for batch insert */
467468
TupleTableSlot**ri_PlanSlots;

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp