Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commite7cb7ee

Browse files
committed
Allow FDWs and custom scan providers to replace joins with scans.
Foreign data wrappers can use this capability for so-called "joinpushdown"; that is, instead of executing two separate foreign scansand then joining the results locally, they can generate a path whichperforms the join on the remote server and then is scanned locally.This commit does not extend postgres_fdw to take advantage of thiscapability; it just provides the infrastructure.Custom scan providers can use this in a similar way. Previously,it was only possible for a custom scan provider to scan a singlerelation. Now, it can scan an entire join tree, provided of coursethat it knows how to produce the same results that the join wouldhave produced if executed normally.KaiGai Kohei, reviewed by Shigeru Hanada, Ashutosh Bapat, and me.
1 parent2b22795 commite7cb7ee

File tree

20 files changed

+449
-72
lines changed

20 files changed

+449
-72
lines changed

‎doc/src/sgml/custom-scan.sgml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,28 @@ typedef struct CustomPath
8181
detailed below.
8282
</para>
8383

84+
<para>
85+
A custom scan provider can also add join paths; in this case, the scan
86+
must produce the same output as would normally be produced by the join
87+
it replaces. To do this, the join provider should set the following hook.
88+
This hook may be invoked repeatedly for the same pair of relations, with
89+
different combinations of inner and outer relations; it is the
90+
responsibility of the hook to minimize duplicated work.
91+
<programlisting>
92+
typedef void (*set_join_pathlist_hook_type) (PlannerInfo *root,
93+
RelOptInfo *joinrel,
94+
RelOptInfo *outerrel,
95+
RelOptInfo *innerrel,
96+
List *restrictlist,
97+
JoinType jointype,
98+
SpecialJoinInfo *sjinfo,
99+
SemiAntiJoinFactors *semifactors,
100+
Relids param_source_rels,
101+
Relids extra_lateral_rels);
102+
extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
103+
</programlisting>
104+
</para>
105+
84106
<sect2 id="custom-scan-path-callbacks">
85107
<title>Custom Path Callbacks</title>
86108

@@ -124,7 +146,9 @@ typedef struct CustomScan
124146
Scan scan;
125147
uint32 flags;
126148
List *custom_exprs;
149+
List *custom_ps_tlist;
127150
List *custom_private;
151+
List *custom_relids;
128152
const CustomScanMethods *methods;
129153
} CustomScan;
130154
</programlisting>
@@ -141,11 +165,27 @@ typedef struct CustomScan
141165
is only used by the custom scan provider itself. Plan trees must be able
142166
to be duplicated using <function>copyObject</>, so all the data stored
143167
within these two fields must consist of nodes that function can handle.
168+
<literal>custom_relids</> is set by the core code to the set of relations
169+
which this scan node must handle; except when this scan is replacing a
170+
join, it will have only one member.
144171
<structfield>methods</> must point to a (usually statically allocated)
145172
object implementing the required custom scan methods, which are further
146173
detailed below.
147174
</para>
148175

176+
<para>
177+
When a <structname>CustomScan</> scans a single relation,
178+
<structfield>scan.scanrelid</> should be the range table index of the table
179+
to be scanned, and <structfield>custom_ps_tlist</> should be
180+
<literal>NULL</>. When it replaces a join, <structfield>scan.scanrelid</>
181+
should be zero, and <structfield>custom_ps_tlist</> should be a list of
182+
<structname>TargetEntry</> nodes. This is necessary because, when a join
183+
is replaced, the target list cannot be constructed from the table
184+
definition. At execution time, this list will be used to initialize the
185+
tuple descriptor of the <structname>TupleTableSlot</>. It will also be
186+
used by <command>EXPLAIN</>, when deparsing.
187+
</para>
188+
149189
<sect2 id="custom-scan-plan-callbacks">
150190
<title>Custom Scan Callbacks</title>
151191
<para>

‎doc/src/sgml/fdwhandler.sgml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -598,6 +598,42 @@ IsForeignRelUpdatable (Relation rel);
598598

599599
</sect2>
600600

601+
<sect2>
602+
<title>FDW Routines For Remote Joins</title>
603+
<para>
604+
<programlisting>
605+
void
606+
GetForeignJoinPaths(PlannerInfo *root,
607+
RelOptInfo *joinrel,
608+
RelOptInfo *outerrel,
609+
RelOptInfo *innerrel,
610+
List *restrictlist,
611+
JoinType jointype,
612+
SpecialJoinInfo *sjinfo,
613+
SemiAntiJoinFactors *semifactors,
614+
Relids param_source_rels,
615+
Relids extra_lateral_rels);
616+
</programlisting>
617+
Create possible access paths for a join of two foreign tables managed
618+
by the same foreign data wrapper.
619+
This optional function is called during query planning.
620+
</para>
621+
<para>
622+
This function the FDW to add <structname>ForeignScan</> paths for the
623+
supplied <literal>joinrel</>. Typically, the FDW will send the whole
624+
join to the remote server as a single query, as performing the join
625+
remotely rather than locally is typically much more efficient.
626+
</para>
627+
<para>
628+
Since we cannot construct the slot descriptor for a remote join from
629+
the catalogs, the FDW should set the <structfield>scanrelid</> of the
630+
<structname>ForeignScan</> to zero and <structfield>fdw_ps_tlist</>
631+
to an appropriate list of <structfield>TargetEntry</> nodes.
632+
Junk entries will be ignored, but can be present for the benefit of
633+
deparsing performed by <command>EXPLAIN</>.
634+
</para>
635+
</sect2>
636+
601637
<sect2 id="fdw-callbacks-explain">
602638
<title>FDW Routines for <command>EXPLAIN</></title>
603639

‎src/backend/commands/explain.c‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -730,11 +730,17 @@ ExplainPreScanNode(PlanState *planstate, Bitmapset **rels_used)
730730
caseT_ValuesScan:
731731
caseT_CteScan:
732732
caseT_WorkTableScan:
733-
caseT_ForeignScan:
734-
caseT_CustomScan:
735733
*rels_used=bms_add_member(*rels_used,
736734
((Scan*)plan)->scanrelid);
737735
break;
736+
caseT_ForeignScan:
737+
*rels_used=bms_add_members(*rels_used,
738+
((ForeignScan*)plan)->fdw_relids);
739+
break;
740+
caseT_CustomScan:
741+
*rels_used=bms_add_members(*rels_used,
742+
((CustomScan*)plan)->custom_relids);
743+
break;
738744
caseT_ModifyTable:
739745
*rels_used=bms_add_member(*rels_used,
740746
((ModifyTable*)plan)->nominalRelation);
@@ -1072,9 +1078,12 @@ ExplainNode(PlanState *planstate, List *ancestors,
10721078
caseT_ValuesScan:
10731079
caseT_CteScan:
10741080
caseT_WorkTableScan:
1081+
ExplainScanTarget((Scan*)plan,es);
1082+
break;
10751083
caseT_ForeignScan:
10761084
caseT_CustomScan:
1077-
ExplainScanTarget((Scan*)plan,es);
1085+
if (((Scan*)plan)->scanrelid>0)
1086+
ExplainScanTarget((Scan*)plan,es);
10781087
break;
10791088
caseT_IndexScan:
10801089
{

‎src/backend/executor/execScan.c‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -251,6 +251,12 @@ ExecAssignScanProjectionInfo(ScanState *node)
251251
/* Vars in an index-only scan's tlist should be INDEX_VAR */
252252
if (IsA(scan,IndexOnlyScan))
253253
varno=INDEX_VAR;
254+
/* Also foreign or custom scan on pseudo relation should be INDEX_VAR */
255+
elseif (scan->scanrelid==0)
256+
{
257+
Assert(IsA(scan,ForeignScan)||IsA(scan,CustomScan));
258+
varno=INDEX_VAR;
259+
}
254260
else
255261
varno=scan->scanrelid;
256262

‎src/backend/executor/nodeCustom.c‎

Lines changed: 24 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ CustomScanState *
2323
ExecInitCustomScan(CustomScan*cscan,EState*estate,inteflags)
2424
{
2525
CustomScanState*css;
26-
Relationscan_rel;
26+
Indexscan_relid=cscan->scan.scanrelid;
2727

2828
/* populate a CustomScanState according to the CustomScan */
2929
css= (CustomScanState*)cscan->methods->CreateCustomScanState(cscan);
@@ -48,12 +48,26 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
4848
ExecInitScanTupleSlot(estate,&css->ss);
4949
ExecInitResultTupleSlot(estate,&css->ss.ps);
5050

51-
/* initialize scan relation */
52-
scan_rel=ExecOpenScanRelation(estate,cscan->scan.scanrelid,eflags);
53-
css->ss.ss_currentRelation=scan_rel;
54-
css->ss.ss_currentScanDesc=NULL;/* set by provider */
55-
ExecAssignScanType(&css->ss,RelationGetDescr(scan_rel));
56-
51+
/*
52+
* open the base relation and acquire an appropriate lock on it;
53+
* also, get and assign the scan type
54+
*/
55+
if (scan_relid>0)
56+
{
57+
Relationscan_rel;
58+
59+
scan_rel=ExecOpenScanRelation(estate,scan_relid,eflags);
60+
css->ss.ss_currentRelation=scan_rel;
61+
css->ss.ss_currentScanDesc=NULL;/* set by provider */
62+
ExecAssignScanType(&css->ss,RelationGetDescr(scan_rel));
63+
}
64+
else
65+
{
66+
TupleDescps_tupdesc;
67+
68+
ps_tupdesc=ExecCleanTypeFromTL(cscan->custom_ps_tlist, false);
69+
ExecAssignScanType(&css->ss,ps_tupdesc);
70+
}
5771
css->ss.ps.ps_TupFromTlist= false;
5872

5973
/*
@@ -89,11 +103,11 @@ ExecEndCustomScan(CustomScanState *node)
89103

90104
/* Clean out the tuple table */
91105
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
92-
if (node->ss.ss_ScanTupleSlot)
93-
ExecClearTuple(node->ss.ss_ScanTupleSlot);
106+
ExecClearTuple(node->ss.ss_ScanTupleSlot);
94107

95108
/* Close the heap relation */
96-
ExecCloseScanRelation(node->ss.ss_currentRelation);
109+
if (node->ss.ss_currentRelation)
110+
ExecCloseScanRelation(node->ss.ss_currentRelation);
97111
}
98112

99113
void

‎src/backend/executor/nodeForeignscan.c‎

Lines changed: 20 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ ForeignScanState *
102102
ExecInitForeignScan(ForeignScan*node,EState*estate,inteflags)
103103
{
104104
ForeignScanState*scanstate;
105-
RelationcurrentRelation;
105+
Indexscanrelid=node->scan.scanrelid;
106106
FdwRoutine*fdwroutine;
107107

108108
/* check for unsupported flags */
@@ -141,16 +141,24 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
141141
ExecInitScanTupleSlot(estate,&scanstate->ss);
142142

143143
/*
144-
* open the base relation and acquire appropriate lock on it.
144+
* open the base relation and acquire an appropriate lock on it;
145+
* also, get and assign the scan type
145146
*/
146-
currentRelation=ExecOpenScanRelation(estate,node->scan.scanrelid,eflags);
147-
scanstate->ss.ss_currentRelation=currentRelation;
147+
if (scanrelid>0)
148+
{
149+
RelationcurrentRelation;
148150

149-
/*
150-
* get the scan type from the relation descriptor. (XXX at some point we
151-
* might want to let the FDW editorialize on the scan tupdesc.)
152-
*/
153-
ExecAssignScanType(&scanstate->ss,RelationGetDescr(currentRelation));
151+
currentRelation=ExecOpenScanRelation(estate,scanrelid,eflags);
152+
scanstate->ss.ss_currentRelation=currentRelation;
153+
ExecAssignScanType(&scanstate->ss,RelationGetDescr(currentRelation));
154+
}
155+
else
156+
{
157+
TupleDescps_tupdesc;
158+
159+
ps_tupdesc=ExecCleanTypeFromTL(node->fdw_ps_tlist, false);
160+
ExecAssignScanType(&scanstate->ss,ps_tupdesc);
161+
}
154162

155163
/*
156164
* Initialize result tuple type and projection info.
@@ -161,7 +169,7 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
161169
/*
162170
* Acquire function pointers from the FDW's handler, and init fdw_state.
163171
*/
164-
fdwroutine=GetFdwRoutineForRelation(currentRelation, true);
172+
fdwroutine=GetFdwRoutine(node->fdw_handler);
165173
scanstate->fdwroutine=fdwroutine;
166174
scanstate->fdw_state=NULL;
167175

@@ -193,7 +201,8 @@ ExecEndForeignScan(ForeignScanState *node)
193201
ExecClearTuple(node->ss.ss_ScanTupleSlot);
194202

195203
/* close the relation. */
196-
ExecCloseScanRelation(node->ss.ss_currentRelation);
204+
if (node->ss.ss_currentRelation)
205+
ExecCloseScanRelation(node->ss.ss_currentRelation);
197206
}
198207

199208
/* ----------------------------------------------------------------

‎src/backend/foreign/foreign.c‎

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -304,11 +304,11 @@ GetFdwRoutine(Oid fdwhandler)
304304

305305

306306
/*
307-
*GetFdwRoutineByRelId - look up the handler of the foreign-data wrapper
308-
* for the given foreign table, and retrieve its FdwRoutine struct.
307+
*GetFdwHandlerByRelId - look up the handler of the foreign-data wrapper
308+
* for the given foreign table
309309
*/
310-
FdwRoutine*
311-
GetFdwRoutineByRelId(Oidrelid)
310+
Oid
311+
GetFdwHandlerByRelId(Oidrelid)
312312
{
313313
HeapTupletp;
314314
Form_pg_foreign_data_wrapperfdwform;
@@ -350,7 +350,18 @@ GetFdwRoutineByRelId(Oid relid)
350350

351351
ReleaseSysCache(tp);
352352

353-
/* And finally, call the handler function. */
353+
returnfdwhandler;
354+
}
355+
356+
/*
357+
* GetFdwRoutineByRelId - look up the handler of the foreign-data wrapper
358+
* for the given foreign table, and retrieve its FdwRoutine struct.
359+
*/
360+
FdwRoutine*
361+
GetFdwRoutineByRelId(Oidrelid)
362+
{
363+
Oidfdwhandler=GetFdwHandlerByRelId(relid);
364+
354365
returnGetFdwRoutine(fdwhandler);
355366
}
356367

‎src/backend/nodes/copyfuncs.c‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -592,8 +592,11 @@ _copyForeignScan(const ForeignScan *from)
592592
/*
593593
* copy remainder of node
594594
*/
595+
COPY_SCALAR_FIELD(fdw_handler);
595596
COPY_NODE_FIELD(fdw_exprs);
597+
COPY_NODE_FIELD(fdw_ps_tlist);
596598
COPY_NODE_FIELD(fdw_private);
599+
COPY_BITMAPSET_FIELD(fdw_relids);
597600
COPY_SCALAR_FIELD(fsSystemCol);
598601

599602
returnnewnode;
@@ -617,7 +620,9 @@ _copyCustomScan(const CustomScan *from)
617620
*/
618621
COPY_SCALAR_FIELD(flags);
619622
COPY_NODE_FIELD(custom_exprs);
623+
COPY_NODE_FIELD(custom_ps_tlist);
620624
COPY_NODE_FIELD(custom_private);
625+
COPY_BITMAPSET_FIELD(custom_relids);
621626

622627
/*
623628
* NOTE: The method field of CustomScan is required to be a pointer to a

‎src/backend/nodes/outfuncs.c‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -558,8 +558,11 @@ _outForeignScan(StringInfo str, const ForeignScan *node)
558558

559559
_outScanInfo(str, (constScan*)node);
560560

561+
WRITE_OID_FIELD(fdw_handler);
561562
WRITE_NODE_FIELD(fdw_exprs);
563+
WRITE_NODE_FIELD(fdw_ps_tlist);
562564
WRITE_NODE_FIELD(fdw_private);
565+
WRITE_BITMAPSET_FIELD(fdw_relids);
563566
WRITE_BOOL_FIELD(fsSystemCol);
564567
}
565568

@@ -572,7 +575,9 @@ _outCustomScan(StringInfo str, const CustomScan *node)
572575

573576
WRITE_UINT_FIELD(flags);
574577
WRITE_NODE_FIELD(custom_exprs);
578+
WRITE_NODE_FIELD(custom_ps_tlist);
575579
WRITE_NODE_FIELD(custom_private);
580+
WRITE_BITMAPSET_FIELD(custom_relids);
576581
appendStringInfoString(str," :methods ");
577582
_outToken(str,node->methods->CustomName);
578583
if (node->methods->TextOutCustomScan)

‎src/backend/optimizer/path/joinpath.c‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,13 @@
1717
#include<math.h>
1818

1919
#include"executor/executor.h"
20+
#include"foreign/fdwapi.h"
2021
#include"optimizer/cost.h"
2122
#include"optimizer/pathnode.h"
2223
#include"optimizer/paths.h"
2324

25+
/* Hook for plugins to get control in add_paths_to_joinrel() */
26+
set_join_pathlist_hook_typeset_join_pathlist_hook=NULL;
2427

2528
#definePATH_PARAM_BY_REL(path,rel) \
2629
((path)->param_info && bms_overlap(PATH_REQ_OUTER(path), (rel)->relids))
@@ -260,6 +263,27 @@ add_paths_to_joinrel(PlannerInfo *root,
260263
restrictlist,jointype,
261264
sjinfo,&semifactors,
262265
param_source_rels,extra_lateral_rels);
266+
267+
/*
268+
* 5. If both inner and outer relations are managed by the same FDW,
269+
* give it a chance to push down joins.
270+
*/
271+
if (joinrel->fdwroutine&&
272+
joinrel->fdwroutine->GetForeignJoinPaths)
273+
joinrel->fdwroutine->GetForeignJoinPaths(root,joinrel,
274+
outerrel,innerrel,
275+
restrictlist,jointype,sjinfo,
276+
&semifactors,
277+
param_source_rels,
278+
extra_lateral_rels);
279+
/*
280+
* 6. Finally, give extensions a chance to manipulate the path list.
281+
*/
282+
if (set_join_pathlist_hook)
283+
set_join_pathlist_hook(root,joinrel,outerrel,innerrel,
284+
restrictlist,jointype,
285+
sjinfo,&semifactors,
286+
param_source_rels,extra_lateral_rels);
263287
}
264288

265289
/*

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp