| 54.2. Foreign Data Wrapper Callback Routines | ||||
|---|---|---|---|---|
| Prev | Up | Chapter 54. Writing A Foreign Data Wrapper | Home | Next |
54.2. Foreign Data Wrapper Callback Routines
The FDW handler function returns a palloc'dFdwRoutine struct containing pointers to the callback functions described below. The scan-related functions are required, the rest are optional.
TheFdwRoutine struct type is declared insrc/include/foreign/fdwapi.h, which see for additional details.
54.2.1. FDW Routines For Scanning Foreign Tables
voidGetForeignRelSize (PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid);
Obtain relation size estimates for a foreign table. This is called at the beginning of planning for a query that scans a foreign table.root is the planner's global information about the query;baserel is the planner's information about this table; andforeigntableid is thepg_class OID of the foreign table. (foreigntableid could be obtained from the planner data structures, but it's passed explicitly to save effort.)
This function should updatebaserel->rows to be the expected number of rows returned by the table scan, after accounting for the filtering done by the restriction quals. The initial value ofbaserel->rows is just a constant default estimate, which should be replaced if at all possible. The function may also choose to updatebaserel->width if it can compute a better estimate of the average result row width.
SeeSection 54.4 for additional information.
voidGetForeignPaths (PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid);
Create possible access paths for a scan on a foreign table. This is called during query planning. The parameters are the same as forGetForeignRelSize, which has already been called.
This function must generate at least one access path (ForeignPath node) for a scan on the foreign table and must calladd_path to add each such path tobaserel->pathlist. It's recommended to usecreate_foreignscan_path to build theForeignPath nodes. The function can generate multiple access paths, e.g., a path which has validpathkeys to represent a pre-sorted result. Each access path must contain cost estimates, and can contain any FDW-private information that is needed to identify the specific scan method intended.
SeeSection 54.4 for additional information.
ForeignScan *GetForeignPlan (PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid, ForeignPath *best_path, List *tlist, List *scan_clauses, Plan *outer_plan);
Create aForeignScan plan node from the selected foreign access path. This is called at the end of query planning. The parameters are as forGetForeignRelSize, plus the selectedForeignPath (previously produced byGetForeignPaths orGetForeignJoinPaths), the target list to be emitted by the plan node, the restriction clauses to be enforced by the plan node, and the outer subplan of theForeignScan, which is used for rechecks performed byRecheckForeignScan. (If the path is for a join rather than a base relation,foreigntableid isInvalidOid.)
This function must create and return aForeignScan plan node; it's recommended to usemake_foreignscan to build theForeignScan node.
SeeSection 54.4 for additional information.
voidBeginForeignScan (ForeignScanState *node, int eflags);
Begin executing a foreign scan. This is called during executor startup. It should perform any initialization needed before the scan can start, but not start executing the actual scan (that should be done upon the first call toIterateForeignScan). TheForeignScanState node has already been created, but itsfdw_state field is still NULL. Information about the table to scan is accessible through theForeignScanState node (in particular, from the underlyingForeignScan plan node, which contains any FDW-private information provided byGetForeignPlan).eflags contains flag bits describing the executor's operating mode for this plan node.
Note that when(eflags & EXEC_FLAG_EXPLAIN_ONLY) is true, this function should not perform any externally-visible actions; it should only do the minimum required to make the node state valid forExplainForeignScan andEndForeignScan.
TupleTableSlot *IterateForeignScan (ForeignScanState *node);
Fetch one row from the foreign source, returning it in a tuple table slot (the node'sScanTupleSlot should be used for this purpose). Return NULL if no more rows are available. The tuple table slot infrastructure allows either a physical or virtual tuple to be returned; in most cases the latter choice is preferable from a performance standpoint. Note that this is called in a short-lived memory context that will be reset between invocations. Create a memory context inBeginForeignScan if you need longer-lived storage, or use thees_query_cxt of the node'sEState.
The rows returned must match thefdw_scan_tlist target list if one was supplied, otherwise they must match the row type of the foreign table being scanned. If you choose to optimize away fetching columns that are not needed, you should insert nulls in those column positions, or else generate afdw_scan_tlist list with those columns omitted.
Note thatPostgres Pro's executor doesn't care whether the rows returned violate any constraints that were defined on the foreign table — but the planner does care, and may optimize queries incorrectly if there are rows visible in the foreign table that do not satisfy a declared constraint. If a constraint is violated when the user has declared that the constraint should hold true, it may be appropriate to raise an error (just as you would need to do in the case of a data type mismatch).
voidReScanForeignScan (ForeignScanState *node);
Restart the scan from the beginning. Note that any parameters the scan depends on may have changed value, so the new scan does not necessarily return exactly the same rows.
voidEndForeignScan (ForeignScanState *node);
End the scan and release resources. It is normally not important to release palloc'd memory, but for example open files and connections to remote servers should be cleaned up.
54.2.2. FDW Routines For Scanning Foreign Joins
If an FDW supports performing foreign joins remotely (rather than by fetching both tables' data and doing the join locally), it should provide this callback function:
voidGetForeignJoinPaths (PlannerInfo *root, RelOptInfo *joinrel, RelOptInfo *outerrel, RelOptInfo *innerrel, JoinType jointype, JoinPathExtraData *extra);
Create possible access paths for a join of two (or more) foreign tables that all belong to the same foreign server. This optional function is called during query planning. As withGetForeignPaths, this function should generateForeignPath path(s) for the suppliedjoinrel, and calladd_path to add these paths to the set of paths considered for the join. But unlikeGetForeignPaths, it is not necessary that this function succeed in creating at least one path, since paths involving local joining are always possible.
Note that this function will be invoked repeatedly for the same join relation, with different combinations of inner and outer relations; it is the responsibility of the FDW to minimize duplicated work.
If aForeignPath path is chosen for the join, it will represent the entire join process; paths generated for the component tables and subsidiary joins will not be used. Subsequent processing of the join path proceeds much as it does for a path scanning a single foreign table. One difference is that thescanrelid of the resultingForeignScan plan node should be set to zero, since there is no single relation that it represents; instead, thefs_relids field of theForeignScan node represents the set of relations that were joined. (The latter field is set up automatically by the core planner code, and need not be filled by the FDW.) Another difference is that, because the column list for a remote join cannot be found from the system catalogs, the FDW must fillfdw_scan_tlist with an appropriate list ofTargetEntry nodes, representing the set of columns it will supply at run time in the tuples it returns.
SeeSection 54.4 for additional information.
54.2.3. FDW Routines For Updating Foreign Tables
If an FDW supports writable foreign tables, it should provide some or all of the following callback functions depending on the needs and capabilities of the FDW:
voidAddForeignUpdateTargets (Query *parsetree, RangeTblEntry *target_rte, Relation target_relation);
UPDATE andDELETE operations are performed against rows previously fetched by the table-scanning functions. The FDW may need extra information, such as a row ID or the values of primary-key columns, to ensure that it can identify the exact row to update or delete. To support that, this function can add extra hidden, or“junk”, target columns to the list of columns that are to be retrieved from the foreign table during anUPDATE orDELETE.
To do that, addTargetEntry items toparsetree->targetList, containing expressions for the extra values to be fetched. Each such entry must be markedresjunk =true, and must have a distinctresname that will identify it at execution time. Avoid using names matchingctid,Nwholerow, orwholerow, as the core system can generate junk columns of these names. If the extra expressions are more complex than simple Vars, they must be run throughNeval_const_expressions before adding them to the targetlist.
Although this function is called during planning, the information provided is a bit different from that available to other planning routines.parsetree is the parse tree for theUPDATE orDELETE command, whiletarget_rte andtarget_relation describe the target foreign table.
If theAddForeignUpdateTargets pointer is set toNULL, no extra target expressions are added. (This will make it impossible to implementDELETE operations, thoughUPDATE may still be feasible if the FDW relies on an unchanging primary key to identify rows.)
List *PlanForeignModify (PlannerInfo *root, ModifyTable *plan, Index resultRelation, int subplan_index);
Perform any additional planning actions needed for an insert, update, or delete on a foreign table. This function generates the FDW-private information that will be attached to theModifyTable plan node that performs the update action. This private information must have the form of aList, and will be delivered toBeginForeignModify during the execution stage.
root is the planner's global information about the query.plan is theModifyTable plan node, which is complete except for thefdwPrivLists field.resultRelation identifies the target foreign table by its range table index.subplan_index identifies which target of theModifyTable plan node this is, counting from zero; use this if you want to index intoplan->plans or other substructure of theplan node.
SeeSection 54.4 for additional information.
If thePlanForeignModify pointer is set toNULL, no additional plan-time actions are taken, and thefdw_private list delivered toBeginForeignModify will be NIL.
voidBeginForeignModify (ModifyTableState *mtstate, ResultRelInfo *rinfo, List *fdw_private, int subplan_index, int eflags);
Begin executing a foreign table modification operation. This routine is called during executor startup. It should perform any initialization needed prior to the actual table modifications. Subsequently,ExecForeignInsert,ExecForeignUpdate orExecForeignDelete will be called for each tuple to be inserted, updated, or deleted.
mtstate is the overall state of theModifyTable plan node being executed; global data about the plan and execution state is available via this structure.rinfo is theResultRelInfo struct describing the target foreign table. (Theri_FdwState field ofResultRelInfo is available for the FDW to store any private state it needs for this operation.)fdw_private contains the private data generated byPlanForeignModify, if any.subplan_index identifies which target of theModifyTable plan node this is.eflags contains flag bits describing the executor's operating mode for this plan node.
Note that when(eflags & EXEC_FLAG_EXPLAIN_ONLY) is true, this function should not perform any externally-visible actions; it should only do the minimum required to make the node state valid forExplainForeignModify andEndForeignModify.
If theBeginForeignModify pointer is set toNULL, no action is taken during executor startup.
TupleTableSlot *ExecForeignInsert (EState *estate, ResultRelInfo *rinfo, TupleTableSlot *slot, TupleTableSlot *planSlot);
Insert one tuple into the foreign table.estate is global execution state for the query.rinfo is theResultRelInfo struct describing the target foreign table.slot contains the tuple to be inserted; it will match the row-type definition of the foreign table.planSlot contains the tuple that was generated by theModifyTable plan node's subplan; it differs fromslot in possibly containing additional“junk” columns. (TheplanSlot is typically of little interest forINSERT cases, but is provided for completeness.)
The return value is either a slot containing the data that was actually inserted (this might differ from the data supplied, for example as a result of trigger actions), or NULL if no row was actually inserted (again, typically as a result of triggers). The passed-inslot can be re-used for this purpose.
The data in the returned slot is used only if theINSERT query has aRETURNING clause or the foreign table has anAFTER ROW trigger. Triggers require all columns, but the FDW could choose to optimize away returning some or all columns depending on the contents of theRETURNING clause. Regardless, some slot must be returned to indicate success, or the query's reported row count will be wrong.
If theExecForeignInsert pointer is set toNULL, attempts to insert into the foreign table will fail with an error message.
TupleTableSlot *ExecForeignUpdate (EState *estate, ResultRelInfo *rinfo, TupleTableSlot *slot, TupleTableSlot *planSlot);
Update one tuple in the foreign table.estate is global execution state for the query.rinfo is theResultRelInfo struct describing the target foreign table.slot contains the new data for the tuple; it will match the row-type definition of the foreign table.planSlot contains the tuple that was generated by theModifyTable plan node's subplan; it differs fromslot in possibly containing additional“junk” columns. In particular, any junk columns that were requested byAddForeignUpdateTargets will be available from this slot.
The return value is either a slot containing the row as it was actually updated (this might differ from the data supplied, for example as a result of trigger actions), or NULL if no row was actually updated (again, typically as a result of triggers). The passed-inslot can be re-used for this purpose.
The data in the returned slot is used only if theUPDATE query has aRETURNING clause or the foreign table has anAFTER ROW trigger. Triggers require all columns, but the FDW could choose to optimize away returning some or all columns depending on the contents of theRETURNING clause. Regardless, some slot must be returned to indicate success, or the query's reported row count will be wrong.
If theExecForeignUpdate pointer is set toNULL, attempts to update the foreign table will fail with an error message.
TupleTableSlot *ExecForeignDelete (EState *estate, ResultRelInfo *rinfo, TupleTableSlot *slot, TupleTableSlot *planSlot);
Delete one tuple from the foreign table.estate is global execution state for the query.rinfo is theResultRelInfo struct describing the target foreign table.slot contains nothing useful upon call, but can be used to hold the returned tuple.planSlot contains the tuple that was generated by theModifyTable plan node's subplan; in particular, it will carry any junk columns that were requested byAddForeignUpdateTargets. The junk column(s) must be used to identify the tuple to be deleted.
The return value is either a slot containing the row that was deleted, or NULL if no row was deleted (typically as a result of triggers). The passed-inslot can be used to hold the tuple to be returned.
The data in the returned slot is used only if theDELETE query has aRETURNING clause or the foreign table has anAFTER ROW trigger. Triggers require all columns, but the FDW could choose to optimize away returning some or all columns depending on the contents of theRETURNING clause. Regardless, some slot must be returned to indicate success, or the query's reported row count will be wrong.
If theExecForeignDelete pointer is set toNULL, attempts to delete from the foreign table will fail with an error message.
voidEndForeignModify (EState *estate, ResultRelInfo *rinfo);
End the table update and release resources. It is normally not important to release palloc'd memory, but for example open files and connections to remote servers should be cleaned up.
If theEndForeignModify pointer is set toNULL, no action is taken during executor shutdown.
intIsForeignRelUpdatable (Relation rel);
Report which update operations the specified foreign table supports. The return value should be a bit mask of rule event numbers indicating which operations are supported by the foreign table, using theCmdType enumeration; that is,(1 << CMD_UPDATE) = 4 forUPDATE,(1 << CMD_INSERT) = 8 forINSERT, and(1 << CMD_DELETE) = 16 forDELETE.
If theIsForeignRelUpdatable pointer is set toNULL, foreign tables are assumed to be insertable, updatable, or deletable if the FDW providesExecForeignInsert,ExecForeignUpdate, orExecForeignDelete respectively. This function is only needed if the FDW supports some tables that are updatable and some that are not. (Even then, it's permissible to throw an error in the execution routine instead of checking in this function. However, this function is used to determine updatability for display in theinformation_schema views.)
54.2.4. FDW Routines For Row Locking
If an FDW wishes to supportlate row locking (as described inSection 54.5), it must provide the following callback functions:
RowMarkTypeGetForeignRowMarkType (RangeTblEntry *rte, LockClauseStrength strength);
Report which row-marking option to use for a foreign table.rte is theRangeTblEntry node for the table andstrength describes the lock strength requested by the relevantFOR UPDATE/SHARE clause, if any. The result must be a member of theRowMarkType enum type.
This function is called during query planning for each foreign table that appears in anUPDATE,DELETE, orSELECT FOR UPDATE/SHARE query and is not the target ofUPDATE orDELETE.
If theGetForeignRowMarkType pointer is set toNULL, theROW_MARK_COPY option is always used. (This implies thatRefetchForeignRow will never be called, so it need not be provided either.)
SeeSection 54.5 for more information.
HeapTupleRefetchForeignRow (EState *estate, ExecRowMark *erm, Datum rowid, bool *updated);
Re-fetch one tuple from the foreign table, after locking it if required.estate is global execution state for the query.erm is theExecRowMark struct describing the target foreign table and the row lock type (if any) to acquire.rowid identifies the tuple to be fetched.updated is an output parameter.
This function should return a palloc'ed copy of the fetched tuple, orNULL if the row lock couldn't be obtained. The row lock type to acquire is defined byerm->markType, which is the value previously returned byGetForeignRowMarkType. (ROW_MARK_REFERENCE means to just re-fetch the tuple without acquiring any lock, andROW_MARK_COPY will never be seen by this routine.)
In addition,*updated should be set totrue if what was fetched was an updated version of the tuple rather than the same version previously obtained. (If the FDW cannot be sure about this, always returningtrue is recommended.)
Note that by default, failure to acquire a row lock should result in raising an error; aNULL return is only appropriate if theSKIP LOCKED option is specified byerm->waitPolicy.
Therowid is thectid value previously read for the row to be re-fetched. Although therowid value is passed as aDatum, it can currently only be atid. The function API is chosen in hopes that it may be possible to allow other data types for row IDs in future.
If theRefetchForeignRow pointer is set toNULL, attempts to re-fetch rows will fail with an error message.
SeeSection 54.5 for more information.
boolRecheckForeignScan (ForeignScanState *node, TupleTableSlot *slot);
Recheck that a previously-returned tuple still matches the relevant scan and join qualifiers, and possibly provide a modified version of the tuple. For foreign data wrappers which do not perform join pushdown, it will typically be more convenient to set this toNULL and instead setfdw_recheck_quals appropriately. When outer joins are pushed down, however, it isn't sufficient to reapply the checks relevant to all the base tables to the result tuple, even if all needed attributes are present, because failure to match some qualifier might result in some attributes going to NULL, rather than in no tuple being returned.RecheckForeignScan can recheck qualifiers and return true if they are still satisfied and false otherwise, but it can also store a replacement tuple into the supplied slot.
To implement join pushdown, a foreign data wrapper will typically construct an alternative local join plan which is used only for rechecks; this will become the outer subplan of theForeignScan. When a recheck is required, this subplan can be executed and the resulting tuple can be stored in the slot. This plan need not be efficient since no base table will return more than one row; for example, it may implement all joins as nested loops.
54.2.5. FDW Routines forEXPLAIN
voidExplainForeignScan (ForeignScanState *node, ExplainState *es);
Print additionalEXPLAIN output for a foreign table scan. This function can callExplainPropertyText and related functions to add fields to theEXPLAIN output. The flag fields ines can be used to determine what to print, and the state of theForeignScanState node can be inspected to provide run-time statistics in theEXPLAIN ANALYZE case.
If theExplainForeignScan pointer is set toNULL, no additional information is printed duringEXPLAIN.
voidExplainForeignModify (ModifyTableState *mtstate, ResultRelInfo *rinfo, List *fdw_private, int subplan_index, struct ExplainState *es);
Print additionalEXPLAIN output for a foreign table update. This function can callExplainPropertyText and related functions to add fields to theEXPLAIN output. The flag fields ines can be used to determine what to print, and the state of theModifyTableState node can be inspected to provide run-time statistics in theEXPLAIN ANALYZE case. The first four arguments are the same as forBeginForeignModify.
If theExplainForeignModify pointer is set toNULL, no additional information is printed duringEXPLAIN.
54.2.6. FDW Routines forANALYZE
boolAnalyzeForeignTable (Relation relation, AcquireSampleRowsFunc *func, BlockNumber *totalpages);
This function is called whenANALYZE is executed on a foreign table. If the FDW can collect statistics for this foreign table, it should returntrue, and provide a pointer to a function that will collect sample rows from the table infunc, plus the estimated size of the table in pages intotalpages. Otherwise, returnfalse.
If the FDW does not support collecting statistics for any tables, theAnalyzeForeignTable pointer can be set toNULL.
If provided, the sample collection function must have the signature
intAcquireSampleRowsFunc (Relation relation, int elevel, HeapTuple *rows, int targrows, double *totalrows, double *totaldeadrows);
A random sample of up totargrows rows should be collected from the table and stored into the caller-providedrows array. The actual number of rows collected must be returned. In addition, store estimates of the total numbers of live and dead rows in the table into the output parameterstotalrows andtotaldeadrows. (Settotaldeadrows to zero if the FDW does not have any concept of dead rows.)
54.2.7. FDW Routines ForIMPORT FOREIGN SCHEMA
List *ImportForeignSchema (ImportForeignSchemaStmt *stmt, Oid serverOid);
Obtain a list of foreign table creation commands. This function is called when executingIMPORT FOREIGN SCHEMA, and is passed the parse tree for that statement, as well as the OID of the foreign server to use. It should return a list of C strings, each of which must contain aCREATE FOREIGN TABLE command. These strings will be parsed and executed by the core server.
Within theImportForeignSchemaStmt struct,remote_schema is the name of the remote schema from which tables are to be imported.list_type identifies how to filter table names:FDW_IMPORT_SCHEMA_ALL means that all tables in the remote schema should be imported (in this casetable_list is empty),FDW_IMPORT_SCHEMA_LIMIT_TO means to include only tables listed intable_list, andFDW_IMPORT_SCHEMA_EXCEPT means to exclude the tables listed intable_list.options is a list of options used for the import process. The meanings of the options are up to the FDW. For example, an FDW could use an option to define whether theNOT NULL attributes of columns should be imported. These options need not have anything to do with those supported by the FDW as database object options.
The FDW may ignore thelocal_schema field of theImportForeignSchemaStmt, because the core server will automatically insert that name into the parsedCREATE FOREIGN TABLE commands.
The FDW does not have to concern itself with implementing the filtering specified bylist_type andtable_list, either, as the core server will automatically skip any returned commands for tables excluded according to those options. However, it's often useful to avoid the work of creating commands for excluded tables in the first place. The functionIsImportableForeignTable() may be useful to test whether a given foreign-table name will pass the filter.
If the FDW does not support importing table definitions, theImportForeignSchema pointer can be set toNULL.