Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Commit885c868

Browse files
committed
pathman: documentation
1 parentf2ae7a4 commit885c868

File tree

4 files changed

+95
-98
lines changed

4 files changed

+95
-98
lines changed

‎contrib/pg_pathman/README.md

Lines changed: 93 additions & 92 deletions
Original file line numberDiff line numberDiff line change
@@ -4,23 +4,17 @@ The `pg_pathman` module provides optimized partitioning mechanism and functions
44

55
##pg_pathman Concepts
66

7-
Partitioning refers to splitting one large table into smaller pieces. Each row in such table assigns to a single partition based on partitioning key. Common partitioning strategies are:
8-
9-
* HASH - maps rows to partitions based on hash function values;
10-
* RANGE - maps data to partitions based on ranges that you establish for each partition;
11-
* LIST - maps data to partitions based on explicitly specified values of partitioning key for each partition.
12-
13-
PostgreSQL supports partitioning via table inheritance. Each partition must be created as child table with CHECK CONSTRAINT. For example:
7+
Partitioning refers to splitting one large table into smaller pieces. Each row in such table assigns to a single partition based on partitioning key. PostgreSQL supports partitioning via table inheritance. Each partition must be created as child table with CHECK CONSTRAINT. For example:
148

159
```
1610
CREATE TABLE test (id SERIAL PRIMARY KEY, title TEXT);
1711
CREATE TABLE test_1 (CHECK ( id >= 100 AND id < 200 )) INHERITS (test);
1812
CREATE TABLE test_2 (CHECK ( id >= 200 AND id < 300 )) INHERITS (test);
1913
```
2014

21-
Despite the flexibilityofthis approachit has weakness. If query uses filteringtheoptimizer forcedto perform an exhaustive search and check constraints for each partition to determinepartitions fromwhichit shouldselect data. If the number of partitions is large the overhead may be significant.
15+
Despite the flexibility this approachforcestheplannerto perform an exhaustive search and check constraints for each partition to determine whichone shouldpresent in the plan. If the number of partitions is large the overhead may be significant.
2216

23-
The`pg_pathman` module provides functions to manage partitions and partitioning mechanism optimized based on knowledge of the partitions structure. It stores partitioning configuration in the`pathman_config` table, each row of which contains single entry for partitioned table (relation name, partitioning key and type). During initialization the`pg_pathman` module caches information about child partitions in shared memory in form convenient to perform rapid search. Whenuser executesSELECT query pg_pathman analyzes conditions tree looking for conditions like:
17+
The`pg_pathman` module provides functions to manage partitions and partitioning mechanism optimized based on knowledge of the partitions structure. It stores partitioning configuration in the`pathman_config` table, each row of which contains single entry for partitioned table (relation name, partitioning key and type). During initialization the`pg_pathman` module caches information about child partitions in shared memory in form convenient to perform rapid search. When SELECT queryexecutes`pg_pathman` analyzes conditions tree looking for conditions like:
2418

2519
```
2620
VARIABLE OP CONST
@@ -31,7 +25,16 @@ where `VARIABLE` is partitioning key, `OP` is comparison operator (supported ope
3125
WHERE id = 150
3226
```
3327

34-
Based on partitioning type and operator the`pg_pathman` searches corresponding partitions and builds the plan.
28+
Based on partitioning type and operator the`pg_pathman` searches corresponding partitions and builds the plan. Current version of`pg_pathman` supports two partitioning types:
29+
30+
* RANGE - maps data to partitions based on ranges of partitioning key. Optimization is achieved by using binary search algorithm.
31+
* HASH - maps rows to partitions based on hash function values (only INTEGER attributes at the moment);
32+
33+
##Roadmap
34+
35+
* Optimize the execution of the NestedLoop join method;
36+
* LIST-partitioning;
37+
* HASH-partitioning for non integer attributes.
3538

3639
##Installation
3740

@@ -155,120 +158,118 @@ Disables `pg_pathman` partitioning mechanism for the specified parent table and
155158
###HASH
156159
Consider an example of HASH partitioning. First create a table with some integer column:
157160
```
158-
CREATE TABLE hash_rel (
159-
id SERIAL PRIMARY KEY,
160-
value INTEGER);
161-
INSERT INTO hash_rel (value) SELECT g FROM generate_series(1, 10000) as g;
161+
CREATE TABLE items (
162+
id SERIAL PRIMARY KEY,
163+
name TEXT,
164+
code BIGINT);
165+
166+
INSERT INTO items (id, name, code)
167+
SELECT g, md5(g::text), random() * 100000
168+
FROM generate_series(1, 100000) as g;
162169
```
163170
If partitions are supposed to have indexes, then they should be created for parent table before partitioning. In this case pg_pathman will automaticaly create indexes for partitions. Then run create_hash_partitions() function with appropriate arguments:
164171
```
165-
SELECT create_hash_partitions('hash_rel', 'value', 100);
172+
SELECT create_hash_partitions('items', 'id', 100);
166173
```
167174
This will create new partitions and move the data from parent to partitions.
168175
Here is an example of the query with filtering by partitioning key and its plan:
169176
```
170-
SELECT * FROM hash_rel WHERE value = 1234;
171-
id | value
172-
------+-------
173-
1234 | 1234
177+
SELECT * FROM items WHERE id = 1234;
178+
id | name | code
179+
------+----------------------------------+------
180+
1234 | 81dc9bdb52d04dc20036dbd8313ed055 | 1855
181+
(1 row)
174182
175-
EXPLAIN SELECT * FROMhash_rel WHEREvalue = 1234;
176-
QUERY PLAN
177-
-----------------------------------------------------------------
178-
Append (cost=0.00..2.00 rows=0 width=0)
179-
->Seq Scan onhash_rel_34 (cost=0.00..2.00 rows=0 width=0)
180-
Filter: (value = 1234)
183+
EXPLAIN SELECT * FROMitems WHEREid = 1234;
184+
QUERY PLAN
185+
------------------------------------------------------------------------------------
186+
Append (cost=0.28..8.29 rows=0 width=0)
187+
->Index Scanusing items_34_pkeyonitems_34 (cost=0.28..8.29 rows=0 width=0)
188+
Index Cond: (id = 1234)
181189
```
182190
Note that pg_pathman exludes parent table from the query plan. To access parent table use ONLY modifier:
183191
```
184-
EXPLAIN SELECT * FROM ONLYhash_rel;
185-
QUERY PLAN
186-
--------------------------------------------------------
187-
Seq Scan onhash_rel (cost=0.00..0.00 rows=1 width=8)
192+
EXPLAIN SELECT * FROM ONLYitems;
193+
QUERY PLAN
194+
------------------------------------------------------
195+
Seq Scan onitems (cost=0.00..0.00 rows=1 width=45)
188196
```
189197
###RANGE
190-
Consider an example of RANGE partitioning. Create a table with numerical or date or timestamp column:
191-
```
192-
CREATE TABLE range_rel (
193-
id SERIAL PRIMARY KEY,
194-
dt TIMESTAMP);
195-
INSERT INTO range_rel (dt) SELECT g FROM generate_series('2010-01-01'::date, '2014-12-31'::date, '1 day') as g;
198+
Consider an example of RANGE partitioning. Let's create a table to store log data:
196199
```
197-
Run create_range_partitions() function to create partitions so that each partition would contain data for one month:
198-
```
199-
SELECT create_range_partitions('range_rel', 'dt', '2010-01-01'::date, '1 month'::interval, 60);
200-
```
201-
It will create 60 partitions and move the data from parent to partitions.
202-
To merge to adjacent partitions run merge_range_partitions() function:
203-
```
204-
SELECT merge_range_partitions('range_rel_1', 'range_rel_2');
200+
CREATE TABLE journal (
201+
id SERIAL,
202+
dt TIMESTAMP NOT NULL,
203+
level INTEGER,
204+
msg TEXT
205+
);
206+
CREATE INDEX ON journal(dt);
207+
208+
INSERT INTO journal (dt, level, msg)
209+
SELECT g, random()*6, md5(g::text)
210+
FROM generate_series('2015-01-01'::date, '2015-12-31'::date, '1 minute') as g;
205211
```
206-
To splitpartitionuse split_range_partition() function:
212+
Run create_range_partitions() function to create partitions so that eachpartitionwould contain data for one day:
207213
```
208-
SELECTsplit_range_partition('range_rel_1', '2010-02-15'::date);
214+
SELECTcreate_range_partitions('journal', 'dt', '2015-01-01'::date, '1 day'::interval);
209215
```
210-
Now let's create new partition. You can use append_partition() or prepend_partition() functions:
216+
It will create 365 partitions and move the data from parent to partitions.
217+
218+
New partitions are appended automaticaly by insert trigger. But it can be done manually with the following functions:
211219
```
212-
SELECT append_partition('range_rel');
220+
SELECT add_range_partition('journal', '2016-01-01'::date, '2016-01-07'::date);
221+
SELECT append_range_partition('journal');
213222
```
214-
Here is an example of the query with filtering by partitioning key and its plan:
223+
The first one creates partition with specified range. The second one creates partition with default interval and appends it to the partition list. It is also possible to attach an existing table as partition. For example we may want to attach an archive table (or even foreign table from another server) for outdated data:
224+
215225
```
216-
SELECT * FROM range_rel WHERE dt >= '2012-04-30' AND dt <= '2012-05-01';
217-
id | dt
218-
-----+---------------------
219-
851 | 2012-04-30 00:00:00
220-
852 | 2012-05-01 00:00:00
226+
CREATE FOREIGN TABLE journal_archive (
227+
id INTEGER NOT NULL,
228+
dt TIMESTAMP NOT NULL,
229+
level INTEGER,
230+
msg TEXT
231+
) SERVER archive_server;
221232
222-
EXPLAIN SELECT * FROM range_rel WHERE dt >= '2012-04-30' AND dt <= '2012-05-01';
223-
QUERY PLAN
224-
----------------------------------------------------------------------------
225-
Append (cost=0.00..60.80 rows=0 width=0)
226-
-> Seq Scan on range_rel_28 (cost=0.00..30.40 rows=0 width=0)
227-
Filter: (dt >= '2012-04-30 00:00:00'::timestamp without time zone)
228-
-> Seq Scan on range_rel_29 (cost=0.00..30.40 rows=0 width=0)
229-
Filter: (dt <= '2012-05-01 00:00:00'::timestamp without time zone)
233+
SELECT attach_range_partition('journal', 'journal_archive', '2014-01-01'::date, '2015-01-01'::date);
230234
```
235+
>Important: the structure of the table being attched must exactly match the parent.
231236
232-
###Disable pg_pathman
233-
To disable pg_pathman for some previously partitioned table use disable_partitioning() function:
237+
To merge to adjacent partitions use function:
234238
```
235-
SELECTdisable_partitioning('range_rel');
239+
SELECTmerge_range_partitions('journal_archive', 'journal_1');
236240
```
237-
All sections and data will stay available and will be handled by standard PostgreSQL partitioning mechanism.
238-
###Manual partitions management
239-
It is possible to manage partitions manually. After creating or removing child tables it's necessary to invoke function:
241+
To split partition by value use function:
240242
```
241-
on_update_partitions(oid),
243+
SELECT split_range_partition('journal_366', '2016-01-03'::date);
242244
```
243-
which updates internal structures in memory of`pg_pathman module`. For example, let's create new section for the`range_rel` from above:
245+
To detach partition use:
244246
```
245-
CREATE TABLE range_rel_archive (CHECK (dt >= '2000-01-01' AND dt < '2010-01-01')) INHERITS (range_rel);
246-
SELECT on_update_partitions('range_rel'::regclass::oid);
247+
SELECT detach_range_partition('journal_archive');
247248
```
248-
CHECK CONSTRAINT must have the exact format:
249-
* (VARIABLE >= CONST AND VARIABLE < CONST) for RANGE partitioned tables;
250-
* (VARIABLE % CONST = CONST) for HASH partitioned tables.
251249

252-
It ispossible to create partition from foreign table as well:
250+
Here isan example of the query with filtering by partitioning key and its plan:
253251
```
254-
CREATE FOREIGN TABLE range_rel_archive (
255-
id INTEGER NOT NULL,
256-
dt TIMESTAMP)
257-
SERVER archive_server;
258-
ALTER TABLE range_rel_archive INHERIT range_rel;
259-
ALTER TABLE range_rel_archive ADD CHECK (dt >= '2000-01-01' AND dt < '2010-01-01');
260-
SELECT on_update_partitions('range_rel'::regclass::oid);
252+
SELECT * FROM journal WHERE dt >= '2015-06-01' AND dt < '2015-06-03';
253+
id | dt | level | msg
254+
--------+---------------------+-------+----------------------------------
255+
217441 | 2015-06-01 00:00:00 | 2 | 15053892d993ce19f580a128f87e3dbf
256+
217442 | 2015-06-01 00:01:00 | 1 | 3a7c46f18a952d62ce5418ac2056010c
257+
217443 | 2015-06-01 00:02:00 | 0 | 92c8de8f82faf0b139a3d99f2792311d
258+
...
259+
(2880 rows)
260+
261+
EXPLAIN SELECT * FROM journal WHERE dt >= '2015-06-01' AND dt < '2015-06-03';
262+
QUERY PLAN
263+
------------------------------------------------------------------
264+
Append (cost=0.00..58.80 rows=0 width=0)
265+
-> Seq Scan on journal_152 (cost=0.00..29.40 rows=0 width=0)
266+
-> Seq Scan on journal_153 (cost=0.00..29.40 rows=0 width=0)
267+
(3 rows)
261268
```
262-
Foreign table structure must exactly match the parent table.
263269

264-
In case when parent table is being dropped by DROP TABLE, you should invoke on_remove_partitions() function and delete particular entry from`pathman_config` table:
270+
###Disable pg_pathman
271+
To disable pg_pathman for some previously partitioned table use disable_partitioning() function:
265272
```
266-
SELECT on_remove_partitions('range_rel'::regclass::oid);
267-
DROP TABLE range_rel CASCADE;
268-
DELETE FROM pathman_config WHERE relname = 'public.range_rel';
273+
SELECT disable_partitioning('range_rel');
269274
```
270-
271-
##Author
272-
Ildar Musin <i.musin@postgrespro.ru> Postgres Professional Ltd., Russia
273-
274-
This module is sponsored by Postgres Professional Ltd., Russia
275+
All sections and data will stay available and will be handled by standard PostgreSQL partitioning mechanism.

‎contrib/pg_pathman/README.rus.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -231,6 +231,7 @@ CREATE FOREIGN TABLE journal_archive (
231231
) SERVER archive_server;
232232
```
233233
>Важно: структура подключаемой таблицы должна полностью совпадать с родительской.
234+
234235
Подключим ее к имеющемуся разбиению:
235236
```
236237
SELECT attach_range_partition('journal', 'journal_archive', '2014-01-01'::date, '2015-01-01'::date);
@@ -243,7 +244,7 @@ SELECT merge_range_partitions('journal_archive', 'journal_1');
243244
```
244245
SELECT split_range_partition('journal_366', '2016-01-03'::date);
245246
```
246-
Чтобы отсоединить ранее созданнуюили присоединеннуюсекцию воспользуйтесь функцией:
247+
Чтобы отсоединить ранее созданную секцию, воспользуйтесь функцией:
247248
```
248249
SELECT detach_range_partition('journal_archive');
249250
```

‎contrib/pg_pathman/pathman.h

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -82,8 +82,6 @@ typedef struct HashRelation
8282
typedefstructRangeEntry
8383
{
8484
Oidchild_oid;
85-
// Datummin;
86-
// Datummax;
8785
#ifdefHAVE_INT64_TIMESTAMP
8886
int64min;
8987
int64max;

‎contrib/pg_pathman/worker.c

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -163,9 +163,6 @@ create_partitions(Oid relid, Datum value, Oid value_type)
163163
FmgrInfocmp_func;
164164
char*schema;
165165

166-
// elog(WARNING, "WORKER PID >>>%d<<<", MyProcPid);
167-
// sleep(10);
168-
169166
schema=get_extension_schema();
170167

171168
prel=get_pathman_relation_info(relid,NULL);

0 commit comments

Comments
 (0)

[8]ページ先頭

©2009-2025 Movatter.jp