Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Commit1cd9845

Browse files
committed
Merge commit '200a391b38de54de9180b2d6432ffc98e9988919' into PGPROEE9_6
Bring in pathman 1.4.2Conflicts:contrib/pg_pathman/src/utility_stmt_hooking.c
2 parents5cd2188 +200a391 commit1cd9845

File tree

119 files changed

+18715
-6979
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

119 files changed

+18715
-6979
lines changed

‎contrib/pg_pathman/.editorconfig

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[*]
2+
indent_style =tab
3+
indent_size =4

‎contrib/pg_pathman/.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,5 @@ regression.out
1010
*.gcno
1111
*.gcov
1212
pg_pathman--*.sql
13+
tags
14+
cscope*

‎contrib/pg_pathman/.travis.yml

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,21 @@ compiler:
1111
-gcc
1212

1313
before_install:
14-
-sudo sh ./travis/apt.postgresql.org.sh
14+
-if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then sudo apt-get -y install -qq wget ca-certificates; fi
15+
-if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then source ./travis/dep-ubuntu-postgres.sh; fi
16+
-if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then source ./travis/dep-ubuntu-llvm.sh; fi
17+
-if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then sudo apt-get update -qq; fi
1518

1619
env:
17-
-PGVERSION=9.6 CHECK_CODE=true
18-
-PGVERSION=9.6 CHECK_CODE=false
19-
-PGVERSION=9.5 CHECK_CODE=true
20-
-PGVERSION=9.5 CHECK_CODE=false
20+
global:
21+
-LLVM_VER=4.0
22+
matrix:
23+
-PG_VER=10 CHECK_CODE=true
24+
-PG_VER=10 CHECK_CODE=false
25+
-PG_VER=9.6 CHECK_CODE=true
26+
-PG_VER=9.6 CHECK_CODE=false
27+
-PG_VER=9.5 CHECK_CODE=true
28+
-PG_VER=9.5 CHECK_CODE=false
2129

2230
script:bash ./travis/pg-travis-test.sh
2331

‎contrib/pg_pathman/META.json

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"name":"pg_pathman",
33
"abstract":"Partitioning tool",
44
"description":"The `pg_pathman` module provides optimized partitioning mechanism and functions to manage partitions.",
5-
"version":"1.3.1",
5+
"version":"1.4.2",
66
"maintainer": [
77
"Ildar Musin <i.musin@postgrespro.ru>",
88
"Dmitry Ivanov <d.ivanov@postgrespro.ru>",
@@ -22,9 +22,9 @@
2222
"generated_by":"Ildar Musin",
2323
"provides": {
2424
"pg_pathman": {
25-
"file":"pg_pathman--1.3.sql",
25+
"file":"pg_pathman--1.4.sql",
2626
"docfile":"README.md",
27-
"version":"1.3.1",
27+
"version":"1.4.2",
2828
"abstract":"Partitioning tool"
2929
}
3030
},

‎contrib/pg_pathman/Makefile

Lines changed: 32 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -6,36 +6,48 @@ OBJS = src/init.o src/relation_info.o src/utils.o src/partition_filter.o \
66
src/runtimeappend.o src/runtime_merge_append.o src/pg_pathman.o src/rangeset.o\
77
src/pl_funcs.o src/pl_range_funcs.o src/pl_hash_funcs.o src/pathman_workers.o\
88
src/hooks.o src/nodes_common.o src/xact_handling.o src/utility_stmt_hooking.o\
9-
src/planner_tree_modification.o src/debug_print.o src/pg_compat.o\
10-
src/partition_creation.o$(WIN32RES)
9+
src/planner_tree_modification.o src/debug_print.o src/partition_creation.o\
10+
src/compat/pg_compat.o src/compat/relation_tags.o src/compat/rowmarks_fix.o\
11+
$(WIN32RES)
12+
13+
overridePG_CPPFLAGS += -I$(CURDIR)/src/include
1114

1215
EXTENSION = pg_pathman
1316

14-
EXTVERSION = 1.3
17+
EXTVERSION = 1.4
1518

1619
DATA_built = pg_pathman--$(EXTVERSION).sql
1720

1821
DATA = pg_pathman--1.0--1.1.sql\
1922
pg_pathman--1.1--1.2.sql\
20-
pg_pathman--1.2--1.3.sql
23+
pg_pathman--1.2--1.3.sql\
24+
pg_pathman--1.3--1.4.sql
2125

22-
PGFILEDESC = "pg_pathman - partitioning tool"
26+
PGFILEDESC = "pg_pathman - partitioning tool for PostgreSQL"
2327

24-
REGRESS =pathman_basic\
25-
pathman_cte\
28+
REGRESS =pathman_array_qual\
29+
pathman_basic\
2630
pathman_bgw\
27-
pathman_inserts\
28-
pathman_updates\
29-
pathman_domains\
30-
pathman_interval\
31+
pathman_calamity\
3132
pathman_callbacks\
33+
pathman_column_type\
34+
pathman_cte\
35+
pathman_domains\
36+
pathman_expressions\
3237
pathman_foreign_keys\
38+
pathman_inserts\
39+
pathman_interval\
40+
pathman_join_clause\
41+
pathman_lateral\
42+
pathman_mergejoin\
43+
pathman_only\
44+
pathman_param_upd_del\
3345
pathman_permissions\
46+
pathman_rebuild_updates\
3447
pathman_rowmarks\
3548
pathman_runtime_nodes\
36-
pathman_utility_stmt_hooking\
37-
pathman_calamity\
38-
pathman_join_clause
49+
pathman_update_trigger\
50+
pathman_utility_stmt
3951

4052
EXTRA_REGRESS_OPTS=--temp-config=$(top_srcdir)/$(subdir)/conf.add
4153

@@ -66,3 +78,9 @@ isolationcheck: | submake-isolation
6678
--temp-config=$(top_srcdir)/$(subdir)/conf.add\
6779
--outputdir=./isolation_output\
6880
$(ISOLATIONCHECKS)
81+
82+
python_tests:
83+
$(MAKE) -C tests/python partitioning_tests
84+
85+
cmocka_tests:
86+
$(MAKE) -C tests/cmocka check

‎contrib/pg_pathman/README.md

Lines changed: 82 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,12 @@
77

88
The`pg_pathman` module provides optimized partitioning mechanism and functions to manage partitions.
99

10-
The extension is compatible with PostgreSQL 9.5+.
10+
The extension is compatible with:
11+
* PostgreSQL 9.5, 9.6, 10;
12+
* Postgres Pro Standard 9.5, 9.6;
13+
* Postgres Pro Enterprise;
14+
15+
By the way, we have a growing Wiki[out there](https://github.com/postgrespro/pg_pathman/wiki).
1116

1217
##Overview
1318
**Partitioning** means splitting one large table into smaller pieces. Each row in such table is moved to a single partition according to the partitioning key. PostgreSQL supports partitioning via table inheritance: each partition must be created as a child table with CHECK CONSTRAINT. For example:
@@ -41,6 +46,7 @@ More interesting features are yet to come. Stay tuned!
4146
##Feature highlights
4247

4348
* HASH and RANGE partitioning schemes;
49+
* Partitioning by expression and composite key;
4450
* Both automatic and manual partition management;
4551
* Support for integer, floating point, date and other types, including domains;
4652
* Effective query planning for partitioned tables (JOINs, subselects etc);
@@ -55,9 +61,11 @@ More interesting features are yet to come. Stay tuned!
5561
* Various GUC toggles and configurable settings.
5662

5763
##Roadmap
64+
65+
* Multi-level partitioning (ver 1.5);
66+
* Improved referential integrity + foreign keys on partitioned tables (ver 1.5);
5867

59-
* Implement LIST partitioning scheme;
60-
* Optimize hash join (both tables are partitioned by join key).
68+
Take a look at[this page](https://github.com/postgrespro/pg_pathman/wiki/Roadmap);
6169

6270
##Installation guide
6371
To install`pg_pathman`, execute this in the module's directory:
@@ -97,47 +105,49 @@ SET pg_pathman.enable = t;
97105
###Partition creation
98106
```plpgsql
99107
create_hash_partitions(relation REGCLASS,
100-
attributeTEXT,
108+
exprTEXT,
101109
partitions_countINTEGER,
102110
partition_dataBOOLEAN DEFAULT TRUE,
103111
partition_namesTEXT[] DEFAULTNULL,
104112
tablespacesTEXT[] DEFAULTNULL)
105113
```
106-
Performs HASH partitioning for`relation` byinteger key`attribute`. The`partitions_count` parameter specifies the number of partitions to create; it cannot be changed afterwards. If`partition_data` is`true` then all the data will be automatically copied from the parent table to partitions. Note that data migration may took a while to finish and the table will be locked until transaction commits. See`partition_table_concurrently()` for a lock-free way to migrate data. Partition creation callback is invoked for each partition if set beforehand (see`set_init_callback()`).
114+
Performs HASH partitioning for`relation` bypartitioning expression`expr`. The`partitions_count` parameter specifies the number of partitions to create; it cannot be changed afterwards. If`partition_data` is`true` then all the data will be automatically copied from the parent table to partitions. Note that data migration may took a while to finish and the table will be locked until transaction commits. See`partition_table_concurrently()` for a lock-free way to migrate data. Partition creation callback is invoked for each partition if set beforehand (see`set_init_callback()`).
107115

108116
```plpgsql
109-
create_range_partitions(relation REGCLASS,
110-
attributeTEXT,
111-
start_value ANYELEMENT,
112-
p_interval ANYELEMENT,
113-
p_countINTEGER DEFAULTNULL
114-
partition_dataBOOLEAN DEFAULT TRUE)
117+
create_range_partitions(relation REGCLASS,
118+
expressionTEXT,
119+
start_value ANYELEMENT,
120+
p_interval ANYELEMENT,
121+
p_countINTEGER DEFAULTNULL
122+
partition_dataBOOLEAN DEFAULT TRUE)
123+
124+
create_range_partitions(relation REGCLASS,
125+
expressionTEXT,
126+
start_value ANYELEMENT,
127+
p_interval INTERVAL,
128+
p_countINTEGER DEFAULTNULL,
129+
partition_dataBOOLEAN DEFAULT TRUE)
115130

116-
create_range_partitions(relation REGCLASS,
117-
attributeTEXT,
118-
start_valueANYELEMENT,
119-
p_interval INTERVAL,
120-
p_countINTEGER DEFAULTNULL,
121-
partition_dataBOOLEAN DEFAULT TRUE)
131+
create_range_partitions(relationREGCLASS,
132+
expressionTEXT,
133+
bounds ANYARRAY,
134+
partition_namesTEXT[] DEFAULTNULL,
135+
tablespacesTEXT[] DEFAULTNULL,
136+
partition_dataBOOLEAN DEFAULT TRUE)
122137
```
123-
Performs RANGE partitioning for`relation` by partitioningkey`attribute`,`start_value` argument specifies initial value,`p_interval` sets the default range for auto created partitions or partitions created with`append_range_partition()` or`prepend_range_partition()` (if`NULL` then auto partition creation feature won't work),`p_count` is the number of premade partitions (if not set then`pg_pathman` tries to determine it based onattribute values). Partition creation callback is invoked for each partition if set beforehand.
138+
Performs RANGE partitioning for`relation` by partitioningexpression`expr`,`start_value` argument specifies initial value,`p_interval` sets the default range for auto created partitions or partitions created with`append_range_partition()` or`prepend_range_partition()` (if`NULL` then auto partition creation feature won't work),`p_count` is the number of premade partitions (if not set then`pg_pathman` tries to determine it based onexpression's values). The`bounds` array can be built using`generate_range_bounds()`. Partition creation callback is invoked for each partition if set beforehand.
124139

125140
```plpgsql
126-
create_partitions_from_range(relation REGCLASS,
127-
attributeTEXT,
128-
start_value ANYELEMENT,
129-
end_value ANYELEMENT,
130-
p_interval ANYELEMENT,
131-
partition_dataBOOLEAN DEFAULT TRUE)
141+
generate_range_bounds(p_start ANYELEMENT,
142+
p_interval INTERVAL,
143+
p_countINTEGER)
132144

133-
create_partitions_from_range(relation REGCLASS,
134-
attributeTEXT,
135-
start_value ANYELEMENT,
136-
end_value ANYELEMENT,
137-
p_interval INTERVAL,
138-
partition_dataBOOLEAN DEFAULT TRUE)
145+
generate_range_bounds(p_start ANYELEMENT,
146+
p_interval ANYELEMENT,
147+
p_countINTEGER)
139148
```
140-
Performs RANGE-partitioning from specified range for`relation` by partitioning key`attribute`. Partition creation callback is invoked for each partition if set beforehand.
149+
Builds`bounds` array for`create_range_partitions()`.
150+
141151

142152
###Data migration
143153

@@ -157,7 +167,7 @@ Stops a background worker performing a concurrent partitioning task. Note: worke
157167
```plpgsql
158168
create_hash_update_trigger(parent REGCLASS)
159169
```
160-
Creates the trigger on UPDATE for HASH partitions. The UPDATE trigger isn't created by default because of the overhead. It's useful in cases when thekey attribute might change.
170+
Creates the trigger on UPDATE for HASH partitions. The UPDATE trigger isn't created by default because of the overhead. It's useful in cases when thepartitioning expression's value might change.
161171
```plpgsql
162172
create_range_update_trigger(parent REGCLASS)
163173
```
@@ -267,18 +277,22 @@ Set partition creation callback to be invoked for each attached or created parti
267277
```json
268278
/* RANGE-partitioned table abc (child abc_4) */
269279
{
270-
"parent":"abc",
271-
"parttype":"2",
272-
"partition":"abc_4",
273-
"range_max":"401",
274-
"range_min":"301"
280+
"parent":"abc",
281+
"parent_schema":"public",
282+
"parttype":"2",
283+
"partition":"abc_4",
284+
"partition_schema":"public",
285+
"range_max":"401",
286+
"range_min":"301"
275287
}
276288

277289
/* HASH-partitioned table abc (child abc_0) */
278290
{
279-
"parent":"abc",
280-
"parttype":"1",
281-
"partition":"abc_0"
291+
"parent":"abc",
292+
"parent_schema":"public",
293+
"parttype":"1",
294+
"partition":"abc_0",
295+
"partition_schema":"public"
282296
}
283297
```
284298

@@ -293,9 +307,10 @@ When INSERTing new data beyond the partitioning range, use SpawnPartitionsWorker
293307
```plpgsql
294308
CREATETABLEIF NOT EXISTS pathman_config (
295309
partrel REGCLASSNOT NULLPRIMARY KEY,
296-
attnameTEXTNOT NULL,
310+
exprTEXTNOT NULL,
297311
parttypeINTEGERNOT NULL,
298-
range_intervalTEXT);
312+
range_intervalTEXT,
313+
cooked_exprTEXT);
299314
```
300315
This table stores a list of partitioned tables.
301316

@@ -306,7 +321,7 @@ CREATE TABLE IF NOT EXISTS pathman_config_params (
306321
enable_parentBOOLEANNOT NULL DEFAULT TRUE,
307322
autoBOOLEANNOT NULL DEFAULT TRUE,
308323
init_callback REGPROCEDURENOT NULL DEFAULT0,
309-
spawn_using_bgwBOOLEANNOT NULL DEFAULT FALSE);
324+
spawn_using_bgwBOOLEANNOT NULL DEFAULT FALSE);
310325
```
311326
This table stores optional parameters which override standard behavior.
312327

@@ -337,7 +352,7 @@ RETURNS TABLE (
337352
parent REGCLASS,
338353
partition REGCLASS,
339354
parttype INT4,
340-
partattrTEXT,
355+
exprTEXT,
341356
range_minTEXT,
342357
range_maxTEXT)
343358
AS'pg_pathman','show_partition_list_internal'
@@ -348,6 +363,23 @@ AS SELECT * FROM show_partition_list();
348363
```
349364
This view lists all existing partitions, as well as their parents and range boundaries (NULL for HASH partitions).
350365

366+
####`pathman_cache_stats` --- per-backend memory consumption
367+
```plpgsql
368+
-- helper SRF function
369+
CREATEOR REPLACE FUNCTION @extschema@.show_cache_stats()
370+
RETURNS TABLE (
371+
contextTEXT,
372+
size INT8,
373+
used INT8,
374+
entries INT8)
375+
AS'pg_pathman','show_cache_stats_internal'
376+
LANGUAGE C STRICT;
377+
378+
CREATEOR REPLACE VIEW @extschema@.pathman_cache_stats
379+
ASSELECT*FROM @extschema@.show_cache_stats();
380+
```
381+
Shows memory consumption of various caches.
382+
351383

352384
##Custom plan nodes
353385
`pg_pathman` provides a couple of[custom plan nodes](https://wiki.postgresql.org/wiki/CustomScanAPI) which aim to reduce execution time, namely:
@@ -653,6 +685,7 @@ There are several user-accessible [GUC](https://www.postgresql.org/docs/9.5/stat
653685
-`pg_pathman.enable_runtimemergeappend` --- toggle`RuntimeMergeAppend` custom node on\off
654686
-`pg_pathman.enable_partitionfilter` --- toggle`PartitionFilter` custom node on\off
655687
-`pg_pathman.enable_auto_partition` --- toggle automatic partition creation on\off (per session)
688+
-`pg_pathman.enable_bounds_cache` --- toggle bounds cache on\off (faster updates of partitioning scheme)
656689
-`pg_pathman.insert_into_fdw` --- allow INSERTs into various FDWs`(disabled | postgres | any_fdw)`
657690
-`pg_pathman.override_copy` --- toggle COPY statement hooking on\off
658691

@@ -666,7 +699,8 @@ All sections and data will remain unchanged and will be handled by the standard
666699
Do not hesitate to post your issues, questions and new ideas at the[issues](https://github.com/postgrespro/pg_pathman/issues) page.
667700

668701
##Authors
669-
Ildar Musin <i.musin@postgrespro.ru> Postgres Professional Ltd., Russia
670-
Alexander Korotkov <a.korotkov@postgrespro.ru> Postgres Professional Ltd., Russia
671-
Dmitry Ivanov <d.ivanov@postgrespro.ru> Postgres Professional Ltd., Russia
672-
702+
Ildar Musin <i.musin(at)postgrespro.ru> Postgres Professional Ltd., Russia
703+
Alexander Korotkov <a.korotkov(at)postgrespro.ru> Postgres Professional Ltd., Russia
704+
Dmitry Ivanov <d.ivanov(at)postgrespro.ru> Postgres Professional Ltd., Russia
705+
Maksim Milyutin <m.milyutin(at)postgrespro.ru> Postgres Professional Ltd., Russia
706+
Ildus Kurbangaliev <i.kurbangaliev(at)postgrespro.ru> Postgres Professional Ltd., Russia

0 commit comments

Comments
 (0)

[8]ページ先頭

©2009-2025 Movatter.jp