Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit66b8221

Browse files
authored
support PARTITION for mysqldef and psqldef, adding --skip-partition to psqldef (#1036)
* psqldef: --skip-partition* support PARTITION* support pg partitions* implement partition diff* fix minor issues* resolve a parser rule conflict* tweaks* handle mariadb/tidb variance for partition names* rewrite flavor algorithm: test with flavor is always run, and mis-setting of flavor is not allowed* disable git diff for parser.go* fix flavor behavior* remove unused code* fix tests
1 parent95ce31f commit66b8221

29 files changed

+7706
-6008
lines changed

‎AGENTS.md‎

Lines changed: 47 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -12,14 +12,14 @@ Each command follows the same pattern: it accepts connection parameters similar
1212
##General Rules
1313

1414
* Never commit changes unless the user explicitly requests it
15-
*Write comments to explain non-obvious code. Focus on explaining the "why" rather than the "what"
16-
* Format SQLstatementsin string literals
15+
*Only write comments to explain non-obvious code. Focus on explaining the "why" rather than the "what"
16+
* Format SQL in string literals
1717
* Use`log/slog` to trace internal state. Set`LOG_LEVEL=debug` to enable debug logging
1818
* Use`panic` to assert unreachable code paths
1919
* Be aware of the two escaping modes:
20-
*`legacy_ignore_quotes: true` (thedefault, backwardcompatible mode) generates SQL with always quoted identifiers
21-
*`legacy_ignore_quotes: false` (quote-aware mode) generates SQL with identifiers quoted only when they are quoted in thesource SQL
22-
* If youfoundunsupportedfeatures, don't"fix" testsby rewriting nottouse such features. Instead, comment out the test case and mark it as`FIXME`
20+
*`legacy_ignore_quotes: true` (default; backward-compatible) generates SQL withidentifiersalways quoted
21+
*`legacy_ignore_quotes: false` (quote-aware) generates SQL with identifiers quoted only when they are quoted in theinput SQL
22+
* If youencounter anunsupportedfeature, don'trewrite tests toavoid it. Instead, comment out the test case and mark it as`FIXME`
2323

2424
##Build
2525

@@ -29,7 +29,7 @@ Build all the sqldef commands (`mysqldef`, `psqldef`, `sqlite3def`, `mssqldef`):
2929
make build
3030
```
3131

32-
The executable binaries will be placed in the`build/$os-$arch/` directory.
32+
The executable binaries will be placed in the`build/$os-$arch/` directory, where`$os` is`go env GOOS` and`$arch` is`go env GOARCH`.
3333

3434
###The Parser
3535

@@ -42,7 +42,7 @@ make parser-v # same as above, also writes a conflict report to y.output
4242

4343
Requirements:
4444
- No reduce/reduce conflicts are allowed
45-
-No moreshift/reduce conflicts are allowed unless absolutely necessary
45+
-Do not introduce newshift/reduce conflicts unless absolutely necessary
4646
- To resolve conflicts, use`make parser-v` and inspect`y.output`
4747

4848
Usage notes:
@@ -51,50 +51,29 @@ Usage notes:
5151
-`PSQLDEF_PARSER=generic` - Use only the generic parser (no fallback to pgquery)
5252
-`PSQLDEF_PARSER=pgquery` - Use only the pgquery parser (no fallback to generic)
5353
- Not set (default) - Use generic parser with fallback to pgquery
54-
- The generic parser builds ASTs, which the generatormanipulates for normalization and comparison. Do not parsestrings with regular expressions
54+
- The generic parser builds ASTs that the generatoruses for normalization and comparison. Do not parseSQL with regular expressions
5555
- The pgquery parser is obsolete and will be removed in the future; no maintenance is needed
5656
- Map iteration order is non-deterministic. Use`util.CanonicalMapIter` to iterate maps in a deterministic order
5757

5858
##Local Development
5959

60-
To have trial and error locally, you can usethefollowing commands:
60+
For local iteration,thetypical workflow is:
6161

6262
```sh
63-
# psqldef - export current schema
64-
build/$os-$arch/psqldef psqldef_test --export> schema.sql
63+
# Pick a tool/target (where $os is `go env GOOS` and $arch is `go env GOARCH`):
64+
TOOL="build/$(go env-$arch/psqldef psqldef_test"
65+
# TOOL="build/$os-$arch/mysqldef mysqldef_test"
66+
# TOOL="build/$os-$arch/sqlite3def sqlite3def.db"
67+
# TOOL="build/$os-$arch/mssqldef -PPassw0rd mssqldef_test" # password is mandatory
6568
66-
#psqldef - dry run to preview changes
67-
build/$os-$arch/psqldef psqldef_test --dry-run --file schema.sql
69+
#Export current schema
70+
$TOOL --export > schema.sql
6871
69-
#psqldef - apply schema from file
70-
build/$os-$arch/psqldef psqldef_test --apply --file schema.sql
72+
#Preview changes (dry run)
73+
$TOOL --dry-run --file schema.sql
7174
72-
# mysqldef - export current schema
73-
build/$os-$arch/mysqldef mysqldef_test --export> schema.sql
74-
75-
# mysqldef - dry run to preview changes
76-
build/$os-$arch/mysqldef mysqldef_test --dry-run --file schema.sql
77-
78-
# mysqldef - apply schema from file
79-
build/$os-$arch/mysqldef mysqldef_test --apply --file schema.sql
80-
81-
# mssqldef - export current schema (password is mandatory)
82-
build/$os-$arch/mssqldef -PPassw0rd mssqldef_test --export> schema.sql
83-
84-
# mssqldef - dry run to preview changes
85-
build/$os-$arch/mssqldef -PPassw0rd mssqldef_test --dry-run --file schema.sql
86-
87-
# mssqldef - apply schema from file
88-
build/$os-$arch/mssqldef -PPassw0rd mssqldef_test --apply --file schema.sql
89-
90-
# sqlite3def - export current schema
91-
build/$os-$arch/sqlite3def sqlite3def.db --export> schema.sql
92-
93-
# sqlite3def - dry run to preview changes
94-
build/$os-$arch/sqlite3def sqlite3def.db --dry-run --file schema.sql
95-
96-
# sqlite3def - apply schema from file
97-
build/$os-$arch/sqlite3def sqlite3def.db --apply --file schema.sql
75+
# Apply schema from file
76+
$TOOL --apply --file schema.sql
9877
```
9978
10079
## Running Tests
@@ -107,7 +86,7 @@ For development iterations, use these commands to run tests:
10786
make test # Takes approximately 10 minutes to complete
10887
```
10988
110-
###Run tests for specific`*def` tools
89+
### Run tests foraspecificsqldef tool
11190
11291
Thetest runner is`gotestsum`, which is a wrapper around`go test` that provides a more readable output.
11392
@@ -142,7 +121,7 @@ If you encounter `tls: handshake failure` errors with MySQL 5.7, enable RSA key
142121
GODEBUG=tlsrsakex=1 go test ./cmd/mysqldef
143122
```
144123
145-
The tests for mssqldef are flaky due tomssqlinstance issues. Insuch acase, restart it with`docker compose down mssql && docker compose up -d --wait mssql`,and run the tests again.
124+
The testsfor mssqldef are flaky due toSQL Serverinstance issues. Inthatcase, restart it with`docker compose down mssql && docker compose up -d --wait mssql`,then run the tests again.
146125
147126
Fortest coverage:
148127
@@ -209,8 +188,10 @@ TestCaseName:
209188
# Maximum database version supported
210189
max_version:"14.0"
211190
212-
# Database flavor requirement (mysqldef only)
213-
# One of "mysql", "mariadb", "tidb"
191+
# Database flavor requirement for flavor-specific features
192+
# One of "mysql", "mariadb", "tidb" for mysqldef, and "pgvector" for psqldef
193+
# Supports positive and negative matching like "!tidb" to exclude TiDB
194+
# See "Flavor Behavior" section below for details
214195
flavor:"mariadb"
215196
216197
# User to run the test as
@@ -227,6 +208,11 @@ TestCaseName:
227208
# Use offline testing only for proprietary SQL dialects such as Aurora DSQL (defaults to false)
228209
offline:true
229210
211+
# Quote handling mode
212+
# true = ignore quotes (incorrectly ignore case-sensitivity; legacy default)
213+
# false = preserve quotes (correctly handle case-sensitivity; future default)
214+
legacy_ignore_quotes:false
215+
230216
# Configuration options for the test
231217
config:
232218
# Create indexes concurrently (psqldef only)
@@ -237,6 +223,19 @@ The `up` and `down` fields must both be specified or both be omitted:
237223
- Both specified: asserts`current → desired` produces`up` and`desired → current` produces`down`
238224
- Both omitted: idempotency-only test; DDLs are applied but not asserted (must be valid SQL unless`offline: true`)
239225
226+
#### Flavor Behavior
227+
228+
The`flavor` field controls flavor-specifictest behavior, which validates that tests correctly fail on non-matching flavors:
229+
230+
| Scenario| Result|
231+
|----------|--------|
232+
| Flavor matches,test passes| PASS|
233+
| Flavor matches,test fails| FAIL|
234+
| Flavor doesn't match, test fails | SKIP |
235+
| Flavor doesn't match,test passes| FAIL|
236+
237+
This design ensures that flavor annotations are accurate. If you add`flavor: mariadb` to a test, thetest must actually fail on MySQL/TiDB. If it passes, the flavor annotation is wrong and should be removed.
238+
240239
#### Notes for Writing Tests
241240
242241
* YAMLtest cases default to`enable_drop: true`, which differs from the default behavior of sqldef tools
@@ -246,6 +245,7 @@ The `up` and `down` fields must both be specified or both be omitted:
246245
- SQLite3
247246
- SQL Server
248247
* Add`legacy_ignore_quotes: false`fornew test cases. This is the default behaviorin the future.
248+
* Do not add trivial comments totest cases. Instead, describe what is testedintestcase names.
249249
250250
## Documentation
251251
@@ -261,8 +261,6 @@ Markdown files document the usage of each command. Keep them up to date:
261261
Before considering a task complete, run these commands to ensure the code isin a good state:
262262
263263
*`make build`
264-
* `make test`
265-
* `make modernize`
266-
* `make lint`
267-
* `make vulncheck`
264+
*`make test-all-flavors`
268265
*`make format`
266+
*`make lint`

‎Makefile‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,7 @@
11
# This doesn't work for psqldef due to lib/pq
22
GOFLAGS := -tags netgo -installsuffix netgo -ldflags '-w -s'
3-
GOVERSION=$(shell go version)
4-
GOOS ?=$(word 1,$(subst /, ,$(lastword$(GOVERSION))))
5-
GOARCH ?=$(word 2,$(subst /, ,$(lastword$(GOVERSION))))
3+
GOOS ?=$(shell go env GOOS)
4+
GOARCH ?=$(shell go env GOARCH)
65
BUILD_DIR=build/$(GOOS)-$(GOARCH)
76
SQLDEF=$(shell pwd)
87
MACOS_VERSION := 11.3
@@ -23,7 +22,7 @@ else
2322
GOTEST := go run gotest.tools/gotestsum@latest --hide-summary=skipped --$(GOTESTFLAGS)
2423
endif
2524

26-
.PHONY: all build clean deps goyacc package package-zip package-targz parser parser-v build-mysqldef build-sqlite3def build-mssqldef build-psqldef test-cov test-cov-xml test-core test test-example test-example-offline vulncheck
25+
.PHONY: all build clean deps goyacc package package-zip package-targz parser parser-v build-mysqldef build-sqlite3def build-mssqldef build-psqldef test-cov test-cov-xml test-core test test-example test-example-offlinetest-all-flavorsvulncheck
2726

2827
all: build
2928

@@ -104,6 +103,11 @@ test-example-offline:
104103
./example/run-offline.sh sqlite3def
105104
./example/run-offline.sh mssqldef
106105

106+
test-all-flavors: test
107+
MYSQL_FLAVOR=mariadb MYSQL_PORT=3307$(GOTEST) ./cmd/mysqldef
108+
MYSQL_FLAVOR=tidb MYSQL_PORT=4000$(GOTEST) ./cmd/mysqldef
109+
PG_FLAVOR=pgvector PGPORT=55432$(GOTEST) ./cmd/psqldef
110+
107111
test-example:
108112
./example/run.sh psqldef
109113
./example/run.sh mysqldef

‎cmd-psqldef.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ Application Options:
2222
--enable-drop Enable destructive changes such as DROP for TABLE, SCHEMA, ROLE, USER, FUNCTION, PROCEDURE, TRIGGER, VIEW, INDEX, SEQUENCE, TYPE
2323
--skip-view Skip managing views/materialized views
2424
--skip-extension Skip managing extensions
25+
--skip-partition Skip managing partitioned tables
2526
--before-apply=SQL Execute the given string before applying the regular DDLs
2627
--config=PATH YAML configuration file (can be specified multiple times)
2728
--config-inline=YAML YAML configuration as inline string (can be specified multiple times)

‎cmd/mysqldef/tests_constraints.yml‎

Lines changed: 16 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ ConstraintCheck:
1212
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin ROW_FORMAT=DYNAMIC;
1313
min_version:'8.0'
1414
ColumnCheck:
15+
flavor:"!mariadb"
1516
desired:|
1617
CREATE TABLE `books` (
1718
`id` int(10) UNSIGNED AUTO_INCREMENT PRIMARY KEY,
@@ -20,7 +21,6 @@ ColumnCheck:
2021
`updated_at` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
2122
`deleted_at` datetime DEFAULTnull
2223
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin ROW_FORMAT=DYNAMIC;
23-
flavor:mysql# Different versioning of MariaDB
2424
min_version:'8.0'
2525
ForeignKeyNormalizeRestrict:
2626
desired:|
@@ -144,6 +144,7 @@ CheckInClauseWithMultipleConditions:
144144
);
145145
min_version:'8.0'
146146
AddCheckConstraintWithIn:
147+
flavor:"!mariadb"
147148
current:|
148149
CREATE TABLE `documents` (
149150
`id` int PRIMARY KEY,
@@ -159,9 +160,9 @@ AddCheckConstraintWithIn:
159160
ALTER TABLE `documents` ADD CONSTRAINT `documents_state_chk` CHECK (state in ('draft', 'review', 'published', 'archived'));
160161
down:|
161162
ALTER TABLE `documents` DROP CHECK `documents_state_chk`;
162-
flavor:mysql
163163
min_version:'8.0'
164164
ModifyCheckConstraintIn:
165+
flavor:"!mariadb"
165166
current:|
166167
CREATE TABLE `tasks` (
167168
`id` int PRIMARY KEY,
@@ -180,9 +181,9 @@ ModifyCheckConstraintIn:
180181
down:|
181182
ALTER TABLE `tasks` DROP CHECK `tasks_status_chk`;
182183
ALTER TABLE `tasks` ADD CONSTRAINT `tasks_status_chk` CHECK (status in ('todo', 'in_progress'));
183-
flavor:mysql
184184
min_version:'8.0'
185185
DropCheckConstraintWithIn:
186+
flavor:"!mariadb"
186187
current:|
187188
CREATE TABLE `events` (
188189
`id` int PRIMARY KEY,
@@ -198,7 +199,6 @@ DropCheckConstraintWithIn:
198199
ALTER TABLE `events` DROP CHECK `events_type_chk`;
199200
down:|
200201
ALTER TABLE `events` ADD CONSTRAINT `events_type_chk` CHECK (type in ('public', 'private'));
201-
flavor:mysql
202202
min_version:'8.0'
203203
CheckInClauseWithNegation:
204204
desired:|
@@ -248,7 +248,6 @@ CheckTableLevelToColumnLevel:
248248
);
249249
up:""
250250
down:""
251-
flavor:mysql
252251
min_version:'8.0'
253252
CheckTableLevelToUnnamedColumnLevel:
254253
current:|
@@ -264,7 +263,6 @@ CheckTableLevelToUnnamedColumnLevel:
264263
);
265264
up:""
266265
down:""
267-
flavor:mysql
268266
min_version:'8.0'
269267
CheckDuplicateValuesInOrChain:
270268
current:|
@@ -281,9 +279,9 @@ CheckDuplicateValuesInOrChain:
281279
);
282280
up:""
283281
down:""
284-
flavor:mysql
285282
min_version:'8.0'
286283
CheckRowValueConstructorMultipleInnerValues:
284+
flavor:"!mariadb"
287285
current:|
288286
CREATE TABLE `test_multi` (
289287
`col_a` int,
@@ -302,7 +300,6 @@ CheckRowValueConstructorMultipleInnerValues:
302300
down:|
303301
ALTER TABLE `test_multi` DROP CHECK `test_multi_chk`;
304302
ALTER TABLE `test_multi` ADD CONSTRAINT `test_multi_chk` CHECK ((col_a, col_b) in ((3, 1), (2, 4)));
305-
flavor:mysql
306303
min_version:'8.0'
307304
CheckRowValueConstructorOuterOrderShouldMatch:
308305
current:|
@@ -319,5 +316,15 @@ CheckRowValueConstructorOuterOrderShouldMatch:
319316
);
320317
up:""
321318
down:""
322-
flavor:mysql
323319
min_version:'8.0'
320+
321+
# Table with CHECK constraint via COMMENT (works on both MySQL and MariaDB)
322+
TableWithCommentAndDefaults:
323+
desired:|
324+
CREATE TABLE `books` (
325+
`id` int(10) UNSIGNED AUTO_INCREMENT PRIMARY KEY,
326+
`type` VARCHAR(255) NOT NULL COMMENT 'manga, novel',
327+
`created_at` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
328+
`updated_at` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
329+
`deleted_at` datetime DEFAULTnull
330+
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin ROW_FORMAT=DYNAMIC;

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp