NotificationsYou must be signed in to change notification settings
Fork146
Star1.5k

[WIP] Simple table level sharding implementation#129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Open

auxten wants to merge29 commits intodevelop

base:develop

Choose a base branch

fromfeature/shard

Open

[WIP] Simple table level sharding implementation#129

auxten wants to merge29 commits intodevelopfromfeature/shard

Conversation

Copy link

Contributor

auxten commentedDec 3, 2018

Simple SELECT without GROUP, LIMIT offset, HAVING, DISTINCT
Almost all kinds of INSERT without subqueries

auxten added17 commits

November 22, 2018 21:02

Shard framework can work now fill in the logic latter

e6ff8a0

Add shard plan

a1c02d9

Sharding driver INSERT complete

4111d0e

Recover xenomint/sqlite/sqlite.go

3b242f0

Simplify buildInsertPlan

46884ce

Fix test case for shard

2b0b704

Fix named args in shard driver

8fa4453

Make Primitive.ExecContext args with a sql.Tx

382fe43

Add separate func ShardingConn.splitQueryArgs

29c8f71

Add complex select for test

42c0e9a

Add doc.go to todo list

e29769e

Sharding supports INSERT, simple SELECT.

4d7a05a

Merge remote-tracking branch 'origin/develop' into feature/shard

e928daf

Add test cases, fix bugs

c911c3c

Disable usage of github.com/y0ssar1an/q for debug

6caa75f

Merge branch 'feature/cancelQuery' into feature/shard

0d84304

Shard driver supports UPDATE and DELETE

a9d67a3

Copy link

codecovbot commentedDec 6, 2018

Codecov Report

Merging#129 intodevelop willdecrease coverage by0.95%.
The diff coverage is69.81%.

@@             Coverage Diff             @@##           develop     #129      +/-   ##===========================================- Coverage    77.47%   76.51%   -0.96%===========================================  Files          142      149       +7       Lines        10251    11115     +864     ===========================================+ Hits          7942     8505     +563- Misses        1684     1910     +226- Partials       625      700      +75

Copy link

codecovbot commentedDec 6, 2018•
edited
Loading

Codecov Report

Merging#129 intodevelop willincrease coverage by0.26%.
The diff coverage is73.29%.

@@            Coverage Diff             @@##           develop    #129      +/-   ##==========================================+ Coverage    76.74%     77%   +0.26%==========================================  Files          146     151       +5       Lines        10890   11313     +423     ==========================================+ Hits          8357    8712     +355- Misses        1845    1898      +53- Partials       688     703      +15

auxten added5 commits

December 6, 2018 20:50

Merge remote-tracking branch 'origin/develop' into feature/shard

d813032

Mock query context for db_test.go

c5f737e

Move context to the first arg

fbe4d42

Merge remote-tracking branch 'origin/develop' into feature/shard

66da2ef

Add more test cases for shard

dfe8701

auxten requested review fromggicci,leventeliu,xq262144 andzeqing-guo

December 14, 2018 07:47

xq262144 reviewed

Dec 17, 2018

View reviewed changes

shard/engine.go OutdatedShow resolvedHide resolved

xq262144 reviewed

Dec 17, 2018

View reviewed changes

shard/engine.go OutdatedShow resolvedHide resolved

leventeliu reviewed

Dec 17, 2018

View reviewed changes

shard/engine_test.go Outdated

		So(err, ShouldNotBeNil)
		So(its, ShouldEqual, -1)
		})
		}

Copy link

Contributor

leventeliuDec 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

File is not formatted.

shard/insert.go OutdatedShow resolvedHide resolved

shard/insert.goShow resolvedHide resolved

shard/plan.goShow resolvedHide resolved

xq262144 reviewed

Dec 17, 2018

View reviewed changes

shard/plan.go

		rowCh <-chan []interface{}
		)
		for i, ins := range instructions {
		colNames, _, rowCh, err = plan.selector(ctx, errCh, stopCh, ins, TempTableRowBufSize)

Copy link

Contributor

xq262144Dec 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

colNames may not be the same schema as the original table.

xq262144 reviewed

Dec 17, 2018

View reviewed changes

shard/plan.go OutdatedShow resolvedHide resolved

auxtenand others added7 commits

December 17, 2018 18:30

Format code

7e9fe7c

Forbid sub query and unsupported funcs

e51b713

Remove redundant mutex init

c8a7c4e

Fix miss used ctx.Err()

dad7354

Add test case "select id, count(id), *, sum(id) from foo"

fe7ed10

Merge remote-tracking branch 'origin/develop' into feature/shard

264cc99

Merge branch 'develop' into feature/shard

503c530

ggicci approved these changes

Dec 29, 2018

View reviewed changes

auxten changed the title~~Simple table level sharding implementation~~[WIP] Simple table level sharding implementation

Jan 3, 2019

Copy link

stalebot commentedApr 29, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.