Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit38867b0

Browse files
authored
fix: Re-enable parallel run of Postgres-backed tests (#119)
@kylecarbs and I were debugging a gnarly postgres issue over the weekend, and unfortunately it looks like it is still coming up occassionally:https://github.com/coder/coder/runs/5014420662?check_suite_focus=true#step:8:35 - so thought this might be a good testing Monday task.Intermittently, the test would fail with something like a `401` - invalid e-mail, or a `409` - initial user already created. This was quite surprising, because the tests are designed to spin up their own, isolated database.We tried a few things to debug this...## Attempt 1: Log out the generated port numbers when running the docker image.Based on the errors, it seemed like one test must be connecting to another test's database - that would explain why we'd get these conflicts! However, logging out the port number that came from docker always gave a unique number... and we couldn't find evidence of one database connecting to another.## Attempt 2: Store the database in unique, temporary folder.@kylecarbs and I found that the there was a [volume](https://github.com/docker-library/postgres/blob/a83005b407ee6d810413500d8a041c957fb10cf0/11/alpine/Dockerfile#L155) for the postgres data... so@kylecarbs implemented mounting the volume to a unique, per-test temporary folder in#89It sounded really promising... but unfortunately we hit the issue again!### Attempt 3... this PRAfter we hit the failure again, we noticed in the `docker ps` logs something quite strange:![image](https://user-images.githubusercontent.com/88213859/151913133-522a6c2e-977a-4a65-9315-804531ab7d77.png)When the docker image is run - it creates two port bindings, an IPv4 and an IPv6 one. These _should be the same_ - but surprisingly, they can sometimes be different. It isn't deterministic, and seems to be more common when there are multiple containers running. Importantly, __they can overlap__ as in the above image. Turns out, it seems this is a docker bug:moby/moby#42442 - which may be fixed in newer versions.To work around this bug, we have to manipulate the port bindings (like you would with `-p`) at the command line. We can do this with `docker`/`dockertest`, but it means we have to get a free port ahead of time to know which port to map.With that fix in - the `docker ps` is a little more sane:![image](https://user-images.githubusercontent.com/88213859/151913432-5f86bc09-8604-4355-ad49-0abeaf8cc0fe.png)...and hopefully means we can safely run the containers in parallel again.
1 parentc54d61e commit38867b0

File tree

2 files changed

+31
-1
lines changed

2 files changed

+31
-1
lines changed

‎.github/workflows/coder.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -164,7 +164,7 @@ jobs:
164164
run:
165165
DB=true gotestsum --jsonfile="gotests.json" --packages="./..." --
166166
-covermode=atomic -coverprofile="gotests.coverage" -timeout=3m
167-
-count=1 -race -parallel=1
167+
-count=1 -race -parallel=2
168168

169169
-uses:codecov/codecov-action@v2
170170
if:github.actor != 'dependabot[bot]'

‎database/postgres/postgres.go‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import (
44
"database/sql"
55
"fmt"
66
"io/ioutil"
7+
"net"
78
"os"
89
"time"
910

@@ -22,6 +23,13 @@ func Open() (string, func(), error) {
2223
iferr!=nil {
2324
return"",nil,xerrors.Errorf("create tempdir: %w",err)
2425
}
26+
// Pick an explicit port on the host to connect to 5432.
27+
// This is necessary so we can configure the port to only use ipv4.
28+
port,err:=getFreePort()
29+
iferr!=nil {
30+
return"",nil,xerrors.Errorf("Unable to get free port: %w",err)
31+
}
32+
2533
resource,err:=pool.RunWithOptions(&dockertest.RunOptions{
2634
Repository:"postgres",
2735
Tag:"11",
@@ -33,6 +41,15 @@ func Open() (string, func(), error) {
3341
"PGDATA=/tmp",
3442
"listen_addresses = '*'",
3543
},
44+
PortBindings:map[docker.Port][]docker.PortBinding{
45+
"5432/tcp": {{
46+
// Manually specifying a host IP tells Docker just to use an IPV4 address.
47+
// If we don't do this, we hit a fun bug:
48+
// https://github.com/moby/moby/issues/42442
49+
// where the ipv4 and ipv6 ports might be _different_ and collide with other running docker containers.
50+
HostIP:"0.0.0.0",
51+
HostPort:fmt.Sprintf("%d",port)}},
52+
},
3653
Mounts: []string{
3754
// The postgres image has a VOLUME parameter in it's image.
3855
// If we don't mount at this point, Docker will allocate a
@@ -76,3 +93,16 @@ func Open() (string, func(), error) {
7693
_=os.RemoveAll(tempDir)
7794
},nil
7895
}
96+
97+
// getFreePort asks the kernel for a free open port that is ready to use.
98+
funcgetFreePort() (portint,errerror) {
99+
// Binding to port 0 tells the OS to grab a port for us:
100+
// https://stackoverflow.com/questions/1365265/on-localhost-how-do-i-pick-a-free-port-number
101+
listener,err:=net.Listen("tcp","localhost:0")
102+
iferr!=nil {
103+
return0,err
104+
}
105+
106+
deferlistener.Close()
107+
returnlistener.Addr().(*net.TCPAddr).Port,nil
108+
}

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp