Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitb47d54d

Browse files
authored
chore: cache terraform providers between CI test runs (#17373)
Addressescoder/internal#322.This PR starts caching Terraform providers used by `TestProvision` in`provisioner/terraform/provision_test.go`. The goal is to improve thereliability of this test by cutting down on the number of network callsto external services. It leverages GitHub Actions cache, which [on depotrunners is persisted for 14 days bydefault](https://depot.dev/docs/github-actions/overview#cache-retention-policy).Other than the aforementioned `TestProvision`, I couldn't find any othertests which depend on external terraform providers.
1 parent08ad910 commitb47d54d

File tree

7 files changed

+393
-34
lines changed

7 files changed

+393
-34
lines changed
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
name:"Download Test Cache"
2+
description:|
3+
Downloads the test cache and outputs today's cache key.
4+
A PR job can use a cache if it was created by its base branch, its current
5+
branch, or the default branch.
6+
https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache
7+
outputs:
8+
cache-key:
9+
description:"Today's cache key"
10+
value:${{ steps.vars.outputs.cache-key }}
11+
inputs:
12+
key-prefix:
13+
description:"Prefix for the cache key"
14+
required:true
15+
cache-path:
16+
description:"Path to the cache directory"
17+
required:true
18+
# This path is defined in testutil/cache.go
19+
default:"~/.cache/coderv2-test"
20+
runs:
21+
using:"composite"
22+
steps:
23+
-name:Get date values and cache key
24+
id:vars
25+
shell:bash
26+
run:|
27+
export YEAR_MONTH=$(date +'%Y-%m')
28+
export PREV_YEAR_MONTH=$(date -d 'last month' +'%Y-%m')
29+
export DAY=$(date +'%d')
30+
echo "year-month=$YEAR_MONTH" >> $GITHUB_OUTPUT
31+
echo "prev-year-month=$PREV_YEAR_MONTH" >> $GITHUB_OUTPUT
32+
echo "cache-key=${{ inputs.key-prefix }}-${YEAR_MONTH}-${DAY}" >> $GITHUB_OUTPUT
33+
34+
# TODO: As a cost optimization, we could remove caches that are older than
35+
# a day or two. By default, depot keeps caches for 14 days, which isn't
36+
# necessary for the test cache.
37+
# https://depot.dev/docs/github-actions/overview#cache-retention-policy
38+
-name:Download test cache
39+
uses:actions/cache/restore@5a3ec84eff668545956fd18022155c47e93e2684# v4.2.3
40+
with:
41+
path:${{ inputs.cache-path }}
42+
key:${{ steps.vars.outputs.cache-key }}
43+
# > If there are multiple partial matches for a restore key, the action returns the most recently created cache.
44+
# https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows#matching-a-cache-key
45+
# The second restore key allows non-main branches to use the cache from the previous month.
46+
# This prevents PRs from rebuilding the cache on the first day of the month.
47+
# It also makes sure that once a month, the cache is fully reset.
48+
restore-keys:|
49+
${{ inputs.key-prefix }}-${{ steps.vars.outputs.year-month }}-
50+
${{ github.ref != 'refs/heads/main' && format('{0}-{1}-', inputs.key-prefix, steps.vars.outputs.prev-year-month) || '' }}
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
name:"Upload Test Cache"
2+
description:Uploads the test cache. Only works on the main branch.
3+
inputs:
4+
cache-key:
5+
description:"Cache key"
6+
required:true
7+
cache-path:
8+
description:"Path to the cache directory"
9+
required:true
10+
# This path is defined in testutil/cache.go
11+
default:"~/.cache/coderv2-test"
12+
runs:
13+
using:"composite"
14+
steps:
15+
-name:Upload test cache
16+
if:${{ github.ref == 'refs/heads/main' }}
17+
uses:actions/cache/save@5a3ec84eff668545956fd18022155c47e93e2684# v4.2.3
18+
with:
19+
path:${{ inputs.cache-path }}
20+
key:${{ inputs.cache-key }}

‎.github/workflows/ci.yaml

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -341,6 +341,12 @@ jobs:
341341
-name:Setup Terraform
342342
uses:./.github/actions/setup-tf
343343

344+
-name:Download Test Cache
345+
id:download-cache
346+
uses:./.github/actions/test-cache/download
347+
with:
348+
key-prefix:test-go-${{ runner.os }}-${{ runner.arch }}
349+
344350
-name:Test with Mock Database
345351
id:test
346352
shell:bash
@@ -365,6 +371,11 @@ jobs:
365371
gotestsum --junitfile="gotests.xml" --jsonfile="gotests.json" \
366372
--packages="./..." -- $PARALLEL_FLAG -short -failfast
367373
374+
-name:Upload Test Cache
375+
uses:./.github/actions/test-cache/upload
376+
with:
377+
cache-key:${{ steps.download-cache.outputs.cache-key }}
378+
368379
-name:Upload test stats to Datadog
369380
timeout-minutes:1
370381
continue-on-error:true
@@ -462,6 +473,12 @@ jobs:
462473
if:runner.os == 'Windows'
463474
uses:./.github/actions/setup-imdisk
464475

476+
-name:Download Test Cache
477+
id:download-cache
478+
uses:./.github/actions/test-cache/download
479+
with:
480+
key-prefix:test-go-pg-${{ runner.os }}-${{ runner.arch }}
481+
465482
-name:Test with PostgreSQL Database
466483
env:
467484
POSTGRES_VERSION:"13"
@@ -476,6 +493,11 @@ jobs:
476493
477494
make test-postgres
478495
496+
-name:Upload Test Cache
497+
uses:./.github/actions/test-cache/upload
498+
with:
499+
cache-key:${{ steps.download-cache.outputs.cache-key }}
500+
479501
-name:Upload test stats to Datadog
480502
timeout-minutes:1
481503
continue-on-error:true
@@ -514,13 +536,24 @@ jobs:
514536
-name:Setup Terraform
515537
uses:./.github/actions/setup-tf
516538

539+
-name:Download Test Cache
540+
id:download-cache
541+
uses:./.github/actions/test-cache/download
542+
with:
543+
key-prefix:test-go-pg-16-${{ runner.os }}-${{ runner.arch }}
544+
517545
-name:Test with PostgreSQL Database
518546
env:
519547
POSTGRES_VERSION:"16"
520548
TS_DEBUG_DISCO:"true"
521549
run:|
522550
make test-postgres
523551
552+
-name:Upload Test Cache
553+
uses:./.github/actions/test-cache/upload
554+
with:
555+
cache-key:${{ steps.download-cache.outputs.cache-key }}
556+
524557
-name:Upload test stats to Datadog
525558
timeout-minutes:1
526559
continue-on-error:true
@@ -551,6 +584,12 @@ jobs:
551584
-name:Setup Terraform
552585
uses:./.github/actions/setup-tf
553586

587+
-name:Download Test Cache
588+
id:download-cache
589+
uses:./.github/actions/test-cache/download
590+
with:
591+
key-prefix:test-go-race-${{ runner.os }}-${{ runner.arch }}
592+
554593
# We run race tests with reduced parallelism because they use more CPU and we were finding
555594
# instances where tests appear to hang for multiple seconds, resulting in flaky tests when
556595
# short timeouts are used.
@@ -559,6 +598,11 @@ jobs:
559598
run:|
560599
gotestsum --junitfile="gotests.xml" -- -race -parallel 4 -p 4 ./...
561600
601+
-name:Upload Test Cache
602+
uses:./.github/actions/test-cache/upload
603+
with:
604+
cache-key:${{ steps.download-cache.outputs.cache-key }}
605+
562606
-name:Upload test stats to Datadog
563607
timeout-minutes:1
564608
continue-on-error:true
@@ -589,6 +633,12 @@ jobs:
589633
-name:Setup Terraform
590634
uses:./.github/actions/setup-tf
591635

636+
-name:Download Test Cache
637+
id:download-cache
638+
uses:./.github/actions/test-cache/download
639+
with:
640+
key-prefix:test-go-race-pg-${{ runner.os }}-${{ runner.arch }}
641+
592642
# We run race tests with reduced parallelism because they use more CPU and we were finding
593643
# instances where tests appear to hang for multiple seconds, resulting in flaky tests when
594644
# short timeouts are used.
@@ -600,6 +650,11 @@ jobs:
600650
make test-postgres-docker
601651
DB=ci gotestsum --junitfile="gotests.xml" -- -race -parallel 4 -p 4 ./...
602652
653+
-name:Upload Test Cache
654+
uses:./.github/actions/test-cache/upload
655+
with:
656+
cache-key:${{ steps.download-cache.outputs.cache-key }}
657+
603658
-name:Upload test stats to Datadog
604659
timeout-minutes:1
605660
continue-on-error:true

‎provisioner/terraform/executor.go

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,9 @@ type executor struct {
3535
mut*sync.Mutex
3636
binaryPathstring
3737
// cachePath and workdir must not be used by multiple processes at once.
38-
cachePathstring
39-
workdirstring
38+
cachePathstring
39+
cliConfigPathstring
40+
workdirstring
4041
// used to capture execution times at various stages
4142
timings*timingAggregator
4243
}
@@ -50,6 +51,9 @@ func (e *executor) basicEnv() []string {
5051
ife.cachePath!=""&&runtime.GOOS=="linux" {
5152
env=append(env,"TF_PLUGIN_CACHE_DIR="+e.cachePath)
5253
}
54+
ife.cliConfigPath!="" {
55+
env=append(env,"TF_CLI_CONFIG_FILE="+e.cliConfigPath)
56+
}
5357
returnenv
5458
}
5559

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp