Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

refactor: change template archive extraction to be on provisioner#9264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
spikecurtis merged 17 commits intomainfromspike/9165-provisioner-filesystem
Aug 25, 2023
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
Show all changes
17 commits
Select commitHold shift + click to select a range
935719c
refactor provisionersdk protocol
spikecurtisAug 23, 2023
a29592a
refactor provisioners to use new protocol
spikecurtisAug 23, 2023
1f0efb4
refactor provisionerd to use new protocol
spikecurtisAug 23, 2023
2be6cf0
refactor tests & proto renames
spikecurtisAug 23, 2023
39d6065
Fixes from self-review
spikecurtisAug 23, 2023
e39b8a2
appease fmt & link
spikecurtisAug 23, 2023
d92608e
code review fixes & e2e fixes
spikecurtisAug 23, 2023
0412773
More fmt
spikecurtisAug 23, 2023
83deae6
Merge branch 'main' of https://github.com/coder/coder into spike/9165…
spikecurtisAug 23, 2023
d1a9b2a
Code review fixes
spikecurtisAug 24, 2023
8d7261f
Merge branch 'main' of https://github.com/coder/coder into spike/9165…
spikecurtisAug 24, 2023
ef1820f
new gen; use uuid for session workdir
spikecurtisAug 24, 2023
96e1e64
Revert nix-based gen CI task until dogfood is on nix
spikecurtisAug 24, 2023
2805dda
Merge branch 'main' of https://github.com/coder/coder into spike/9165…
spikecurtisAug 24, 2023
9762158
revert deleting dogfood Docker stuff
spikecurtisAug 24, 2023
cf8a48b
Merge branch 'main' of https://github.com/coder/coder into spike/9165…
spikecurtisAug 25, 2023
3d7b6a8
Revert "revert deleting dogfood Docker stuff"
spikecurtisAug 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletionsMakefile
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -456,10 +456,10 @@ DB_GEN_FILES := \

# all gen targets should be added here and to gen/mark-fresh
gen: \
coderd/database/dump.sql \
$(DB_GEN_FILES) \
provisionersdk/proto/provisioner.pb.go \
provisionerd/proto/provisionerd.pb.go \
coderd/database/dump.sql \
$(DB_GEN_FILES) \
site/src/api/typesGenerated.ts \
coderd/rbac/object_gen.go \
docs/admin/prometheus.md \
Expand All@@ -478,10 +478,10 @@ gen: \
# used during releases so we don't run generation scripts.
gen/mark-fresh:
files="\
coderd/database/dump.sql \
$(DB_GEN_FILES) \
provisionersdk/proto/provisioner.pb.go \
provisionerd/proto/provisionerd.pb.go \
coderd/database/dump.sql \
$(DB_GEN_FILES) \
site/src/api/typesGenerated.ts \
coderd/rbac/object_gen.go \
docs/admin/prometheus.md \
Expand Down
18 changes: 9 additions & 9 deletionscli/agent_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -75,9 +75,9 @@ func TestWorkspaceAgent(t *testing.T) {
user := coderdtest.CreateFirstUser(t, client)
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: echo.ParseComplete,
ProvisionApply: []*proto.Provision_Response{{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{
ProvisionApply: []*proto.Response{{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I can see that this PR does a lot of renaming. Is it possible to extract them to a separate PR? Maybe it could shrink this PR a bit.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It's not really possible to put in a separate PR, since the protocol changes are fundamental, and without renames nothing builds.

I've split it out into different commits to reduce the noise

Copy link
Member

@mtojekmtojekAug 23, 2023•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I admit that I focus mainly on changes in the CLI package like:

  • echo.ProvisionComplete ->echo.ApplyComplete
  • proto.Provision_Response ->proto.Response
  • completeWithAgent()

but if these are strongly coupled with the protocol, then please ignore the comment.

Type: &proto.Response_Apply{
Apply: &proto.ApplyComplete{
Resources: []*proto.Resource{{
Name: "somename",
Type: "someinstance",
Expand DownExpand Up@@ -127,9 +127,9 @@ func TestWorkspaceAgent(t *testing.T) {
user := coderdtest.CreateFirstUser(t, client)
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: echo.ParseComplete,
ProvisionApply: []*proto.Provision_Response{{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{
ProvisionApply: []*proto.Response{{
Type: &proto.Response_Apply{
Apply: &proto.ApplyComplete{
Resources: []*proto.Resource{{
Name: "somename",
Type: "someinstance",
Expand DownExpand Up@@ -179,9 +179,9 @@ func TestWorkspaceAgent(t *testing.T) {
user := coderdtest.CreateFirstUser(t, client)
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: echo.ParseComplete,
ProvisionApply: []*proto.Provision_Response{{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{
ProvisionApply: []*proto.Response{{
Type: &proto.Response_Apply{
Apply: &proto.ApplyComplete{
Resources: []*proto.Resource{{
Name: "somename",
Type: "someinstance",
Expand Down
21 changes: 5 additions & 16 deletionscli/configssh_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -82,9 +82,9 @@ func TestConfigSSH(t *testing.T) {
authToken := uuid.NewString()
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: echo.ParseComplete,
ProvisionPlan: []*proto.Provision_Response{{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{
ProvisionPlan: []*proto.Response{{
Type: &proto.Response_Plan{
Plan: &proto.PlanComplete{
Resources: []*proto.Resource{{
Name: "example",
Type: "aws_instance",
Expand DownExpand Up@@ -720,22 +720,11 @@ func TestConfigSSH_Hostnames(t *testing.T) {
resources = append(resources, resource)
}

provisionResponse := []*proto.Provision_Response{{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{
Resources: resources,
},
},
}}

client := coderdtest.New(t, &coderdtest.Options{IncludeProvisionerDaemon: true})
user := coderdtest.CreateFirstUser(t, client)
// authToken := uuid.NewString()
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: echo.ParseComplete,
ProvisionPlan: provisionResponse,
ProvisionApply: provisionResponse,
})
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID,
echo.WithResources(resources))
coderdtest.AwaitTemplateVersionJob(t, client, version.ID)
template := coderdtest.CreateTemplate(t, client, user.OrganizationID, version.ID)
workspace := coderdtest.CreateWorkspace(t, client, user.OrganizationID, template.ID)
Expand Down
95 changes: 29 additions & 66 deletionscli/create_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -29,11 +29,7 @@ func TestCreate(t *testing.T) {
t.Parallel()
client := coderdtest.New(t, &coderdtest.Options{IncludeProvisionerDaemon: true})
user := coderdtest.CreateFirstUser(t, client)
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: echo.ParseComplete,
ProvisionApply: provisionCompleteWithAgent,
ProvisionPlan: provisionCompleteWithAgent,
})
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, completeWithAgent())
coderdtest.AwaitTemplateVersionJob(t, client, version.ID)
template := coderdtest.CreateTemplate(t, client, user.OrganizationID, version.ID)
args := []string{
Expand DownExpand Up@@ -84,11 +80,7 @@ func TestCreate(t *testing.T) {
t.Parallel()
client := coderdtest.New(t, &coderdtest.Options{IncludeProvisionerDaemon: true})
owner := coderdtest.CreateFirstUser(t, client)
version := coderdtest.CreateTemplateVersion(t, client, owner.OrganizationID, &echo.Responses{
Parse: echo.ParseComplete,
ProvisionApply: provisionCompleteWithAgent,
ProvisionPlan: provisionCompleteWithAgent,
})
version := coderdtest.CreateTemplateVersion(t, client, owner.OrganizationID, completeWithAgent())
coderdtest.AwaitTemplateVersionJob(t, client, version.ID)
template := coderdtest.CreateTemplate(t, client, owner.OrganizationID, version.ID)
_, user := coderdtest.CreateAnotherUser(t, client, owner.OrganizationID)
Expand DownExpand Up@@ -141,11 +133,7 @@ func TestCreate(t *testing.T) {
t.Parallel()
client := coderdtest.New(t, &coderdtest.Options{IncludeProvisionerDaemon: true})
user := coderdtest.CreateFirstUser(t, client)
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: echo.ParseComplete,
ProvisionApply: provisionCompleteWithAgent,
ProvisionPlan: provisionCompleteWithAgent,
})
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, completeWithAgent())
coderdtest.AwaitTemplateVersionJob(t, client, version.ID)
template := coderdtest.CreateTemplate(t, client, user.OrganizationID, version.ID, func(ctr *codersdk.CreateTemplateRequest) {
var defaultTTLMillis int64 = 2 * 60 * 60 * 1000 // 2 hours
Expand DownExpand Up@@ -240,6 +228,22 @@ func TestCreate(t *testing.T) {
})
}

func prepareEchoResponses(parameters []*proto.RichParameter) *echo.Responses {
return &echo.Responses{
Parse: echo.ParseComplete,
ProvisionPlan: []*proto.Response{
{
Type: &proto.Response_Plan{
Plan: &proto.PlanComplete{
Parameters: parameters,
},
},
},
},
ProvisionApply: echo.ApplyComplete,
}
}

func TestCreateWithRichParameters(t *testing.T) {
t.Parallel()

Expand All@@ -258,27 +262,12 @@ func TestCreateWithRichParameters(t *testing.T) {
immutableParameterValue = "4"
)

echoResponses := &echo.Responses{
Parse: echo.ParseComplete,
ProvisionPlan: []*proto.Provision_Response{
{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{
Parameters: []*proto.RichParameter{
{Name: firstParameterName, Description: firstParameterDescription, Mutable: true},
{Name: secondParameterName, DisplayName: secondParameterDisplayName, Description: secondParameterDescription, Mutable: true},
{Name: immutableParameterName, Description: immutableParameterDescription, Mutable: false},
},
},
},
},
},
ProvisionApply: []*proto.Provision_Response{{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{},
},
}},
}
echoResponses := prepareEchoResponses([]*proto.RichParameter{
{Name: firstParameterName, Description: firstParameterDescription, Mutable: true},
{Name: secondParameterName, DisplayName: secondParameterDisplayName, Description: secondParameterDescription, Mutable: true},
{Name: immutableParameterName, Description: immutableParameterDescription, Mutable: false},
},
)

t.Run("InputParameters", func(t *testing.T) {
t.Parallel()
Expand DownExpand Up@@ -427,28 +416,6 @@ func TestCreateValidateRichParameters(t *testing.T) {
{Name: boolParameterName, Type: "bool", Mutable: true},
}

prepareEchoResponses := func(richParameters []*proto.RichParameter) *echo.Responses {
return &echo.Responses{
Parse: echo.ParseComplete,
ProvisionPlan: []*proto.Provision_Response{
{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{
Parameters: richParameters,
},
},
},
},
ProvisionApply: []*proto.Provision_Response{
{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{},
},
},
},
}
}

t.Run("ValidateString", func(t *testing.T) {
t.Parallel()

Expand DownExpand Up@@ -626,20 +593,16 @@ func TestCreateWithGitAuth(t *testing.T) {
t.Parallel()
echoResponses := &echo.Responses{
Parse: echo.ParseComplete,
ProvisionPlan: []*proto.Provision_Response{
ProvisionPlan: []*proto.Response{
{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{
Type: &proto.Response_Plan{
Plan: &proto.PlanComplete{
GitAuthProviders: []string{"github"},
},
},
},
},
ProvisionApply: []*proto.Provision_Response{{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{},
},
}},
ProvisionApply: echo.ApplyComplete,
}

client := coderdtest.New(t, &coderdtest.Options{
Expand Down
2 changes: 1 addition & 1 deletioncli/gitssh_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -48,7 +48,7 @@ func prepareTestGitSSH(ctx context.Context, t *testing.T) (*codersdk.Client, str
agentToken := uuid.NewString()
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: echo.ParseComplete,
ProvisionPlan: echo.ProvisionComplete,
ProvisionPlan: echo.PlanComplete,
ProvisionApply: echo.ProvisionApplyWithAgent(agentToken),
})
template := coderdtest.CreateTemplate(t, client, user.OrganizationID, version.ID)
Expand Down
2 changes: 1 addition & 1 deletioncli/portforward_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -302,7 +302,7 @@ func runAgent(t *testing.T, client *codersdk.Client, userID uuid.UUID) codersdk.
agentToken := uuid.NewString()
version := coderdtest.CreateTemplateVersion(t, client, orgID, &echo.Responses{
Parse: echo.ParseComplete,
ProvisionPlan: echo.ProvisionComplete,
ProvisionPlan: echo.PlanComplete,
ProvisionApply: echo.ProvisionApplyWithAgent(agentToken),
})

Expand Down
42 changes: 11 additions & 31 deletionscli/restart_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -20,30 +20,14 @@ import (
func TestRestart(t *testing.T) {
t.Parallel()

echoResponses := &echo.Responses{
Parse: echo.ParseComplete,
ProvisionPlan: []*proto.Provision_Response{
{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{
Parameters: []*proto.RichParameter{
{
Name: ephemeralParameterName,
Description: ephemeralParameterDescription,
Mutable: true,
Ephemeral: true,
},
},
},
},
},
echoResponses := prepareEchoResponses([]*proto.RichParameter{
{
Name: ephemeralParameterName,
Description: ephemeralParameterDescription,
Mutable: true,
Ephemeral: true,
},
ProvisionApply: []*proto.Provision_Response{{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{},
},
}},
}
})

t.Run("OK", func(t *testing.T) {
t.Parallel()
Expand DownExpand Up@@ -187,10 +171,10 @@ func TestRestartWithParameters(t *testing.T) {

echoResponses := &echo.Responses{
Parse: echo.ParseComplete,
ProvisionPlan: []*proto.Provision_Response{
ProvisionPlan: []*proto.Response{
{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{
Type: &proto.Response_Plan{
Plan: &proto.PlanComplete{
Parameters: []*proto.RichParameter{
{
Name: immutableParameterName,
Expand All@@ -202,11 +186,7 @@ func TestRestartWithParameters(t *testing.T) {
},
},
},
ProvisionApply: []*proto.Provision_Response{{
Type: &proto.Provision_Response_Complete{
Complete: &proto.Provision_Complete{},
},
}},
ProvisionApply: echo.ApplyComplete,
}

t.Run("DoNotAskForImmutables", func(t *testing.T) {
Expand Down
13 changes: 8 additions & 5 deletionscli/server.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -41,7 +41,6 @@ import (
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/collectors"
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/spf13/afero"
"go.opentelemetry.io/otel/trace"
"golang.org/x/mod/semver"
"golang.org/x/oauth2"
Expand DownExpand Up@@ -1304,7 +1303,11 @@ func newProvisionerDaemon(
defer wg.Done()
defer cancel()

err := echo.Serve(ctx, afero.NewOsFs(), &provisionersdk.ServeOptions{Listener: echoServer})
err := echo.Serve(ctx, &provisionersdk.ServeOptions{
Listener: echoServer,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

missing Logger here?

WorkDirectory: workDir,
Logger: logger.Named("echo"),
})
if err != nil {
select {
case errCh <- err:
Expand DownExpand Up@@ -1336,10 +1339,11 @@ func newProvisionerDaemon(

err := terraform.Serve(ctx, &terraform.ServeOptions{
ServeOptions: &provisionersdk.ServeOptions{
Listener: terraformServer,
Listener: terraformServer,
Logger: logger.Named("terraform"),
WorkDirectory: workDir,
},
CachePath: tfDir,
Logger: logger.Named("terraform"),
Tracer: tracer,
})
if err != nil && !xerrors.Is(err, context.Canceled) {
Expand All@@ -1366,7 +1370,6 @@ func newProvisionerDaemon(
UpdateInterval: time.Second,
ForceCancelInterval: cfg.Provisioner.ForceCancelInterval.Value(),
Provisioners: provisioners,
WorkDirectory: workDir,
TracerProvider: coderAPI.TracerProvider,
Metrics: &metrics,
}), nil
Expand Down
7 changes: 1 addition & 6 deletionscli/show_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -7,7 +7,6 @@ import (

"github.com/coder/coder/v2/cli/clitest"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/provisioner/echo"
"github.com/coder/coder/v2/pty/ptytest"
)

Expand All@@ -17,11 +16,7 @@ func TestShow(t *testing.T) {
t.Parallel()
client := coderdtest.New(t, &coderdtest.Options{IncludeProvisionerDaemon: true})
user := coderdtest.CreateFirstUser(t, client)
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: echo.ParseComplete,
ProvisionApply: provisionCompleteWithAgent,
ProvisionPlan: provisionCompleteWithAgent,
})
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, completeWithAgent())
coderdtest.AwaitTemplateVersionJob(t, client, version.ID)
template := coderdtest.CreateTemplate(t, client, user.OrganizationID, version.ID)
workspace := coderdtest.CreateWorkspace(t, client, user.OrganizationID, template.ID)
Expand Down
Loading
[8]ページ先頭
©2009-2025 Movatter.jp