Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat: Read params from file for template/workspace creation#1541

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
AbhineetJain merged 13 commits intomainfrom1377-feat-load-parameters-from-file
May 20, 2022
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
Show all changes
13 commits
Select commitHold shift + click to select a range
b35b7b3
Read params from file for template/workspace creation
AbhineetJainMay 18, 2022
e69ae31
Use os.ReadFile
AbhineetJainMay 18, 2022
dc80543
Refactor reading params into a separate module
AbhineetJainMay 19, 2022
931e7d5
Add comments and unit tests
AbhineetJainMay 20, 2022
0e6b2d4
Fix merge conflicts and refactor
AbhineetJainMay 20, 2022
8a075f4
Rename variable
AbhineetJainMay 20, 2022
71e94e3
Uncomment and fix unit test
AbhineetJainMay 20, 2022
d27d202
Fix comment
AbhineetJainMay 20, 2022
692b975
Refactor tests
AbhineetJainMay 20, 2022
1a12be1
Fix unit tests for windows
AbhineetJainMay 20, 2022
1491620
Fix unit tests for Windows
AbhineetJainMay 20, 2022
74989ac
Add comments for the hotfix
AbhineetJainMay 20, 2022
e584e81
Merge branch 'main' into 1377-feat-load-parameters-from-file
AbhineetJainMay 20, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletionscli/create.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,6 +17,7 @@ func create() *cobra.Command {
var (
workspaceName string
templateName string
parameterFile string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

tfvars is the standard way to handle this with Terraform.

I'd prefer to not have our own solution, because that'd remove the ability for users to doterraform apply.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

@kylecarbs that assumes we use terraform, but our cli should be terraform agnostic right?

Alsotfvars is just HCL or yaml. So we could just support multiple file types

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Developers also won't have thetf file when runningcoder workspace create, only the person who made the template will have thetf file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

We can be Terraform agnostic while still consumingtfvars. Latching onto the existing Terraform ecosystem is a much better user experience than forking it with our own parameter format.

I don't believe it's necessary to have atfvars equivalent when creating a workspace. Or at least, we haven't seen it as a problem yet.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Issue#1377 mentions creating workspaces using parameter files, which is why I went with a non-tfvars approach. However,#1426 does talk about doing this for templates. Would you recommend solving both problems using different approaches or only work on#1426 (template creation via file) viatfvars? Let me quickly check how feasible usingtfvars for template creation is.

Copy link
ContributorAuthor

@AbhineetJainAbhineetJainMay 18, 2022
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Is there a documented flow of how the customers would useterraform apply when setting up templates? I tried using thetfvars file as follows:

$ terraform init$ terraform apply -var-file="<path_to_tfvars>"...Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

It did not create a template and callingcoder templates create after this still asked me for input, so I am not sure what the actual flow should be.

)
cmd := &cobra.Command{
Annotations: workspaceCommand,
Expand DownExpand Up@@ -116,23 +117,33 @@ func create() *cobra.Command {
return err
}

printed := false
// parameterMapFromFile can be nil if parameter file is not specified
var parameterMapFromFile map[string]string
if parameterFile != "" {
_, _ = fmt.Fprintln(cmd.OutOrStdout(), cliui.Styles.Paragraph.Render("Attempting to read the variables from the parameter file.")+"\r\n")
parameterMapFromFile, err = createParameterMapFromFile(parameterFile)
if err != nil {
return err
}
}

disclaimerPrinted := false
parameters := make([]codersdk.CreateParameterRequest, 0)
for _, parameterSchema := range parameterSchemas {
if !parameterSchema.AllowOverrideSource {
continue
}
if !printed {
if !disclaimerPrinted {
_, _ = fmt.Fprintln(cmd.OutOrStdout(), cliui.Styles.Paragraph.Render("This template has customizable parameters. Values can be changed after create, but may have unintended side effects (like data loss).")+"\r\n")
printed = true
disclaimerPrinted = true
}
value, err :=cliui.ParameterSchema(cmd, parameterSchema)
parameterValue, err :=getParameterValueFromMapOrInput(cmd, parameterMapFromFile, parameterSchema)
if err != nil {
return err
}
parameters = append(parameters, codersdk.CreateParameterRequest{
Name: parameterSchema.Name,
SourceValue:value,
SourceValue:parameterValue,
SourceScheme: codersdk.ParameterSourceSchemeData,
DestinationScheme: parameterSchema.DefaultDestinationScheme,
})
Expand DownExpand Up@@ -194,5 +205,6 @@ func create() *cobra.Command {
}

cliflag.StringVarP(cmd.Flags(), &templateName, "template", "t", "CODER_TEMPLATE_NAME", "", "Specify a template name.")
cliflag.StringVarP(cmd.Flags(), &parameterFile, "parameter-file", "", "CODER_PARAMETER_FILE", "", "Specify a file path with parameter values.")
return cmd
}
144 changes: 111 additions & 33 deletionscli/create_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,6 +2,7 @@ package cli_test

import (
"fmt"
"os"
"testing"

"github.com/stretchr/testify/require"
Expand DownExpand Up@@ -113,39 +114,7 @@ func TestCreate(t *testing.T) {

defaultValue := "something"
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: []*proto.Parse_Response{{
Type: &proto.Parse_Response_Complete{
Complete: &proto.Parse_Complete{
ParameterSchemas: []*proto.ParameterSchema{
{
AllowOverrideSource: true,
Name: "region",
Description: "description 1",
DefaultSource: &proto.ParameterSource{
Scheme: proto.ParameterSource_DATA,
Value: defaultValue,
},
DefaultDestination: &proto.ParameterDestination{
Scheme: proto.ParameterDestination_PROVISIONER_VARIABLE,
},
},
{
AllowOverrideSource: true,
Name: "username",
Description: "description 2",
DefaultSource: &proto.ParameterSource{
Scheme: proto.ParameterSource_DATA,
// No default value
Value: "",
},
DefaultDestination: &proto.ParameterDestination{
Scheme: proto.ParameterDestination_PROVISIONER_VARIABLE,
},
},
},
},
},
}},
Parse: createTestParseResponseWithDefault(defaultValue),
Provision: echo.ProvisionComplete,
ProvisionDryRun: echo.ProvisionComplete,
})
Expand DownExpand Up@@ -178,4 +147,113 @@ func TestCreate(t *testing.T) {
}
<-doneChan
})

t.Run("WithParameterFileContainingTheValue", func(t *testing.T) {
t.Parallel()
client := coderdtest.New(t, &coderdtest.Options{IncludeProvisionerD: true})
user := coderdtest.CreateFirstUser(t, client)

defaultValue := "something"
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: createTestParseResponseWithDefault(defaultValue),
Provision: echo.ProvisionComplete,
ProvisionDryRun: echo.ProvisionComplete,
})

coderdtest.AwaitTemplateVersionJob(t, client, version.ID)
_ = coderdtest.CreateTemplate(t, client, user.OrganizationID, version.ID)
tempDir := t.TempDir()
parameterFile, _ := os.CreateTemp(tempDir, "testParameterFile*.yaml")
_, _ = parameterFile.WriteString("region: \"bingo\"\nusername: \"boingo\"")
cmd, root := clitest.New(t, "create", "", "--parameter-file", parameterFile.Name())
clitest.SetupConfig(t, client, root)
doneChan := make(chan struct{})
pty := ptytest.New(t)
cmd.SetIn(pty.Input())
cmd.SetOut(pty.Output())
go func() {
defer close(doneChan)
err := cmd.Execute()
require.NoError(t, err)
}()

matches := []string{
"Specify a name", "my-workspace",
"Confirm create?", "yes",
}
for i := 0; i < len(matches); i += 2 {
match := matches[i]
value := matches[i+1]
pty.ExpectMatch(match)
pty.WriteLine(value)
}
<-doneChan
removeTmpDirUntilSuccess(t, tempDir)
})
t.Run("WithParameterFileNotContainingTheValue", func(t *testing.T) {
t.Parallel()
client := coderdtest.New(t, &coderdtest.Options{IncludeProvisionerD: true})
user := coderdtest.CreateFirstUser(t, client)

defaultValue := "something"
version := coderdtest.CreateTemplateVersion(t, client, user.OrganizationID, &echo.Responses{
Parse: createTestParseResponseWithDefault(defaultValue),
Provision: echo.ProvisionComplete,
ProvisionDryRun: echo.ProvisionComplete,
})
coderdtest.AwaitTemplateVersionJob(t, client, version.ID)
template := coderdtest.CreateTemplate(t, client, user.OrganizationID, version.ID)
tempDir := t.TempDir()
parameterFile, _ := os.CreateTemp(tempDir, "testParameterFile*.yaml")
_, _ = parameterFile.WriteString("zone: \"bananas\"")
cmd, root := clitest.New(t, "create", "my-workspace", "--template", template.Name, "--parameter-file", parameterFile.Name())
clitest.SetupConfig(t, client, root)
doneChan := make(chan struct{})
pty := ptytest.New(t)
cmd.SetIn(pty.Input())
cmd.SetOut(pty.Output())
go func() {
defer close(doneChan)
err := cmd.Execute()
require.EqualError(t, err, "Parameter value absent in parameter file for \"region\"!")
}()
<-doneChan
removeTmpDirUntilSuccess(t, tempDir)
})
}

func createTestParseResponseWithDefault(defaultValue string) []*proto.Parse_Response {
return []*proto.Parse_Response{{
Type: &proto.Parse_Response_Complete{
Complete: &proto.Parse_Complete{
ParameterSchemas: []*proto.ParameterSchema{
{
AllowOverrideSource: true,
Name: "region",
Description: "description 1",
DefaultSource: &proto.ParameterSource{
Scheme: proto.ParameterSource_DATA,
Value: defaultValue,
},
DefaultDestination: &proto.ParameterDestination{
Scheme: proto.ParameterDestination_PROVISIONER_VARIABLE,
},
},
{
AllowOverrideSource: true,
Name: "username",
Description: "description 2",
DefaultSource: &proto.ParameterSource{
Scheme: proto.ParameterSource_DATA,
// No default value
Value: "",
},
DefaultDestination: &proto.ParameterDestination{
Scheme: proto.ParameterDestination_PROVISIONER_VARIABLE,
},
},
},
},
},
}}
}
56 changes: 56 additions & 0 deletionscli/parameter.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
package cli

import (
"os"

"golang.org/x/xerrors"
"gopkg.in/yaml.v3"

"github.com/coder/coder/cli/cliui"
"github.com/coder/coder/codersdk"
"github.com/spf13/cobra"
)

// Reads a YAML file and populates a string -> string map.
// Throws an error if the file name is empty.
func createParameterMapFromFile(parameterFile string) (map[string]string, error) {
if parameterFile != "" {
parameterMap := make(map[string]string)

parameterFileContents, err := os.ReadFile(parameterFile)

if err != nil {
return nil, err
}

err = yaml.Unmarshal(parameterFileContents, &parameterMap)

if err != nil {
return nil, err
}

return parameterMap, nil
}

return nil, xerrors.Errorf("Parameter file name is not specified")
}

// Returns a parameter value from a given map, if the map exists, else takes input from the user.
// Throws an error if the map exists but does not include a value for the parameter.
func getParameterValueFromMapOrInput(cmd *cobra.Command, parameterMap map[string]string, parameterSchema codersdk.ParameterSchema) (string, error) {
var parameterValue string
if parameterMap != nil {
var ok bool
parameterValue, ok = parameterMap[parameterSchema.Name]
if !ok {
return "", xerrors.Errorf("Parameter value absent in parameter file for %q!", parameterSchema.Name)
}
} else {
var err error
parameterValue, err = cliui.ParameterSchema(cmd, parameterSchema)
if err != nil {
return "", err
}
}
return parameterValue, nil
}
79 changes: 79 additions & 0 deletionscli/parameter_internal_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
package cli

import (
"os"
"runtime"
"testing"

"github.com/stretchr/testify/assert"
)

func TestCreateParameterMapFromFile(t *testing.T) {
t.Parallel()
t.Run("CreateParameterMapFromFile", func(t *testing.T) {
t.Parallel()
tempDir := t.TempDir()
parameterFile, _ := os.CreateTemp(tempDir, "testParameterFile*.yaml")
_, _ = parameterFile.WriteString("region: \"bananas\"\ndisk: \"20\"\n")

parameterMapFromFile, err := createParameterMapFromFile(parameterFile.Name())

expectedMap := map[string]string{
"region": "bananas",
"disk": "20",
}

assert.Equal(t, expectedMap, parameterMapFromFile)
assert.Nil(t, err)

removeTmpDirUntilSuccess(t, tempDir)
})
t.Run("WithEmptyFilename", func(t *testing.T) {
t.Parallel()

parameterMapFromFile, err := createParameterMapFromFile("")

assert.Nil(t, parameterMapFromFile)
assert.EqualError(t, err, "Parameter file name is not specified")
})
t.Run("WithInvalidFilename", func(t *testing.T) {
t.Parallel()

parameterMapFromFile, err := createParameterMapFromFile("invalidFile.yaml")

assert.Nil(t, parameterMapFromFile)

// On Unix based systems, it is: `open invalidFile.yaml: no such file or directory`
// On Windows, it is `open invalidFile.yaml: The system cannot find the file specified.`
if runtime.GOOS == "windows" {
assert.EqualError(t, err, "open invalidFile.yaml: The system cannot find the file specified.")
} else {
assert.EqualError(t, err, "open invalidFile.yaml: no such file or directory")
}
})
t.Run("WithInvalidYAML", func(t *testing.T) {
t.Parallel()
tempDir := t.TempDir()
parameterFile, _ := os.CreateTemp(tempDir, "testParameterFile*.yaml")
_, _ = parameterFile.WriteString("region = \"bananas\"\ndisk = \"20\"\n")

parameterMapFromFile, err := createParameterMapFromFile(parameterFile.Name())

assert.Nil(t, parameterMapFromFile)
assert.EqualError(t, err, "yaml: unmarshal errors:\n line 1: cannot unmarshal !!str `region ...` into map[string]string")

removeTmpDirUntilSuccess(t, tempDir)
})
}

// Need this for Windows because of a known issue with Go:
// https://github.com/golang/go/issues/52986
func removeTmpDirUntilSuccess(t *testing.T, tempDir string) {
t.Helper()
t.Cleanup(func() {
err := os.RemoveAll(tempDir)
for err != nil {
err = os.RemoveAll(tempDir)
}
})
}
Loading

[8]ページ先頭

©2009-2025 Movatter.jp