NotificationsYou must be signed in to change notification settings
Fork1k
Star11.1k

feat: validate presets on template import#18844

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Merged

SasSwart merged 5 commits intomainfromjjs/17333

Jul 30, 2025

Merged

feat: validate presets on template import#18844

SasSwart merged 5 commits intomainfromjjs/17333

Jul 30, 2025

Conversation

Copy link

Contributor

SasSwart commentedJul 14, 2025•
edited by coderabbitaibot
Loading

Typos and other errors often result in invalid presets in a template. Coder would import these broken templates and present them to users when they create workspaces. An unsuspecting user who chooses a broken preset would then experience a failed workspace build with no obvious error message.

This PR adds additional validation beyond what is possible in the Terraform provider schema. Coder will now present a more helpful error message to template authors when they upload a new template version:

The frontend warning is less helpful right now, but I'd like to address that in a follow-up since I need frontend help:

closes#17333

Summary by CodeRabbit

New Features
- Improved validation and error reporting for template presets, providing clearer feedback when presets cannot be parsed or reference undefined parameters.
Bug Fixes
- Enhanced error handling during template version creation to better detect and report issues with presets.
Tests
- Added new tests to verify validation of both valid and invalid Terraform presets during template version creation.
- Improved test reliability by enabling dynamic control over error injection in database-related tests.
Chores
- Updated a dependency to the latest version for improved stability and features.

feat: validate presets on template import

ac7af5e

github-actionsbot assignedSasSwart

Jul 14, 2025

remove replace directive and update preview dependency

945a675

SasSwart commented

Jul 14, 2025

View reviewed changes

coderd/templateversions.go OutdatedShow resolvedHide resolved

coderd/templateversions_test.goShow resolvedHide resolved

go.mod OutdatedShow resolvedHide resolved

SasSwart requested review fromjohnstcn,Emyrk andssncferreira

July 14, 2025 10:48

SasSwart marked this pull request as ready for review

July 14, 2025 10:51

Emyrk reviewed

Jul 14, 2025

View reviewed changes

coderd/templateversions.go OutdatedShow resolvedHide resolved

Copy link

ContributorAuthor

SasSwart commentedJul 14, 2025

Maybe we should only do preset validation if using dynamic parameters

A few customers and users are already running into preset validation issues and it breaks their workspaces. I'd prefer to get preset validation in for classic templates if we can.

ssncferreira approved these changes

Jul 15, 2025

View reviewed changes

Copy link

Contributor

ssncferreira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

LGTM 🎉
Just some small nits.

coderd/dynamicparameters/error.go OutdatedShow resolvedHide resolved

coderd/templateversions.go OutdatedShow resolvedHide resolved

coderd/templateversions_test.goShow resolvedHide resolved

coderd/templateversions.go OutdatedShow resolvedHide resolved

Copy link

Member

Emyrk commentedJul 15, 2025

A few customers and users are already running into preset validation issues and it breaks their workspaces. I'd prefer to get preset validation in for classic templates if we can.

@SasSwart I just fear a bug in preview could then break existing templates. We had some parsing bugs that prevented some templates from working.

johnstcn reviewed

Jul 20, 2025

View reviewed changes

coderd/parameters_test.go

Comment on lines 419 to 427

		rejectMu sync.RWMutex
		rejectbool
		}

		// SetReject toggles whether GetGitSSHKey should return an error or passthrough to the underlying store.
		func (d*dbRejectGitSSHKey)SetReject(rejectbool) {
		d.rejectMu.Lock()
		deferd.rejectMu.Unlock()
		d.reject=reject

Copy link

Member

johnstcnJul 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

nit: Is this a fix for a separate flake? If so, might be no harm to separate in its own PR.

coderd/templateversions.go OutdatedShow resolvedHide resolved

github-actionsbot added the staleThis issue is like stale bread. label

Jul 29, 2025

Copy link

coderabbitaibot commentedJul 29, 2025•
edited
Loading

Walkthrough

This change introduces backend validation and error handling for dynamic template version presets. It adds new functions for preset diagnostics, updates the control flow for template version creation to validate presets, and extends tests to cover valid and invalid preset scenarios. Dependency updates and minor test refactoring are also included.

Changes

Cohort / File(s)	Change Summary
Preset Validation Logic `coderd/dynamicparameters/error.go`,`coderd/dynamicparameters/presets.go`	Adds`presetValidationError` and`CheckPresets` functions to encapsulate and handle validation errors for dynamic template version presets.
Template Version Handling Refactor `coderd/templateversions.go`	Refactors dynamic template version creation: moves file extraction and preview logic into the main handler, updates tag parsing to use preview output, and integrates preset validation with error handling.
Preset Validation Testing `coderd/templateversions_test.go`	Adds subtests for valid and invalid preset scenarios, verifying error handling and correct API responses for template version creation with presets.
Test Helper Refactor `coderd/parameters_test.go`	Refactors the`dbRejectGitSSHKey` test helper to allow dynamic, thread-safe toggling of error injection during tests.
Dependency Update `go.mod`	Updates the`github.com/coder/preview` dependency to a newer version.

Sequence Diagram(s)

sequenceDiagram    participant Client    participant API    participant Preview    participant DB    Client->>API: POST /templateversions (with archive)    API->>API: Extract files from archive    API->>DB: Fetch workspace owner data    API->>Preview: Run preview on extracted files    Preview-->>API: Return preview output, diagnostics    API->>API: CheckPresets(preview output, diagnostics)    alt Preset validation error        API-->>Client: Return error response    else Preset validation ok        API->>API: dynamicTemplateVersionTags(preview output, diagnostics)        API-->>Client: Return tags or success response    end

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~15–20 minutes

Assessment against linked issues

Objective	Addressed	Explanation
Preset selection for radio/multi-select parameters is validated and errors surfaced (#17333)	✅
Proper error reporting when presets reference undefined parameters (#17333)	✅
No code changes related to UI/UX rendering of radio button selection (#17333)	❌	Backend changes only; UI state handling is not addressed in this PR.

Assessment against linked issues: Out-of-scope changes

Code Change	Explanation
Refactoring of`dbRejectGitSSHKey` for dynamic error injection (`coderd/parameters_test.go`)	This test helper refactor is unrelated to preset validation or radio button selection logic described in the linked issue.

Possibly related PRs

feat(cli)!: add CLI support for creating a workspace with preset #18912: Adds CLI support for specifying and applying presets during workspace creation, related to preset handling but focused on CLI rather than backend validation.

Suggested reviewers

SasSwart
Emyrk
johnstcn

Poem

A preset checked, a radio gleams,
Backend logic fuels our dreams.
Errors caught, diagnostics tight,
Now templates validate just right!
With every test and code refactor,
This bunny hops, a happy actor.
🐇✨

Warning

Review ran into problems

🔥 Problems

response.data.map is not a function

✨ Finishing Touches

📝 Generate Docstrings

🧪 Generate unit tests

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branchjjs/17333

🪧 Tips

Chat

There are 3 ways to chat withCodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Explain this complex logic.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag@coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai explain this code block.
- @coderabbitai modularize this function.
PR comments: Tag@coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and explain its main purpose.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on oursupport page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings togenerate docstrings for this PR.
@coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
@coderabbitai generate unit tests to generate unit tests for this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add@coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add@coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add@coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a.coderabbit.yaml file to the root of your repository.
Please see theconfiguration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation:# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit ourDocumentation for detailed information on how to use CodeRabbit.
Join ourDiscord Community to get help, request features, and share feedback.
Follow us onX/Twitter for updates and announcements.

coderabbitaibot reviewed

Jul 29, 2025

View reviewed changes

Copy link

coderabbitaibot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (2)

coderd/dynamicparameters/error.go (1)
31-31:Consider updating the error message as suggested in the previous review.
The message "Unable to parse presets" should be "Unable to validate presets" to better reflect the function's purpose and align with the validation context.
go.mod (1)
486-486:Verify that the prerequisite PR has been merged.
Based on the previous comment, this PR should not be merged untilcoder/preview#149 has been merged and the dependency updated. Please confirm that this prerequisite has been satisfied.

🧹 Nitpick comments (1)

coderd/templateversions_test.go (1)
645-755:Well-structured preset validation test.
The test correctly follows the parallel testing pattern, uses unique identifiers, and covers both valid and invalid preset scenarios appropriately. The structure aligns well with other tests in the file.
[past_review_comments]
Regarding the past comment about checking if no provisioner job was created for the invalid case - this would be a good addition to ensure the validation fails before job creation rather than during job execution.
Consider adding a check that no provisioner job is created for the invalid preset case:
 } else {     require.ErrorContains(t, err, tt.expectError)+    // Verify no provisioner job was created for invalid preset+    _, err := store.GetProvisionerJobByID(ctx, tv.Job.ID)+    require.Error(t, err, "no job should be created for invalid preset") }

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between3126f21 and945a675.

⛔ Files ignored due to path filters (1)

go.sum is excluded by!**/*.sum

📒 Files selected for processing (6)

coderd/dynamicparameters/error.go (1 hunks)
coderd/dynamicparameters/presets.go (1 hunks)
coderd/parameters_test.go (3 hunks)
coderd/templateversions.go (3 hunks)
coderd/templateversions_test.go (1 hunks)
go.mod (1 hunks)

🧰 Additional context used

📓 Path-based instructions (3)

**/*.go

📄 CodeRabbit Inference Engine (.cursorrules)

**/*.go: The codebase is rigorously linted with golangci-lint to maintain consistent code quality.
Coder emphasizes clear error handling, with specific patterns required: Concise error messages that avoid phrases like "failed to"; Wrapping errors with%w to maintain error chains; Using sentinel errors with the "err" prefix (e.g.,errNotFound).
**/*.go: NEVER use time.Sleep to mitigate timing issues. If an issue seems like it should use time.Sleep, read throughhttps://github.com/coder/quartz and specifically the README to better understand how to handle timing issues.
Return OAuth2-compliant error responses using writeOAuth2Error(ctx, rw, http.StatusBadRequest, "invalid_grant", "description") for OAuth2 errors.
Follow Uber Go Style Guide for Go code style.
Use dbauthz.AsSystemRestricted(ctx) for public endpoints needing system access and plain ctx for authenticated endpoints with user context.

Files:

coderd/dynamicparameters/error.go
coderd/dynamicparameters/presets.go
coderd/parameters_test.go
coderd/templateversions_test.go
coderd/templateversions.go

**/*.{go,sql,ts,tsx,js,jsx,md}

📄 CodeRabbit Inference Engine (CLAUDE.md)

Ensure files end with a newline to avoid missing newlines.

Files:

coderd/dynamicparameters/error.go
coderd/dynamicparameters/presets.go
coderd/parameters_test.go
coderd/templateversions_test.go
coderd/templateversions.go

**/*_test.go

📄 CodeRabbit Inference Engine (.cursorrules)

**/*_test.go: All tests must uset.Parallel() to run concurrently, which improves test suite performance and helps identify race conditions.
All tests should run in parallel usingt.Parallel() to ensure efficient testing and expose potential race conditions.
Use unique identifiers (e.g., fmt.Sprintf("test-client-%s-%d", t.Name(), time.Now().UnixNano())) and never use hardcoded names in concurrent tests to prevent race conditions.

Files:

coderd/parameters_test.go
coderd/templateversions_test.go

🧠 Learnings (4)

go.mod (2)

Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to **/*.go : Coder emphasizes clear error handling, with specific patterns required: Concise error messages that avoid phrases like "failed to"; Wrapping errors with%w to maintain error chains; Using sentinel errors with the "err" prefix (e.g.,errNotFound).

Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to **/*.go : The codebase is rigorously linted with golangci-lint to maintain consistent code quality.

coderd/parameters_test.go (5)

Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:07.777Z
Learning: Applies to **/*_test.go : Use unique identifiers (e.g., fmt.Sprintf("test-client-%s-%d", t.Name(), time.Now().UnixNano())) and never use hardcoded names in concurrent tests to prevent race conditions.

Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:10.063Z
Learning: Applies to **/*_test.go : Use unique identifiers (e.g., fmt.Sprintf("test-client-%s-%d", t.Name(), time.Now().UnixNano())) and never use hardcoded names in concurrent tests to prevent race conditions.

Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:07.155Z
Learning: Applies to **/*_test.go : Use unique identifiers (e.g., fmt.Sprintf("test-client-%s-%d", t.Name(), time.Now().UnixNano())) and never use hardcoded names in concurrent tests to prevent race conditions.

Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to **/*_test.go : All tests must uset.Parallel() to run concurrently, which improves test suite performance and helps identify race conditions.

Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to **/*_test.go : All tests should run in parallel usingt.Parallel() to ensure efficient testing and expose potential race conditions.

coderd/templateversions_test.go (7)

Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to coderd/coderdtest/**/* : Thecoderdtest package incoderd/coderdtest/ provides utilities for creating test instances of the Coder server, setting up test users and workspaces, and mocking external components.

Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to coderdenttest/**/* : Enterprise features have dedicated test utilities in thecoderdenttest package.

coderd/templateversions.go (4)

Learnt from: CR
PR: coder/coder#0
File: .cursorrules:0-0
Timestamp: 2025-07-21T14:32:43.064Z
Learning: Applies to coderd/coderd.go : The REST API is defined incoderd/coderd.go and uses Chi for HTTP routing.

Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:07.777Z
Learning: Applies to **/*.go : Return OAuth2-compliant error responses using writeOAuth2Error(ctx, rw, http.StatusBadRequest, "invalid_grant", "description") for OAuth2 errors.

Learnt from: CR
PR: coder/coder#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-28T00:33:10.063Z
Learning: Applies to **/*.go : Return OAuth2-compliant error responses using writeOAuth2Error(ctx, rw, http.StatusBadRequest, "invalid_grant", "description") for OAuth2 errors.

🔇 Additional comments (9)

coderd/dynamicparameters/presets.go (1)
13-24:LGTM! Clean and well-structured preset validation logic.
The function correctly aggregates preset-specific diagnostics and follows the established error handling patterns in the codebase. The logic is clear and handles the error aggregation appropriately.
coderd/parameters_test.go (4)
6-6:LGTM: Required import for mutex usage.
Thesync import is needed for the new mutex-based error injection control.
419-428:Excellent improvement to test error injection.
The addition of mutex-protected error control significantly improves test flexibility by allowing dynamic toggling of error behavior instead of always failing. The thread-safety implementation is correct using RWMutex for efficient concurrent access.
[past_review_comments]
Regarding the past comment about this being a separate flake fix - this change does improve test determinism and could help with flakiness by providing more controlled error injection.
430-440:Clean implementation of conditional error injection.
The method correctly uses RLock for reading the shared state and properly delegates to the underlying store when not rejecting. The mutex usage pattern is optimal - releasing the lock before the potentially slow database call.
197-197:Proper usage of the improved error injection mechanism.
The test correctly creates the wrapper, passes it to setup, and enables error injection at the appropriate time. This allows initial setup to succeed while still testing error handling in the target scenario.
Also applies to: 205-205, 212-213
coderd/templateversions.go (4)
19-19:LGTM! Import addition is necessary.
Thehcl/v2 import is required for thehcl.Diagnostics type used in the refactored function signature.
1586-1637:LGTM! Well-structured file processing and preset validation.
The refactoring centralizes file extraction and preview logic effectively. Key strengths:
Comprehensive error handling for unsupported file types and extraction failures
Proper preset validation that addresses the PR objective of preventing broken workspace builds
Clean separation of concerns by moving shared logic to the main handler
Follows coding guidelines with concise error messages and proper error handling
The preset validation usingdynamicparameters.CheckPresets will catch invalid presets early and provide clear error messages, which aligns perfectly with the PR objectives.
1642-1642:Function call updated correctly for refactored signature.
The call todynamicTemplateVersionTags now passes the processed preview results instead of raw file data, which is consistent with the refactoring that moved file processing logic to the main handler.
1819-1819:Excellent refactoring that improves separation of concerns.
The function signature change eliminates duplicate file processing logic and creates a cleaner separation where the main handler manages file extraction while this function focuses solely on tag validation from preview results. This addresses the concerns raised in past review comments about extracting this logic.