Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

fix(cli): port-forward: update workspace last_used_at#12659

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
johnstcn merged 21 commits intomainfromcj/cli-workspace-heartbeat
Mar 20, 2024

Conversation

johnstcn
Copy link
Member

@johnstcnjohnstcn commentedMar 19, 2024
edited
Loading

Fixes#12431

For the CLI to report workspace usage, we currently depend on the agent sending a stats report with a non-zero connection count. This is sourced fromSetConnStatsCallback (tailnet/conn.go:621).
The problem with this approach is that the tailscale connection tracking only appears to track "connection established" events, and is not equivalent to running e.g.ss -plunt. This means:

  • Having an open port-forward on its own will not be detected as an 'active connection'
  • Having an open port-forward with a single long-lived connection (e.g. websocket conn) will only be counted as one activity datapoint, and subsequent stats collection intervals will ignore that active connection.

This PR updates thecoder port-forward command to periodically inform coderd that the workspace is being used:

  • Addsworkspaceusage.Tracker which periodically batch-updates workspace LastUsedAt
  • Adds coderd endpoint to signal workspace usage
  • Updatescoder port-forward to periodically hit this endpoint
  • ModifiesBatchUpdateWorkspacesLastUsedAt to avoid overwriting with stale data

In later PRs, we can:

  • Updatecoder ssh to also use this behaviour,
  • Update the workspace activity tracking package to also callActivityBumpWorkspace so thatlast_used_at anddeadline are handled in the same place,
  • Remove the existing behaviour of updating the workspace when stats are received from the agent.

Follow-ups:

  • We may need to ensure that multiple port-forwards to the same workspace only result in onecoder process on a user's workstation updating the workspace at a time. My assumption here is that most users will have at most one or two port-forwards running at a time.

@johnstcnjohnstcn self-assigned thisMar 19, 2024
@johnstcnjohnstcn changed the titlefeat: tunnel: update workspace last_used_atfeat(cli): port-forward: update workspace last_used_atMar 19, 2024
@johnstcnjohnstcnforce-pushed thecj/cli-workspace-heartbeat branch 2 times, most recently from17cb933 to2412fb5CompareMarch 19, 2024 13:20
@johnstcnjohnstcn changed the titlefeat(cli): port-forward: update workspace last_used_atfix(cli): port-forward: update workspace last_used_atMar 19, 2024
@johnstcnjohnstcnforce-pushed thecj/cli-workspace-heartbeat branch from2412fb5 toc99327cCompareMarch 19, 2024 15:16
@johnstcnjohnstcn marked this pull request as ready for reviewMarch 19, 2024 15:32
Copy link
Contributor

@dannykoppingdannykopping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Mostly nits & small {que,sugge}stions
Very clean! LGTM

johnstcnand others added2 commitsMarch 20, 2024 09:10
Co-authored-by: Danny Kopping <danny@coder.com>
Copy link
Member

@mtojekmtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

High-level review first

LastUsedAt: now,
IDs: ids,
}); err != nil {
wut.log.Error(ctx, "failed updating workspaces last_used_at", slog.F("count", count), slog.Error(err))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Is there any retry mechanism if the batch fails?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

There is a built-in assumption here that any failure of a single batch can be ignored as it gets retried frequently. I suppose consecutive failures should surface some form of error somewhere. I wonder if the database health page could show a summary of queries that have failed multiple consecutive executions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

There is a built-in assumption here that any failure of a single batch can be ignored as it gets retried frequently. I suppose consecutive failures should surface some form of error somewhere

I might have missed it, but it would be cool to leave a comment stating this.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Done

Copy link
Member

@mtojekmtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Nice!

Copy link
Contributor

@dannykoppingdannykopping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Awesome 👍

db, ps = dbtestutil.NewDB(t)
wuTick = make(chan time.Time)
wuFlush = make(chan int, 1)
wut = workspaceusage.New(db, workspaceusage.WithFlushChannel(wuFlush), workspaceusage.WithTickChannel(wuTick))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

jiffy

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

wat

// the number of marked workspaces every time Tracker flushes.
// For testing only and will panic if used outside of tests.
func WithFlushChannel(c chan int) Option {
if flag.Lookup("test.v") == nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Nice strictness check! Feels like this could be a testutil func, even if obvious.

// Calling Loop after Close() is an error.
select {
case <-wut.doneCh:
panic("developer error: Loop called after Close")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

If we need to do this instead of just returning, I'd prefer returning an error here, wouldn't want this to happen for a production instance after some changes to the code.

Feels like this mostly exists for tests (otherwise could be launched fromNew), so more reason not to add a possibility of panic.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

If we return an error, then we have two options:

  1. Exit with the error, which brings us back to potentially impacting prod, but just in a nicer-looking way.
  2. Just log the error and do nothing, which means the error will likely pass unnoticed until folks wonder why their workspace usage isn't being tracked.

I don't think 2) is a valid option. We're not guaranteed to catch it in tests especially because of the possibility of ignoring errors inslogtest.

There's an alternative possibility 1a) where we check the error and recreate the tracker if non-nil. WDYT?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I forgot about option c) - haveNew start the loop and unexport the function.
Now it's un-possible!

Emyrk reacted with thumbs up emoji
Comment on lines 312 to 329
// Did last_used_at not update? Scratching your noggin? Here's why.
// Workspace usage tracking must be triggered manually in tests.
// The vast majority of existing tests do not depend on last_used_at
// and adding an extra background goroutine to all existing tests may
// lead to future flakes and goleak complaints.
// To do this, pass in your own WorkspaceUsageTracker like so:
//
// db, ps = dbtestutil.NewDB(t)
// wuTick = make(chan time.Time)
// wuFlush = make(chan int, 1)
// wut = workspaceusage.New(db, workspaceusage.WithFlushChannel(wuFlush), workspaceusage.WithTickChannel(wuTick))
// client = coderdtest.New(t, &coderdtest.Options{
// WorkspaceUsageTracker: wut,
// Database: db,
// Pubsub: ps,
// })
//
// See TestPortForward for how this works in practice.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Lol, love the personality in this comment 😄

@johnstcnjohnstcn merged commit92aa1eb intomainMar 20, 2024
@johnstcnjohnstcn deleted the cj/cli-workspace-heartbeat branchMarch 20, 2024 16:44
@github-actionsgithub-actionsbot locked and limited conversation to collaboratorsMar 20, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account?Sign in.
Reviewers

@mafredrimafredrimafredri left review comments

@mtojekmtojekmtojek approved these changes

@dannykoppingdannykoppingdannykopping approved these changes

@EmyrkEmyrkEmyrk approved these changes

Assignees

@johnstcnjohnstcn

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

coder tunnel does not update workspacelast_used with long-lived connections
5 participants
@johnstcn@mafredri@dannykopping@Emyrk@mtojek

[8]ページ先頭

©2009-2025 Movatter.jp