Problem statement

Users commonly suffer from crashed/disconnected workspaces due to resource constraints on the agent. The most common of these issues is OOM (out-of-memory) / OOD (out-of-disk) errors that disconnect users while they're working. Users want to know ahead of time when they are running up against the limits of their workspace's allocated resources.

Both users and administrators want to diagnose whether the agent was killed due to OOM/OODafter crashes occur.

Proposal

To remedy this, we're adding a native OOM/OOD notification for workspaces as part of a global initiative to improve the notifications feature.

This feature - in a few words - allow users to set threshold and receive notifications is either the Memory or Disk resource of their workspace is reached.

In term of modifications, it requires :

coder/internal#246

• Some changes on the Agent to fetch the data
• Update of the AgentAPI to push the fetched data to the CP

coder/internal#245

• Update of templates to include the thresholds for both memory and disk.
• Have the threshold stored in the DB and retrievable.

coder/internal#248

• Create a piece of logic processing the data points sent by the Agents
• Include a debouncing logic in it to avoid spamming notifications

coder/internal#247

• Add a migration in the notifiers to send this new kind of email
• Modify both the FE and BE to have a new event configurable underWorkspace events.