We keep many large tables, such as audit logs, API keys, connection logs, and now Bridge's request logs for forensic purposes, meaning that, in the case of an incident, an admin can find all of the details necessary for an event.

However, many customers prefer to use an external data source/SIEM (e.g. Splunk, Datadog) as their source of truth and want to keep the Coder PostgreSQL database relatively small/lightweight. This has several advantages:

• Data centralization (logs are centralized across tools, not just Coder)
• Low database cost (keep PostgreSQL disk/resources low)
• Improved day-to-day database performance (queries by Coder perform better rather than going across all rows)
• Improved migration performance (when upgrading Coder, migrations do not have to go through many sources)
• PostgreSQL typically isn't technically fantastic at holding these types of event-based data indefinitely and there are better options.

In fact, our documentation actually mentions how to purge audit logs:https://coder.com/docs/admin/security/audit-logs#purging-old-audit-logs

Definition of done

• Brief scan of tables to ensure there are not other obvious culprits for a retention policy
• The following tables have a configurable retention policy via a Coder server flag. Each one has an individual policy
  • Connection Logs
  • Audit Logs
  • API Keys
  • Bridge Request Logs
  • Nice to have: A global policy can also be applied to all of these
• The retention policy is disabled by default to prevent breaking changes for customers. Down the road, we can consider changing this to a default value but we should warn customers and provide docs on how they can keep it disabled.
• dbpurge is modified to purge these based on the retention policy
  • API keys are only purged if they are past the policy and the key has expired
• Our docs, changelog, and best practices doc recommend setting a 90-day retention policy as well as mention how these various logs can be ingested, similar to audit logs:https://coder.com/docs/admin/security/audit-logs#capturingexporting-audit-logs