Title: Add support for retention policy on large tables · Issue #20743 · coder/coder · GitHub
Open Graph Title: Add support for retention policy on large tables · Issue #20743 · coder/coder
X Title: Add support for retention policy on large tables · Issue #20743 · coder/coder
Description: We keep many large tables, such as audit logs, API keys, connection logs, and now Bridge's request logs for forensic purposes, meaning that, in the case of an incident, an admin can find all of the details necessary for an event. However...
Open Graph Description: We keep many large tables, such as audit logs, API keys, connection logs, and now Bridge's request logs for forensic purposes, meaning that, in the case of an incident, an admin can find all of the...
X Description: We keep many large tables, such as audit logs, API keys, connection logs, and now Bridge's request logs for forensic purposes, meaning that, in the case of an incident, an admin can find all of...
Opengraph URL: https://github.com/coder/coder/issues/20743
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Add support for retention policy on large tables","articleBody":"We keep many large tables, such as audit logs, API keys, connection logs, and now Bridge's request logs for forensic purposes, meaning that, in the case of an incident, an admin can find all of the details necessary for an event.\n\nHowever, many customers prefer to use an external data source/SIEM (e.g. Splunk, Datadog) as their source of truth and want to keep the Coder PostgreSQL database relatively small/lightweight. This has several advantages:\n\n- Data centralization (logs are centralized across tools, not just Coder)\n- Low database cost (keep PostgreSQL disk/resources low)\n- Improved day-to-day database performance (queries by Coder perform better rather than going across all rows)\n- Improved migration performance (when upgrading Coder, migrations do not have to go through many sources)\n- PostgreSQL typically isn't technically fantastic at holding these types of event-based data indefinitely and there are better options.\n\nIn fact, our documentation actually mentions how to purge audit logs: https://coder.com/docs/admin/security/audit-logs#purging-old-audit-logs\n\n> [!NOTE]\n> For full context, check out [our most recent R&D reto](https://www.notion.so/coderhq/Retros-142c7591854041dcac6707538e07898e?p=2a8d579be592806282c3d4de78d98cfb&pm=s)\n\n## Definition of done\n\n- [x] Brief scan of tables to ensure there are not other obvious culprits for a retention policy\n- [x] The following tables have a configurable retention policy via a Coder server flag. Each one has an individual policy\n - [x] Connection Logs\n - [x] Audit Logs\n - [x] API Keys\n - [x] ~~Bridge Request Logs~~ AI Bridge docs updates #21055\n - [ ] Nice to have: A global policy can also be applied to all of these\n- [x] The retention policy is disabled by default to prevent breaking changes for customers. Down the road, we can consider changing this to a default value but we should warn customers and provide docs on how they can keep it disabled.\n- [x] dbpurge is modified to purge these based on the retention policy\n - [x] API keys are only purged if they are past the policy and the key has expired\n- [x] Our docs, changelog, and best practices doc recommend setting a 90-day retention policy as well as mention how these various logs can be ingested, similar to audit logs: https://coder.com/docs/admin/security/audit-logs#capturingexporting-audit-logs","author":{"url":"https://github.com/bpmct","@type":"Person","name":"bpmct"},"datePublished":"2025-11-12T17:10:25Z","url":"https://github.com/coder/coder/issues/20743"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:3925372b-96ec-ae30-57d2-3ae8db35030e |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | E41C:1201C:1A35F07:22D8D93:6B015077 |
| html-safe-nonce | 7aa44cdf4abea59f42c35b71bf74e9b790d9a6127ee424ce1fab370191c37693 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFNDFDOjEyMDFDOjFBMzVGMDc6MjJEOEQ5Mzo2QjAxNTA3NyIsInZpc2l0b3JfaWQiOiI2NDM0NDQ2NjM4NTgzOTkyNDM5IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 3c45e677736c062ecf331008c3b13290cb26e9a66cac2f50691fedd9182d5e9e |
| hovercard-subject-tag | issue:3617458562 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/coder/coder/20743/issue_layout |
| twitter:image | https://opengraph.githubassets.com/ec89cccbbf3fc8f87635c5867e52c433e0912748cd125e5c89983399ff4f6bd8/coder/coder/issues/20743 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/ec89cccbbf3fc8f87635c5867e52c433e0912748cd125e5c89983399ff4f6bd8/coder/coder/issues/20743 |
| og:image:alt | We keep many large tables, such as audit logs, API keys, connection logs, and now Bridge's request logs for forensic purposes, meaning that, in the case of an incident, an admin can find all of the... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | bpmct |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| go-import | github.com/coder/coder git https://github.com/coder/coder.git |
| octolytics-dimension-user_id | 95932066 |
| octolytics-dimension-user_login | coder |
| octolytics-dimension-repository_id | 440752086 |
| octolytics-dimension-repository_nwo | coder/coder |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 440752086 |
| octolytics-dimension-repository_network_root_nwo | coder/coder |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width