scripts/metricsdocgen generates thislist of available metrics. However, it relies on a static dump of metrics fromscripts/metricsdocgen/metrics; this means that new metrics are not automatically detected.

We can generate this list of metrics more reliably by callingCODER_PROMETHEUS_ENABLE=true coder server and add some kind of flag to force a dump of metrics like so:

--- a/cli/server.go+++ b/cli/server.go@@ -1016,6 +1016,8 @@ func (r *RootCmd) Server(newAPI func(context.Context, *coderd.Options) (*coderd.                                }                        }+                       prometheus.WriteToTextfile("./scripts/metricsdocgen/metrics", promRegistry)+                        eg := errgroup.Group{}                        eg.Go(func() error {                                defer closeListenersNow()

This won't catch any metrics for which there have not been any collected values yet, however, but it's a step closer.

@mtojek what do you think?