Commit Graph

6 Commits

Author SHA1 Message Date
Alex Vandiver
f91bbee395 prometheus: Move weblate metrics to pushgateway.
We leave the weblate metrics server and add a cron job to fetch its
contents and post them to the pushgateway.
2025-10-22 11:57:02 -07:00
Alex Vandiver
25d156cc9c prometheus: Add a pushgateway.
Prometheus declares metrics to be "stale" if they go 5min without an
update[^1], and removes them -- which makes longer scrape intervals
impossible.

Add a pushgateway, which caches output from such jobs.

[^1]: https://prometheus.io/docs/prometheus/latest/querying/basics/#staleness
2025-10-22 11:57:02 -07:00
Alex Vandiver
82b40ac6e2 puppet: Add an exporter to fetch Weblate stats, for graphing.
This is done synchronously, despite taking ~60s.  We can move it to a
background thread later if that's an issue, but generally Prometheus
is tolerant to exporters taking a while to come back with results.
2025-08-14 13:18:56 -07:00
Alex Vandiver
b11cbbab01 smokescreen: Move metrics port from the default 9810, to 4760.
This prevents errors if Smokescreen is running on a host with more
than 10 Tornado shards.
2025-04-11 10:29:26 -07:00
Alex Vandiver
840fa74854 kandra: Add a grok exporter to parse nginx logfiles.
This provides access logging metrics to Prometheus.  For cardinality
reasons, we cannot (nor would we want to) put every request path into
its own label value -- but we do separate out the most-frequent access
paths (as well as some low-frequency but high-interest ones) into
their own label values.

In order to differentiate accesses to https://zulip.com/ from
https://example.zulipchat.com/ (both of which appear at path `/`), we
use a `grok_exporter.realm_names_regex` value in `zulip.conf`, which
is expected to be set to match the hostname of all possible realms.
2025-03-27 21:56:54 -07:00
Alex Vandiver
bd54f0363e kandra: Update prometheus configuration.
This pulls in the more complete production Prometheus configuration.
2025-03-27 21:56:54 -07:00