No health surface a monitoring probe can reach without the admin Host #67

New issue

Open

opened 2026-10-26 17:15:49 +00:00 by james.oates · 1 comment

james.oates commented

2026-10-26 17:15:49 +00:00

Owner

Raised by Roku while reviewing v2.0.0 for a greenfield deploy.

/health lives in session_routes and /ready in admin_routes, and both routers sit behind admin_host_gate (src/main.rs:106). So a probe must send Host: <configured admin host> to reach either, and with no [[admin]] entry neither is served at all — every request falls through to the reverse proxy.

/health was introduced as a liveness probe needing no key, which it is; what it also needs, and does not have, is a way to be addressed without knowing and spoofing the admin hostname. A container runtime health check, a systemd watchdog or a node-level monitor all want to hit 127.0.0.1:<port>/health, and that currently returns whatever the proxy fallback decides.

Workaround today: configure an [[admin]] entry and have the probe set the Host header explicitly. That works but couples liveness monitoring to admin configuration, and a deployment with no admin host has no health surface at all.

Options

A loopback-only health listener — a separate socket bound to 127.0.0.1/::1 serving /health and optionally /ready, independent of [[admin]]. Cleanest for the probe use case, and loopback is its own access control.
Exempt /health from the admin Host gate and serve it on any binding. Simplest change, but it becomes a public endpoint on every listener, which is an information disclosure decision (it reveals a process is alive and which software it is) rather than an obvious yes.
Make a binding able to opt into serving health, so an operator puts it on an internal listener. Most flexible, most configuration.

Option 1 is probably right, and it pairs with the fact that there is currently no way to reach the API on loopback at all without naming an admin host.

Not for v2.0.0 — this is a feature rather than a defect fix. Filed so the gap is recorded against the release rather than discovered per deployment; the release notes name it as a known limit.

Raised by Roku while reviewing v2.0.0 for a greenfield deploy. `/health` lives in `session_routes` and `/ready` in `admin_routes`, and both routers sit behind `admin_host_gate` (`src/main.rs:106`). So a probe must send `Host: <configured admin host>` to reach either, and with **no** `[[admin]]` entry neither is served at all — every request falls through to the reverse proxy. `/health` was introduced as a liveness probe needing no key, which it is; what it also needs, and does not have, is a way to be addressed without knowing and spoofing the admin hostname. A container runtime health check, a systemd watchdog or a node-level monitor all want to hit `127.0.0.1:<port>/health`, and that currently returns whatever the proxy fallback decides. Workaround today: configure an `[[admin]]` entry and have the probe set the Host header explicitly. That works but couples liveness monitoring to admin configuration, and a deployment with no admin host has no health surface at all. ## Options 1. **A loopback-only health listener** — a separate socket bound to `127.0.0.1`/`::1` serving `/health` and optionally `/ready`, independent of `[[admin]]`. Cleanest for the probe use case, and loopback is its own access control. 2. **Exempt `/health` from the admin Host gate** and serve it on any binding. Simplest change, but it becomes a public endpoint on every listener, which is an information disclosure decision (it reveals a process is alive and which software it is) rather than an obvious yes. 3. **Make a binding able to opt into serving health**, so an operator puts it on an internal listener. Most flexible, most configuration. Option 1 is probably right, and it pairs with the fact that there is currently no way to reach the API on loopback at all without naming an admin host. Not for v2.0.0 — this is a feature rather than a defect fix. Filed so the gap is recorded against the release rather than discovered per deployment; the release notes name it as a known limit.

james.oates added this to the v2.1 milestone

2026-10-26 17:53:15 +00:00

james.oates referenced this issue

2026-10-26 17:53:55 +00:00

The WebSocket path has no connect timeout, no idle timeout and no connection cap #62

james.oates referenced this issue

2026-10-27 09:43:38 +00:00

Apply the HTTP→HTTPS redirect ahead of route matching, exempting the liveness routes #70

james.oates commented

2026-10-27 10:15:34 +00:00

Author

Owner

Taken out of v2.1 and left open. Decided by the Axianthe PM on the grounds that this guards a use case nobody has: James has rejected monitoring, no host needs a health surface today, and nothing has hung.

Checked before accepting that, since the issue describes container runtime health checks rather than monitoring alone: devinfra's compose defines healthcheck blocks for postgres, mongodb and others, and none for edge-router. So nothing in the fleet currently probes /health, and this is not latent breakage.

One caveat for whoever picks this up: the absence of an ER healthcheck is weak evidence of no need, because the gap described here is a plausible reason nobody wrote one. If a container healthcheck or a systemd watchdog is ever wanted for Edge Router, this issue is the blocker to clear first, and option 1 (a loopback-only health socket) remains the right shape — loopback is its own access control, it works inside a container where a Docker HEALTHCHECK runs, and it needs no [[admin]] entry.

It does not serve an external monitor on another host; that case is the admin host plus an access rule, which is correct.

Also relevant to #70: whatever shape this eventually takes decides what an app-wide HTTP→HTTPS redirect has to exempt. The two should be settled together rather than separately.

**Taken out of v2.1 and left open.** Decided by the Axianthe PM on the grounds that this guards a use case nobody has: James has rejected monitoring, no host needs a health surface today, and nothing has hung. Checked before accepting that, since the issue describes container runtime health checks rather than monitoring alone: devinfra's compose defines `healthcheck` blocks for postgres, mongodb and others, and **none for `edge-router`**. So nothing in the fleet currently probes `/health`, and this is not latent breakage. One caveat for whoever picks this up: the absence of an ER healthcheck is weak evidence of no need, because the gap described here is a plausible *reason* nobody wrote one. If a container healthcheck or a systemd watchdog is ever wanted for Edge Router, this issue is the blocker to clear first, and option 1 (a loopback-only health socket) remains the right shape — loopback is its own access control, it works inside a container where a Docker `HEALTHCHECK` runs, and it needs no `[[admin]]` entry. It does **not** serve an external monitor on another host; that case is the admin host plus an access rule, which is correct. Also relevant to #70: whatever shape this eventually takes decides what an app-wide HTTP→HTTPS redirect has to exempt. The two should be settled together rather than separately.

james.oates removed this from the v2.1 milestone

2026-10-27 10:15:45 +00:00

james.oates referenced this issue

2026-10-27 15:12:16 +00:00

Apply the HTTP→HTTPS redirect ahead of route matching, exempting the liveness routes #70