One unreachable dependency freezes all TOML config reload, not just TLS #39

New issue

Open

opened 2026-10-20 14:03:15 +00:00 by james.oates · 0 comments

james.oates commented

2026-10-20 14:03:15 +00:00

Owner

health.rs:497, inside reload_routes_config:

for tls_cfg in &route_config.tls_domains {
    let token_enc = crypto.encrypt(&tls_cfg.cloudflare_api_token).await?;

The ? is on the whole function, so with ENCRYPTION_ENGINE=openbao and
OpenBao unreachable or sealed, reload_routes_config returns early at the first
[[tls]] entry. Everything the function reconciles after that point stops
being applied:

[[l4_routes]] and their access_rules
[[static_sites]]
[[admin]] entries (the only refresh path for them — they are TOML-only with
no DB table)

Services are reconciled before the TLS block so they survive, but the ordering
is incidental, not a design.

Net effect: a sealed vault silently freezes the config watcher for resources
that have no dependency on it. An operator editing edge-router.toml to fix
something unrelated sees no change and no error on the proxy's own surface —
the failure is one tracing::error! in the watcher loop.

Fix: handle the encrypt failure per-domain (skip that domain, log it, continue)
rather than aborting the reconciliation. This is the same per-resource principle
as specs/2026-10-20-runtime-resilience-and-error-attribution-design.md,
Part 1 — a dependency failure should scope to the resources that actually need
the dependency.

`health.rs:497`, inside `reload_routes_config`: ```rust for tls_cfg in &route_config.tls_domains { let token_enc = crypto.encrypt(&tls_cfg.cloudflare_api_token).await?; ``` The `?` is on the whole function, so with `ENCRYPTION_ENGINE=openbao` and OpenBao unreachable or sealed, `reload_routes_config` returns early at the first `[[tls]]` entry. Everything the function reconciles *after* that point stops being applied: - `[[l4_routes]]` and their `access_rules` - `[[static_sites]]` - `[[admin]]` entries (the only refresh path for them — they are TOML-only with no DB table) Services are reconciled before the TLS block so they survive, but the ordering is incidental, not a design. Net effect: a sealed vault silently freezes the config watcher for resources that have no dependency on it. An operator editing `edge-router.toml` to fix something unrelated sees no change and no error on the proxy's own surface — the failure is one `tracing::error!` in the watcher loop. Fix: handle the encrypt failure per-domain (skip that domain, log it, continue) rather than aborting the reconciliation. This is the same per-resource principle as `specs/2026-10-20-runtime-resilience-and-error-attribution-design.md`, Part 1 — a dependency failure should scope to the resources that actually need the dependency.

james.oates referenced this issue

2026-10-20 15:03:23 +00:00

Runtime resilience: survive OpenBao being unavailable #40

james.oates referenced this issue

2026-10-22 16:26:09 +00:00

Runtime resilience: survive OpenBao being unavailable #40

james.oates referenced this issue from a commit

2026-10-25 15:32:56 +00:00

feat!: one certificate per name, derived from the route table

james.oates referenced this issue

2026-10-26 13:36:23 +00:00

Runtime resilience: survive OpenBao being unavailable #40