Admin host is excluded from wildcard certificate matching, so it refuses the handshake #52

New issue

Closed

opened 2026-10-26 12:04:35 +00:00 by james.oates · 1 comment

james.oates commented

2026-10-26 12:04:35 +00:00

Owner

Tested on :acmearr (post-40c87a7, [[acme]] array form) against Let's Encrypt staging via a real dns01proxy account, isolated on a scratch network. Not reproduced against production LE.

What happens

With a wildcard route host covering the zone, an arbitrary unconfigured subdomain is served the wildcard, but the configured [[admin]] host is refused.

[[acme]]
name = "letsencrypt"
email = "…"
staging = true

[[admin]]
host = "router.azula.oates.ws"

[[services]]
name = "grafana"
routing_hosts = ["*.azula.oates.ws"]

The wildcard order succeeds:

ACME [*.azula.oates.ws]: publishing _acme-challenge.azula.oates.ws.
ACME [*.azula.oates.ws]: TXT record visible on authoritative nameservers (attempt 1)
certificates: 1 ordered, 0 renewed, 0 removed

SNI probes against the running router:

SNI	Result
`grafana.azula.oates.ws`	`subject=CN=*.azula.oates.ws`
`deep.azula.oates.ws` (not in any config)	`subject=CN=*.azula.oates.ws`
`router.azula.oates.ws` (the admin host)	`tlsv1 alert access denied`

Why this looks like a specific exclusion rather than a lookup gap

deep.azula.oates.ws appears nowhere in the config and is still served the wildcard, so the wildcard lookup-candidate path is generic and working. router.azula.oates.ws matches *.azula.oates.ws by exactly the same rule and is refused — so the admin host appears to take an earlier path that refuses before the generic candidate lookup is reached.

Separately, the reconciler orders the admin host as its own name rather than treating it as covered:

httpreq present for _acme-challenge.router.azula.oates.ws. returned HTTP 403 Forbidden

(The 403 is my dns01proxy account being scoped to allow_domains = ["azula.oates.ws"], which permits the wildcard's _acme-challenge.azula.oates.ws only. That scoping is correct and deliberate — a DNS-01 account that can write arbitrary _acme-challenge.<anything> records is a much broader grant. But it means the per-name order can never succeed, and it retries in a loop roughly every 60s.)

Impact

A deployment using one wildcard — which is the shape that minimises LE rate-limit exposure, and the only shape a narrowly-scoped dns01proxy account permits — has no working HTTPS name for the admin dashboard. host is singular, so there is no second name to fall back to.

What I think is wanted

A name that matches a configured wildcard route host should be (a) served that certificate at handshake regardless of whether it is a service host or the admin host, and (b) not ordered separately. If the admin host is deliberately held to a stricter rule, the refusal needs a log line saying so — currently it is silent at the TLS layer and the only signal is the unrelated-looking 403 retry loop.

Possibly related: #21 (new ACME account per attempt) still reproduces here — created new account was logged separately per name.

Tested on `:acmearr` (post-`40c87a7`, `[[acme]]` array form) against Let's Encrypt **staging** via a real dns01proxy account, isolated on a scratch network. Not reproduced against production LE. ## What happens With a wildcard route host covering the zone, an arbitrary unconfigured subdomain is served the wildcard, but the configured `[[admin]] host` is refused. ```toml [[acme]] name = "letsencrypt" email = "…" staging = true [[admin]] host = "router.azula.oates.ws" [[services]] name = "grafana" routing_hosts = ["*.azula.oates.ws"] ``` The wildcard order succeeds: ``` ACME [*.azula.oates.ws]: publishing _acme-challenge.azula.oates.ws. ACME [*.azula.oates.ws]: TXT record visible on authoritative nameservers (attempt 1) certificates: 1 ordered, 0 renewed, 0 removed ``` SNI probes against the running router: | SNI | Result | |---|---| | `grafana.azula.oates.ws` | `subject=CN=*.azula.oates.ws` | | `deep.azula.oates.ws` (not in any config) | `subject=CN=*.azula.oates.ws` | | `router.azula.oates.ws` (the admin host) | `tlsv1 alert access denied` | ## Why this looks like a specific exclusion rather than a lookup gap `deep.azula.oates.ws` appears nowhere in the config and is still served the wildcard, so the wildcard lookup-candidate path is generic and working. `router.azula.oates.ws` matches `*.azula.oates.ws` by exactly the same rule and is refused — so the admin host appears to take an earlier path that refuses before the generic candidate lookup is reached. Separately, the reconciler orders the admin host as its own name rather than treating it as covered: ``` httpreq present for _acme-challenge.router.azula.oates.ws. returned HTTP 403 Forbidden ``` (The 403 is my dns01proxy account being scoped to `allow_domains = ["azula.oates.ws"]`, which permits the wildcard's `_acme-challenge.azula.oates.ws` only. That scoping is correct and deliberate — a DNS-01 account that can write arbitrary `_acme-challenge.<anything>` records is a much broader grant. But it means the per-name order can never succeed, and it retries in a loop roughly every 60s.) ## Impact A deployment using one wildcard — which is the shape that minimises LE rate-limit exposure, and the only shape a narrowly-scoped dns01proxy account permits — has no working HTTPS name for the admin dashboard. `host` is singular, so there is no second name to fall back to. ## What I think is wanted A name that matches a configured wildcard route host should be (a) served that certificate at handshake regardless of whether it is a service host or the admin host, and (b) not ordered separately. If the admin host is deliberately held to a stricter rule, the refusal needs a log line saying so — currently it is silent at the TLS layer and the only signal is the unrelated-looking 403 retry loop. Possibly related: #21 (new ACME account per attempt) still reproduces here — `created new account` was logged separately per name.

james.oates referenced this issue from a commit

2026-10-26 12:36:45 +00:00

fix: a wildcard route host covers the names beneath it

james.oates commented

2026-10-26 12:37:02 +00:00

Author

Owner

Fixed on main in a55bd0a + e51f080. Not an admin-path exclusion — admin hosts are chained into desired_names like any other, and a [[services]] host with the same spelling reproduces it identically. Two independent defects stacked:

1. A wildcard route host did not cover the names beneath it. desired_names copied every route host into the wanted set verbatim, so *.azula.oates.ws and router.azula.oates.ws were two unrelated strings and ER ordered a second certificate for a name the wildcard already certifies. Your admin host happened to be the one spelled out, which is why it looked admin-specific. Now a name a configured wildcard covers is dropped from the set — one label, never the apex (RFC 6125). That also stops the duplicate spending weekly issuance budget, which is the other reason Caddy made the same collapse its default in 2.10.

2. A tombstone shadowed a wildcard that covered the name. When the redundant order failed, lookup returned None at the first candidate holding a tombstone and never reached *.azula.oates.ws in the next slot. That read #38 too broadly: #38 forbids substituting a different identity — the bare parent's single-name certificate, or the self-signed fallback — and a wildcard over the name's parent genuinely certifies the name. Withholding it was the defect. The tombstone now records the refusal without stopping the walk; if no candidate holds a certificate the lookup still returns None and still never reaches the fallback.

That second fix is why deep.azula.oates.ws worked while router.azula.oates.ws didn't: the unconfigured name had no entry to stop at, so it fell through to the wildcard — which is what should have happened to the admin name.

Both fixes are independent on purpose. The collapse means this shape no longer arises from config alone, but a tombstone can still shadow a wildcard whenever a wildcard route is added after a name has already failed.

The end-to-end rustls test was widened rather than relaxed: the tombstoned name covered by the wildcard must now complete, and a second tombstoned name covered by nothing must still fail — with the fallback carrying that name and trusted by the client, so the refusal can't pass for want of anything to serve. 341 tests green, clippy clean, in rust:1.98.0-bookworm.

Worth re-testing on :develop once CI publishes d3d7d1f. #21 is untouched and still open — with the collapse in place router.azula.oates.ws is no longer ordered at all, so it shouldn't block you either way.

Fixed on main in a55bd0a + e51f080. Not an admin-path exclusion — admin hosts are chained into `desired_names` like any other, and a `[[services]]` host with the same spelling reproduces it identically. Two independent defects stacked: **1. A wildcard route host did not cover the names beneath it.** `desired_names` copied every route host into the wanted set verbatim, so `*.azula.oates.ws` and `router.azula.oates.ws` were two unrelated strings and ER ordered a second certificate for a name the wildcard already certifies. Your admin host happened to be the one spelled out, which is why it looked admin-specific. Now a name a configured wildcard covers is dropped from the set — one label, never the apex (RFC 6125). That also stops the duplicate spending weekly issuance budget, which is the other reason Caddy made the same collapse its default in 2.10. **2. A tombstone shadowed a wildcard that covered the name.** When the redundant order failed, `lookup` returned `None` at the first candidate holding a tombstone and never reached `*.azula.oates.ws` in the next slot. That read #38 too broadly: #38 forbids substituting a *different* identity — the bare parent's single-name certificate, or the self-signed fallback — and a wildcard over the name's parent genuinely certifies the name. Withholding it was the defect. The tombstone now records the refusal without stopping the walk; if no candidate holds a certificate the lookup still returns `None` and still never reaches the fallback. That second fix is why `deep.azula.oates.ws` worked while `router.azula.oates.ws` didn't: the unconfigured name had no entry to stop at, so it fell through to the wildcard — which is what should have happened to the admin name. Both fixes are independent on purpose. The collapse means this shape no longer arises from config alone, but a tombstone can still shadow a wildcard whenever a wildcard route is added after a name has already failed. The end-to-end rustls test was widened rather than relaxed: the tombstoned name covered by the wildcard must now complete, and a second tombstoned name covered by nothing must still fail — with the fallback carrying that name and trusted by the client, so the refusal can't pass for want of anything to serve. 341 tests green, clippy clean, in `rust:1.98.0-bookworm`. Worth re-testing on `:develop` once CI publishes d3d7d1f. #21 is untouched and still open — with the collapse in place `router.azula.oates.ws` is no longer ordered at all, so it shouldn't block you either way.

james.oates closed this issue

2026-10-26 12:43:34 +00:00

james.oates referenced this issue

2026-10-26 13:31:36 +00:00

A removed reserved-name route keeps refusing the handshake until restart #53

james.oates referenced this issue

2026-10-26 13:36:23 +00:00

Runtime resilience: survive OpenBao being unavailable #40