A "scoped service key" has no scope, and the mutating API is the part without the IP gate #55

Closed
opened 2026-10-26 15:37:40 +00:00 by james.oates · 2 comments
Owner

From the v2.0.0 security audit. Two separate gaps that are individually arguable and together make a leaked service key a full control-plane compromise. This is a design decision, not a defect fix, which is why it is a ticket rather than a patch.

1. key_type is decorative

models.rs defines KeyType::Master and KeyType::Service and stores it. auth_middleware (src/middleware.rs:70) validates the key, discards the returned ApiKey, and never reads key_type. A grep finds it only in the model, the DB layer and the hardcoded "service" string in the create-key response — never in an authorization decision.

So a service key can do everything a master key can: GET /api/keys, POST /api/keys, DELETE /api/keys/:id, read request logs, read certificate state, and modify any service or L4 route. README:31 and README:661 call these "scoped service keys".

2. The IP gate is on the read-only half

admin_routes (src/main.rs:56) carries auth_middleware and admin_ip_middleware. service_routes (src/main.rs:79) — POST /api/services, PATCH/DELETE /api/services/:name, POST /api/l4, PATCH/DELETE /api/l4/:name — carries only auth_middleware.

Axum 0.7 keeps per-method layer stacks through merge, so GET /api/services and POST /api/services genuinely differ: the read is IP-gated, the write is not. admin_host_gate still applies to both, so the Host must be a configured admin host — but the [[admin.access]] via/allow_ips rules are never evaluated for the write routes.

README:649 documents this split, so it is intentional. The audit traced it to the initial commit b9f5635, predating the access-rule model added in c99b255 — so it reads as an area the access-rule work did not revisit rather than a decision taken about access rules.

Why the combination matters

Hand a container a service key so it can self-register. The container is compromised. From any internet IP, with only the admin hostname (public DNS):

  • PATCH /api/services/webmail with the attacker's target_host — every request for webmail is proxied to them, and ER still presents the genuine Let's Encrypt certificate, so the browser shows a valid padlock. Credential harvest on an app that was never touched.
  • POST /api/keys — mints a key, so revoking the leaked one evicts nothing.
  • DELETE /api/keys/<master id> — locks the operator out of their own automation.

An operator who restricts [[admin.access]] to Tailscale gets 403 on GET /api/status from the internet and 200 on POST /api/services. The gating is inverted relative to blast radius.

Not CSRF-reachable from a victim's browser — the session cookie is SameSite=Strict and the mutating routes take JSON — so this needs the key itself.

Options

  1. Make key_type gate key management. A service key cannot touch /api/keys at all. Smallest change, removes the persistence-and-lockout half.
  2. Put the write routes behind the same access rules as everything else. An operator who needs a workload to self-register from anywhere grants a broader listener — which is what listener grants are for, rather than having no grant.
  3. Full per-key resource scoping — a key may only modify services it created. A real feature; later.
  4. Drop the pretence — remove KeyType, document one key as full control. Honest, weakest.

1 + 2 are both small and would be the minimum for v2. 3 is the proper answer eventually. Whatever is chosen, the word "scoped" should leave the README until it means something.

From the v2.0.0 security audit. Two separate gaps that are individually arguable and together make a leaked service key a full control-plane compromise. This is a design decision, not a defect fix, which is why it is a ticket rather than a patch. ## 1. `key_type` is decorative `models.rs` defines `KeyType::Master` and `KeyType::Service` and stores it. `auth_middleware` (`src/middleware.rs:70`) validates the key, discards the returned `ApiKey`, and never reads `key_type`. A grep finds it only in the model, the DB layer and the hardcoded `"service"` string in the create-key response — never in an authorization decision. So a service key can do everything a master key can: `GET /api/keys`, `POST /api/keys`, `DELETE /api/keys/:id`, read request logs, read certificate state, and modify any service or L4 route. README:31 and README:661 call these "scoped service keys". ## 2. The IP gate is on the read-only half `admin_routes` (`src/main.rs:56`) carries `auth_middleware` **and** `admin_ip_middleware`. `service_routes` (`src/main.rs:79`) — `POST /api/services`, `PATCH/DELETE /api/services/:name`, `POST /api/l4`, `PATCH/DELETE /api/l4/:name` — carries only `auth_middleware`. Axum 0.7 keeps per-method layer stacks through `merge`, so `GET /api/services` and `POST /api/services` genuinely differ: the read is IP-gated, the write is not. `admin_host_gate` still applies to both, so the Host must be a configured admin host — but the `[[admin.access]]` `via`/`allow_ips` rules are never evaluated for the write routes. README:649 documents this split, so it is intentional. The audit traced it to the initial commit `b9f5635`, predating the access-rule model added in `c99b255` — so it reads as an area the access-rule work did not revisit rather than a decision taken about access rules. ## Why the combination matters Hand a container a service key so it can self-register. The container is compromised. From any internet IP, with only the admin hostname (public DNS): - `PATCH /api/services/webmail` with the attacker's `target_host` — every request for webmail is proxied to them, and **ER still presents the genuine Let's Encrypt certificate**, so the browser shows a valid padlock. Credential harvest on an app that was never touched. - `POST /api/keys` — mints a key, so revoking the leaked one evicts nothing. - `DELETE /api/keys/<master id>` — locks the operator out of their own automation. An operator who restricts `[[admin.access]]` to Tailscale gets 403 on `GET /api/status` from the internet and 200 on `POST /api/services`. The gating is inverted relative to blast radius. Not CSRF-reachable from a victim's browser — the session cookie is `SameSite=Strict` and the mutating routes take JSON — so this needs the key itself. ## Options 1. **Make `key_type` gate key management.** A service key cannot touch `/api/keys` at all. Smallest change, removes the persistence-and-lockout half. 2. **Put the write routes behind the same access rules as everything else.** An operator who needs a workload to self-register from anywhere grants a broader listener — which is what listener grants are for, rather than having no grant. 3. **Full per-key resource scoping** — a key may only modify services it created. A real feature; later. 4. **Drop the pretence** — remove `KeyType`, document one key as full control. Honest, weakest. 1 + 2 are both small and would be the minimum for v2. 3 is the proper answer eventually. Whatever is chosen, the word "scoped" should leave the README until it means something.
Author
Owner

Rescoping: one half of this issue was wrong. Raised by Roku, verified against df35d28.

The body says a leaked service key is exploitable "from any internet IP, with only the admin hostname". That is wrong about reachability. admin_host_gate wraps all three API routers (src/main.rs:106) and sends every request whose Host is not a configured [[admin]] host to proxy_handler. So the API is reachable only on a configured admin host, and with no [[admin]] entry it is not served at all. #54, which claimed the no-entries case left it open, is closed as wrong.

What stands, unchanged:

  1. key_type is never consulted. auth_middleware validates the key and discards it, so a service key can mint keys, revoke the master key and modify any service. The README's "scoped" claim has been removed.
  2. The registration endpoints bypass admin_ip_middleware. POST/PATCH/DELETE on /api/services and /api/l4 carry auth_middleware only, while the read-only endpoints carry the IP gate too. So an [[admin.access]] rule restricting the API to one listener does not restrict the endpoints that change configuration — they still require the admin Host and a valid key, but any source IP that can reach the admin host will do.

The corrected exposure: an attacker needs a key and the ability to reach a configured admin host. On a deployment whose admin host resolves and answers publicly, point 2 means the write endpoints accept that key from anywhere, while a reader of the config would reasonably expect the via/allow_ips rule to apply. On a deployment where the admin host is internal-only, the network boundary holds and this reduces to point 1.

That is narrower than the body claimed and still worth fixing: the asymmetry is invisible from the config, and it is the write surface.

The two suggested minimum changes are unaffected — make key_type gate /api/keys, and put the write routes behind the same access rules as the reads.

**Rescoping: one half of this issue was wrong.** Raised by Roku, verified against `df35d28`. The body says a leaked service key is exploitable "from any internet IP, with only the admin hostname". That is wrong about reachability. `admin_host_gate` wraps all three API routers (`src/main.rs:106`) and sends every request whose Host is not a configured `[[admin]]` host to `proxy_handler`. So the API is reachable only on a configured admin host, and with no `[[admin]]` entry it is not served at all. #54, which claimed the no-entries case left it open, is closed as wrong. **What stands, unchanged:** 1. **`key_type` is never consulted.** `auth_middleware` validates the key and discards it, so a service key can mint keys, revoke the master key and modify any service. The README's "scoped" claim has been removed. 2. **The registration endpoints bypass `admin_ip_middleware`.** `POST`/`PATCH`/`DELETE` on `/api/services` and `/api/l4` carry `auth_middleware` only, while the read-only endpoints carry the IP gate too. So an `[[admin.access]]` rule restricting the API to one listener does **not** restrict the endpoints that change configuration — they still require the admin Host and a valid key, but any source IP that can reach the admin host will do. **The corrected exposure:** an attacker needs a key *and* the ability to reach a configured admin host. On a deployment whose admin host resolves and answers publicly, point 2 means the write endpoints accept that key from anywhere, while a reader of the config would reasonably expect the `via`/`allow_ips` rule to apply. On a deployment where the admin host is internal-only, the network boundary holds and this reduces to point 1. That is narrower than the body claimed and still worth fixing: the asymmetry is invisible from the config, and it is the write surface. The two suggested minimum changes are unaffected — make `key_type` gate `/api/keys`, and put the write routes behind the same access rules as the reads.
james.oates added this to the v2.1 milestone 2026-10-26 17:53:14 +00:00
Author
Owner

Options 1 and 2 are both done in d1bb5a5, with docs in ddadbdd. Option 3 (per-key resource scoping) is left as the later work this issue called it; see below for what the README now claims.

Option 1 as written would have been a no-op, which is the part worth recording. auth_middleware checks the session cookie before the X-API-Key header and returns early (middleware.rs:56), and create_session minted a session from any valid key while storing nothing about which one — Session held created_at and expires_at only. So gating /api/keys on the header's key_type would have left a two-request bypass: POST /api/session with a service key, then mint keys with the cookie.

So a session now records the key type that created it, and the invariant is explicit: a session can never hold more authority than the key that minted it. auth_middleware resolves a single Principal from whichever credential authenticated and inserts it; require_master reads that, never the credential. One authorization path, not one per credential type — the same correction access::request_is_https just made for the TLS test.

What a service key can still do, deliberately: register and modify services and L4 routes, read state, hold a dashboard session, log out. Only key management became master-only. A test asserts each of those, because over-restricting here would break the self-registration the keys exist for.

Option 2 puts admin_ip_middleware on the write routes. Both guards layer inside auth_middleware, since that is what inserts the Principal; a missing Principal denies rather than assuming authority.

Breaking-change check, since this one can lock a deployment out of its own API. Roku is unaffected structurally rather than by luck: with no [[admin]] entry, admin_host_gate hands every request to proxy_handler, so no API route is served there. Kuruk confirmed against both its config and its live database — one [[admin]] on router.kuruk.oates.ws granting via = "lan", dashboard use only, and every one of its 18 service rows and 13 tls_domain rows is source=config with no API-created entries. devinfra has nothing that calls /api/*. So nothing in the fleet self-registers over an ungranted listener. It is in the v2.1 notes for anyone else.

On the README's wording: "scoped" stays gone, but the note is no longer "treat any key as equivalent to the master key". It now says what is true — a service key cannot manage keys, and is not scoped to particular services — and points here for the scoping work.

The discriminating test is the laundering path itself (a_service_key_cannot_launder_itself_into_a_key_minting_session); without it, a header-only check passes every other case identically.

370 tests pass, clippy --all-targets -D warnings clean, fmt clean, in rust:1.98.0-bookworm.

Leaving this open for option 3 unless you would rather close it and track scoping separately.

Options 1 and 2 are both done in `d1bb5a5`, with docs in `ddadbdd`. Option 3 (per-key resource scoping) is left as the later work this issue called it; see below for what the README now claims. **Option 1 as written would have been a no-op**, which is the part worth recording. `auth_middleware` checks the session cookie *before* the `X-API-Key` header and returns early (`middleware.rs:56`), and `create_session` minted a session from any valid key while storing nothing about which one — `Session` held `created_at` and `expires_at` only. So gating `/api/keys` on the header's `key_type` would have left a two-request bypass: `POST /api/session` with a service key, then mint keys with the cookie. So a session now records the key type that created it, and the invariant is explicit: **a session can never hold more authority than the key that minted it.** `auth_middleware` resolves a single `Principal` from whichever credential authenticated and inserts it; `require_master` reads that, never the credential. One authorization path, not one per credential type — the same correction `access::request_is_https` just made for the TLS test. **What a service key can still do,** deliberately: register and modify services and L4 routes, read state, hold a dashboard session, log out. Only key management became master-only. A test asserts each of those, because over-restricting here would break the self-registration the keys exist for. **Option 2** puts `admin_ip_middleware` on the write routes. Both guards layer *inside* `auth_middleware`, since that is what inserts the `Principal`; a missing `Principal` denies rather than assuming authority. **Breaking-change check, since this one can lock a deployment out of its own API.** Roku is unaffected structurally rather than by luck: with no `[[admin]]` entry, `admin_host_gate` hands every request to `proxy_handler`, so no API route is served there. Kuruk confirmed against both its config and its live database — one `[[admin]]` on `router.kuruk.oates.ws` granting `via = "lan"`, dashboard use only, and every one of its 18 service rows and 13 tls_domain rows is `source=config` with no API-created entries. devinfra has nothing that calls `/api/*`. So nothing in the fleet self-registers over an ungranted listener. It is in the v2.1 notes for anyone else. **On the README's wording:** "scoped" stays gone, but the note is no longer "treat any key as equivalent to the master key". It now says what is true — a service key cannot manage keys, and is not scoped to *particular* services — and points here for the scoping work. The discriminating test is the laundering path itself (`a_service_key_cannot_launder_itself_into_a_key_minting_session`); without it, a header-only check passes every other case identically. 370 tests pass, clippy `--all-targets -D warnings` clean, fmt clean, in `rust:1.98.0-bookworm`. Leaving this open for option 3 unless you would rather close it and track scoping separately.
Sign in to join this conversation.
No labels
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
IsoHex/edge-router#55
No description provided.