Composable per-listener access control (bindings, listeners, access rules) #3
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Context
Came up while deploying a Tailscale-only service (Stash) behind EdgeRouter on Kyoshi. "Tailscale-only" for a regular
[[services]]entry is currently achieved purely by omission — no Cloudflare Tunnel ingress rule, relying on Pi-hole DNS overrides — rather than anything EdgeRouter itself enforces. Working through this surfaced a real design gap plus a live bug, and the discussion converged on a general model worth building properly rather than a narrow patch. Design below supersedes the original, narrower version of this issue.Current gaps
allowed_ipsonly exists on[[admin]], not regular services.AdminEntry(middleware.rs) supports a CIDR-basedallowed_ipsrestriction, enforced inadmin_ip_middleware. TheServicemodel/config (models.rs,routes_config.rs) has no equivalent — any regular service is reachable from wherever itsrouting_hosts/DNS resolves, with no source-IP restriction available at all.The existing admin IP check trusts spoofable client headers.
admin_ip_middlewarecurrently does:This prefers the
CF-Connecting-IP/X-Real-IPheaders over the real TCP peer address, only falling back to the genuine connection address if those headers are absent. Since these headers are attacker-controlled on any request that doesn't actually pass through Cloudflare (e.g. a direct connection over LAN or Tailscale), anyone bypassing Cloudflare can simply setCF-Connecting-IP: <allowed-ip>on a raw request and have it trusted. This is a live bug in production today, independent of the Stash use case.Design: bindings, listeners, access rules
Rather than fixing (2) with a Cloudflare-IP-range allowlist (still just trusting a claim, and a list that needs maintaining), make header trust a property of architecture: which literal socket accepted the connection, not what the request claims about itself.
Key mechanism: even a single wildcard bind (
0.0.0.0:443) exposes, per accepted connection, a concrete local address vialocal_addr()(already available on theTcpStreamtypes EdgeRouter uses via Tokio) — the OS fills in exactly which of the host's interface IPs the incoming packet was addressed to, once a specific connection is accepted. A client cannot influence this; it's determined by OS-level routing, not anything in the request. So we don't need multiple physical listeners to distinguish "arrived via Tailscale" vs "arrived via LAN" — one socket, inspectlocal_addr()per connection.Three layers:
0.0.0.0:443for public/LAN/Tailscale traffic; optionally a second, dedicated one for Cloudflare specifically, on a port/address reachable only from Docker's internal network).trust_forwarded_for— this should only be true for listeners structurally guaranteed to be reachable exclusively by a trusted intermediary (e.g. a Cloudflare-only binding), since that's what makes honoringCF-Connecting-IP/X-Real-IPsound: trust by construction, not by checking a claimed IP against a maintained range list.allowed_ipsinto a composable, OR-of-rules list attached to services (and admin entries — this should unify with/replace the current admin-only mechanism rather than living alongside it). Each rule isvia = "<listener-name>"plus an optionalallow_ipsCIDR list further restricting within that listener's trust context. Rules OR together (any matching rule grants access); the IP restriction within a rule ANDs with the listener match.Example config
This composes cleanly for the motivating cases: "Tailscale, any device," "LAN, but only this one IP," "public via Cloudflare, but header-restricted to a specific remote IP" — all expressible together on one service, each only as trustworthy as its actual network path allows. It also fixes gap (2) as a natural consequence of the design rather than a separate patch:
trust_forwarded_foris opt-in per listener and should only ever be set on a listener whose binding is structurally Cloudflare-only.Scope
accessrule lists (replacing standaloneallowed_ips).0.0.0.0listener.local_addr()lookup against configured listener match rules.[[admin]]'s existingallowed_ipsonto the same access-rule mechanism rather than keeping two parallel systems.trust_forwarded_forlisteners honor the header, others never do even if present.Additional gap found: no
X-Forwarded-*/ Host preservation on the upstream requestFound while debugging why Stash's thumbnails and video playback were broken behind EdgeRouter (unrelated to Stash's media — files are plain
h264/aacinmp4, about as browser-compatible as it gets).Stash builds absolute self-referential URLs (screenshot/preview/sprite/stream) from the incoming request's
Hostheader and scheme. EdgeRouter's proxy currently does neither of:Hostheader when forwarding upstream, norX-Forwarded-Host/X-Forwarded-Prototo the original values.Confirmed via Stash's own access log and by testing directly: even when
X-Forwarded-Host: stash.oates.ws/X-Forwarded-Proto: httpsare sent by the client, EdgeRouter does not pass them through (or Stash never sees usable ones), and the backend's request log showsHost: stash:9999, schemehttp— i.e. EdgeRouter is presenting its own upstream target authority as the Host, not the original public one. Result: every asset/stream URL Stash returns points at an internal Docker hostname + port, unreachable from any client, and (since the page is HTTPS) also blocked as mixed content.This isn't Stash-specific — any backend that self-references its own URL (a very common pattern: redirects, websocket URLs, federation, media asset URLs) will hit the same thing behind EdgeRouter today.
Ties into this issue's design because correct behavior here depends on the same listener/trust context this issue introduces: EdgeRouter should forward
X-Forwarded-Proto(alwayshttps, since EdgeRouter terminates TLS) andX-Forwarded-Hostreflecting the original request'sHostheader — populated from data EdgeRouter itself observed (the actual inbound request), never from client-supplied headers being blindly re-forwarded. This is the mirror-image of thetrust_forwarded_forconcept already in this issue: instead of deciding whether to trust inbound forwarded headers, EdgeRouter needs to correctly set outbound forwarded headers toward the backend.Additional scope
X-Forwarded-Proto: httpsandX-Forwarded-Host: <original Host header>toward the backend (and preserve/pass the originalHostheader itself, since some backends read that directly rather thanX-Forwarded-Host).Host/X-Forwarded-*sees the original public hostname andhttps, never the upstreamtarget_host:target_portorhttp, regardless of which listener the request arrived on.Per-service IP allowlisting, and fix header-spoofing gap in admin IP checkto Composable per-listener access control (bindings, listeners, access rules)Forwarded-header gap fixed and deployed in v1.0.6 (both Roku and Kyoshi). Root cause was more specific than first thought: the Host/X-Forwarded-Host code only ever read the literal
hostheader, but HTTP/2 requests (the default for any browser over TLS) carry the authority in the:authoritypseudo-header instead, with no literalhostheader present at all — so every real browser request silently fell back to the backend's own target_host:target_port. Fixed by resolving the original host viaparts.uri.authority()first, falling back to the literal header for HTTP/1.1.While verifying the fix, also found and fixed a related spoofing gap: client-supplied
X-Forwarded-Host/X-Forwarded-Proto/X-Real-IPheaders were being forwarded verbatim before EdgeRouter appended its own authoritative values, so a client could inject its own header of the same name and have it survive alongside (and, for a Header.Get()-style backend, take precedence over) the real one. Now strips client-supplied values for headers EdgeRouter itself sets, same as any perimeter-terminating reverse proxy should.X-Forwarded-Forstill chains as intended.Both fixes covered by regression tests (HTTP/1.1 and HTTP/2 Host preservation, and a spoofed-header test asserting only EdgeRouter's own value survives).
The bindings/listeners/access-rules design above is still open — this comment only covers the forwarded-header piece, not the broader per-listener access control this issue is about.
Cross-reference from a Playground security audit — offered as context for the
trust_forwarded_forlistener design, not a change request.The
trust_forwarded_for-only-on-a-structurally-isolated-binding model here is exactly the right anchor, and it lines up with how Cloudflare intends origins to establish that isolation. If it's useful when this gets built, the concrete "reachable only by the trusted intermediary" guarantees for the Cloudflare hop are, roughly strongest-first:cloudflared) — origin has no inbound port at all; nothing but the tunnel can reach it. Thetrust_forwarded_forbinding becomes moot because there's no non-Cloudflare path to it.Relevant to the
CF-Connecting-IPhandling in gap (2): that header is authoritative only because Cloudflare overwrites it on every request — but that overwrite only happens if the request actually traversed Cloudflare. So its trustworthiness reduces entirely to "can non-Cloudflare traffic reach this listener," which is precisely what a dedicated isolated binding guarantees. The header semantics and the binding design are the same guarantee viewed from two ends. Nothing to change here — just noting the design already has the right shape, and the above are the mechanisms that make atrust_forwarded_forbinding sound in practice.Shipped ✅
Implemented end-to-end and merged to
main. Feature commit range:be5b732..b728117(21 commits on top of baselinefca0b42).What shipped:
match_local_ipCIDR, matched against the connection's real local address, never client-supplied); access rules are{ via = "<listener>", allow_ips = [...] }, OR'd together.ADD COLUMN access_rules TEXT NOT NULL DEFAULT '[]').[[services]],[[admin]], andl4_routes.CF-Connecting-IP/X-Real-IPspoofing bug — those headers are now only honored on a listener explicitly markedtrust_forwarded_for, which is only sound because its binding (the dedicated Cloudflare-Tunnel port) is structurally unreachable by anything else.Key commits:
96da35c— migrate admin access control offallowed_ips; fix theCF-Connecting-IPtrust bugd1a636b+47cb788— generalize the accept loop into onerun_binding_loopper binding (plaintext + TLS uniform); fail-fast bind + bounded graceful drainca68aca— enforce access rules on L4 passthrough before tunneling382dc77— enforce for the admin dashboard UI and proxied services5b9ead1— end-to-end trust-boundary tests over real socketsb728117— networking-requirement docs +.gitignorehardeningVerification: 140/140 tests pass (unsandboxed),
clippy -D warningsclean,fmtclean. Whole-branch review confirmed every enforcement point fails closed.Non-blocking follow-ups spun out into #14, #15, #16. Deployment note:
match_local_ipzone separation (LAN vs Tailscale) requires the router process to see real host interface IPs — see the comments added todocker-compose.yml/edge-router.example.tomlinb728117.