Exercise KV-sourced master key against a real OpenBao instance #33
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What's unproven
#27 shipped two independent OpenBao capabilities: transit (at-rest encryption) and KV-sourced master key. The live round-trip that gated that merge exercised transit. The KV path has only ever run against a mock UDS broker and injected fakes — never a real instance.
So the code is merged and in use, but these specifics are assumed from documentation rather than observed:
extract_kv_field's v1 (data.<field>) vs v2 (data.data.<field>) branch picks the right one against the actual mount.decide_master_keycorrectly against an existing hashed key (seed vs. rotate vs. already-current).Our own rule is that external services get verified against the real contract rather than assumed to behave as documented — that's exactly why transit was gated on a live run. KV never got the same treatment.
How to prove it
The gated test already covers KV when the environment supplies a path; it just has never been run with one. From inside the dev environment, where the broker socket is mounted and bao's cert is trusted:
Prerequisite to confirm first: the shared broker's policy grants read on that KV path.
Worth extending
openbao_live_round_trip(src/broker.rs:512) to assert the KV read explicitly whenBAO_MASTER_KEY_PATHis set, so a future run can't pass while silently skipping it.Severity
Low. Transit is the path in active use; KV master-key sourcing is opt-in and, as far as I know, not currently enabled anywhere. The failure mode is a startup refusal rather than anything silent, so it fails safe. Filed so the gap is recorded rather than rediscovered.
Split out of #27, which is otherwise complete and now closed.
Concrete prerequisites, from the devinfra provisioning notes (2026-08-24) — this is more than "run the test with an extra env var":
devinfra/scripts/provision-broker.sh edge-routercreates transit keyedge-routerand policyedge-router-transitwith encrypt/decrypt only. The policy needs extending with read on the chosen KV path before anything can be exercised. That's a devinfra change, not an ER one.MASTER_API_KEYmust be dropped from ER's environment. Sourcing the master key from KV hard-conflicts with the inline env var under highest-security-source-wins — correctly, it's a startup error rather than a silent precedence. So the deployment can't carry both./run/broker/consumers/edge-router.sock(per-consumer), not the shared/run/broker/broker.sockin the command above.Also worth noting: devinfra is owned by the DevInfra agent, so steps 1 and 3 are requests to them rather than direct edits. And the OpenBao wiring in devinfra is currently parked (
ENCRYPTION_ENGINE/BAO_*commented out, ER on a file-based key), so re-enabling that comes first.None of which raises the severity — it just means this is a small coordinated change rather than a one-line test run.
Closing — this was filed on stale information. KV-sourced master keys have been live in devinfra since 2026-08-31, five weeks before I raised this. My notes were dated 2026-08-24 and I never re-checked them against the deployment.
Per DevInfra:
provision-consumer.shhas anown-kvkind granting read+list onkv/data/<name>/*andkv/metadata/<name>/*(with a comment naming edge-router's master key as the motivating case), and the cert-auth role bindsedge-router-transit,edge-router-cert-consumerandedge-router-own-kv. The live environment ondevinfra-edge-router-1hasBAO_MASTER_KEY_PATH=kv/data/edge-router/master-key,BAO_MASTER_KEY_FIELD=value, and noMASTER_API_KEYat all. So all three prerequisites I listed above were already met.A boot on 2026-10-04 16:32 UTC (and two on 2026-09-29) logged, against a real OpenBao on kv-v2:
Because this path is fail-closed, a successful boot is the evidence: it proves the broker's permissions, the real KV response shape, the kv-v2 field extraction, and
decide_master_key's already-current branch. That is everything I listed as unproven.And the gated test would add nothing. Its KV assertion is only
assert!(!v.is_empty())after akv_read— strictly weaker than what a successful boot demonstrates. Running it in the dev environment would need a cargo toolchain and source tree in a container that deliberately runs the published image, to prove less than the logs already do. Not worth the wiring.Also correcting two things in my original: the KV path is fixed and in use (
kv/data/edge-router/master-key, fieldvalue), not a path for devinfra to choose. And my "the master key will rotate on next boot, API clients holding the old key stop working" warning was wrong here — it has been KV-sourced for five weeks, so there is nothing to rotate.One genuinely useful thing came out of this, which I'll track separately: the 2026-09-29 09:08 boot shows the other side of fail-closed —
OpenBao broker unreachable at startup (fail-closed): broker returned 502 Bad Gateway, exit 1, crash-looping until the broker was up. devinfra solved it withdepends_on: tpm-broker: service_healthy. ER's docs don't mention that an OpenBao-enabled deployment needs the broker healthy before ER starts, and that's something every operator enabling this will hit.