# ADR 0062 — the provider-key relay: the worker never holds the provider key

**Status:** accepted (operator decision D28), delivered as a strangler behind
the `provider_key_relay` kill switch, **default OFF**.
**Related:** ADR 0011 (network proxy), ADR 0019 (the llm-secret exception to
G2), ADR 0047 (CLI agent engines), ADR 0050 (one verb, three faces), ADR 0051
(provider shelf, per-profile keys), ADR 0059 D3 (V6a `provider_hosts`; this
ADR is its deferred V6b). Invariants touched: I5, I8, I9, I11, I12.

## Question

A sandboxed builtin worker that calls its model provider holds the provider
key. It reads `llm-secret` / `llm-secret-<id>` (the B125 carve-out in
`sandbox.secret_paths`) or the profile's `api_key_env` variable when the
operator put it on the env allowlist. Anything that runs in the worker —
a test the model wrote, an install script, a compromised dependency — can
read the key and send it to any host the run may reach. ADR 0059 deferred
the fix (V6b) because netproxy is a CONNECT tunnel: it cannot add a header
inside TLS.

## Decision

### 1. Transport: a route on the run's existing filtering proxy

The relay is not a new listener. It is a second route on the per-run
`FilteringProxy` (`netproxy.py`) that dispatch already starts for every run
with a domain allowlist, and which every backend already lets the worker
reach:

- Seatbelt pins the worker's egress to `localhost:<proxy port>`.
- bubblewrap (`--unshare-net`) reaches it through `skep.netshim`, which
  presents the same port on the sandbox loopback and bridges to the proxy's
  AF_UNIX door. On `es-r5b-landlock` that door is one of the two
  socket-connect rules (the worker's roots and the proxy bridge), so the
  relay needs no new rule.
- With `es-r2a-network` (D6: loopback closed by default) the proxy port is
  still the one loopback port a run may dial. The relay rides it.

A forward-proxy request is always absolute-form (`POST http://host/...`) or
`CONNECT`. The relay takes the two shapes that are addressed to the proxy
itself: an origin-form request (`POST /api/chat`, the worker dialled the
proxy port directly, as Seatbelt's `NO_PROXY=127.0.0.1` makes it) and an
absolute-form request to `127.0.0.1:<the proxy's own port>` (bubblewrap,
where the worker's client goes through `HTTP_PROXY`). Before this ADR both
were refused (400 / a loop to itself). Nothing that worked before changes.

The worker's provider URL becomes `http://127.0.0.1:<port>` plus the
upstream base URL's own path, so every path rule the clients apply to a base
URL (`openai_style_prefix`'s bare-host `/v1`, ollama's `/api` strip) gives
the same path through the relay as direct. The relay replaces only the
origin: it forwards to the upstream's scheme, host and port, over TLS when
the upstream is `https`, refusing a path outside the upstream base path.

A run whose network is allow-all has no proxy today. With the relay ON it
gets a relay-only proxy (empty allowlist); the worker reaches its port
because allow-all permits loopback. A deny-all run never needs one: a caste
that calls a provider always has `provider_hosts`. A podman allow-all run
is not relayed: podman gives every run its own network namespace, so the
host's loopback is out of reach (a podman domain-list run already falls
back to the native backend); it keeps today's path and its record says so.

Rejected:

- **A unix socket bound into the sandbox.** A second door per run, a second
  Landlock rule, a second path for R4 to rebuild, and Seatbelt would need
  its socket allowed by path. The proxy door already exists on every backend.
- **A new loopback port.** On `es-r2a-network` that is a new hole in the
  loopback wall that has to be named in every profile.
- **TLS interception in the CONNECT tunnel.** A CA in the worker's trust
  store, for every host, to rewrite one header.

### 2. Per-run capability: a token bound to the run and to one upstream

The worker sends a per-run token where its client already sends a key
(`Authorization: Bearer <token>`; `x-api-key: <token>` for anthropic), so the
protocol clients in `serve/llm.py` change in no line. The relay compares it
in constant time, strips every credential header (`authorization`,
`x-api-key`, `api-key`, `x-goog-api-key`, `proxy-authorization`) and adds the
real key in the protocol's header. A request without the token gets 403
and no upstream connection.

The token is `HMAC-SHA256(relay-secret, task_id)`. `relay-secret` is a 0600
file the supervisor creates in the supervisor home before it writes any
sandbox profile (ES-F264: the mask list holds only files that exist, so a
secret made on first use was readable by a run already going); its name
matches `*-secret`, so the sandbox mask list and the secret floor already
hide it from every worker and every Queen read tool. One relay serves one
run: another run's worker holds another token, and a relay is bound to one
upstream origin and base path, so a token cannot reach another provider.
Deriving the token, rather than drawing it at random, means a restarted
supervisor recomputes it without storing it anywhere (section 7).

### 3. Kill switch

`provider_key_relay` (bool, default false), a global knob set like every
other supervisor setting: `PUT /api/policy`, the chat `set_policy` card, the
web UI, and listed in the policy registry. Dispatch reads it from the store
for each run, so `skep run` and serve obey the same switch. OFF is exactly
today's path. Each run's `dispatched` line records which path it used.

### 4. Golden-parity gate

`tests/supervisor/test_provider_relay_parity.py` sends the same model calls
(one per relayed protocol: ollama, openai-compat, anthropic,
openai-responses) direct with the key and through the relay with the token,
against the repo's fake providers, and requires the recorded requests to be
identical apart from the auth header: path, query, every other header (Host,
Connection and Content-Length included) and the JSON body, and the parsed
replies identical. The credential header itself must match too: the worker
sent the token, the provider must get the key in the same header and form.
The gate's first run caught one difference (the relay dropped the worker's
`Connection: keep-alive`); the relay now passes a plain keep-alive/close
through. The default may only flip to ON with this gate and the full suite
green.

### 5. Which providers and engines

- **Builtin worker castes (coding, document, reviewer)** — relayed when ON.
  They resolve their provider through one function
  (`llm_plan.worker_provider_from_env`); the supervisor resolves it with the
  same code (`worker_provider_from_home`) and hands the worker the relay URL,
  the token, the protocol and the model. The worker no longer reads a key
  file, and the sandbox masks them (the B125 carve-out is not applied), and
  the profile's `api_key_env` variable is dropped from the worker env.
- **Not relayed, today's path:**
  - `bedrock`: SigV4 signs the host, path and body with the secret, so a
    relay cannot swap a header; it would have to re-sign, i.e. hold the
    secret key in the signer. Possible later; out of scope.
  - a provider with no key (local ollama): nothing to hide.
  - **claude_code** can be pointed at a base URL (`ANTHROPIC_BASE_URL`) and
    sends `x-api-key` or `Authorization: Bearer`. **codex** can
    (`OPENAI_BASE_URL` / a `model_providers` entry) and speaks the Responses
    API. **aider** can, per provider, through litellm (`OPENAI_API_BASE`,
    `ANTHROPIC_API_BASE`, …), but which provider is in the operator's aider
    config, which skep does not read. **pi** is multi-provider by its own
    config. All four authenticate from their own auth env (ADR 0047, W33)
    and stay on that path this round. `skep doctor` says so in one line.
    Relaying claude_code and codex is a follow-up: the same route with the
    engine's base-URL variable, once each engine's full request set
    (count-tokens, model lists, websocket upgrades) passes the parity gate.

### 6. Streaming, logging, accounting

The relay streams: it forwards the response status and headers (minus
hop-by-hop), then the body as it arrives, closing the connection to end it.
SSE and NDJSON reach the worker chunk by chunk, as through the tunnel.

It logs, per request, one counter (`relayed`, `relay_refused`) on the proxy
next to `allowed` / `denied`. Never the key, the token, a header value or a
body: the run record already keeps the prompts it keeps (I8), and the relay
adds none. Usage accounting is unchanged: the worker still counts tokens
from the stream (`ProviderUsageTally`) and the budget's
`max_provider_calls` still binds it; the relay's request count is a check
on that self-report, not a second budget (I5).

### 7. Without serve, and serve restarting mid-run

The relay lives in the process that runs dispatch and spawns the worker. Under
`skep run` that is the CLI process; under serve it is serve. Both start it
through the same code. No serve is needed.

On main, a serve restart ends the run (recovery kills the worker), and the
relay dies with the proxy. With re-adoption (`es-r4-readopt`, ADR 0060) the
new serve restarts the run's proxy on the recorded port from `task.json`;
the relay comes back with it because nothing about it is stored: the
upstream and key are resolved again from the same configuration, and the
token is recomputed from `relay-secret` and the task id. The window between
the old serve's death and the new proxy's start refuses provider calls, as
it refuses all egress today. When R4 lands, its proxy restart must pass the
relay too; that is one call site.

## Strangler order

1. Relay route and kill switch, default OFF (nothing calls it).
2. The golden-parity gate.
3. The builtin worker uses it when ON.
4. The key is absent from the worker (env and files) when ON, proved by a
   real sandboxed worker reading its own environment.
5. Doctor line.

The old path (key files readable to LLM castes, `api_key_env` passthrough)
stays until the relay has run ON as the default through a dogfood period; it
is deleted only then, which is when this door becomes one-way.

## Consequences

- With the relay ON, a worker's only credential is a token that works on its
  own run's relay, for its own provider, while the run lives.
- `supervisor.sqlite3` stays readable to a sandboxed worker this round (other
  worker code reads settings from it). With the relay ON the worker no longer
  needs it for its provider, so masking it is now possible, as a follow-up.
  `~/.npmrc` is unrelated to the provider key and stays readable.
- Workspace mode has no wall: with the relay ON the key leaves the worker's
  environment, but the key files remain readable to a workspace-mode worker.
- A worker can still spend the operator's quota through the relay, at its own
  provider, within its budget. That was true with the key; what it can no
  longer do is take the key away.
