Re-Verification
Skep treats verification as evidence, not a worker promise. A worker can report that tests passed, but Skep independently replays the patch on a clean worktree before presenting the run as confirmed.
When It Runs
Re-verification runs after a worker completes and produces patch evidence. It is recorded separately from the worker's own result so the audit trail shows both:
- what the worker claimed
- what Skep confirmed by replaying the patch
skep status, skep review, the serve API, and pull request bodies can surface
that independent result.
What Skep Replays
For a completed run, Skep:
- Chooses the command. The project's pinned
verify_commandcomes first. The command list from the worker'sverify.resultevent is the fallback, used only when the project pins nothing or the run belongs to no project. - Creates a fresh worktree at the commit the run started from, and records that commit.
- For a
uvor Node project, installs its dependencies from the baseline's lockfile in that worktree first, before the patch is applied (see the dependency priming below). - Applies the worker's patch artifact. If the patch changes the lockfile, the dependencies are installed once more so the new ones are the ones tested.
- Re-runs the command from the clean worktree.
- Records the outcome, exit codes, a detail message, and which command it
was:
[command from the project's pinned verify_command]or[command from the worker's own verify step].
The replay passes only PATH and HOME through from the supervisor
environment. The command runs inside the sandbox with deny-all network and
writes confined to the clean worktree; loopback ports the project opened
(--loopback-port) stay reachable. If the sandbox cannot be built, the command
is not run on the bare host: the outcome is unavailable. Each command is
bounded by the run's wall-clock budget, and never by less than 300 seconds.
To pin the command, see
guide-pin-verify-command.md.
Because the replay has no network, Skep first installs the project's
dependencies from the baseline's lockfile, then again from the patched lockfile
when the patch changes it: uv projects (uv.lock) and Node projects whose
pin runs node, npm, npx, pnpm or yarn (package-lock.json,
pnpm-lock.yaml, yarn.lock), including after VAR=value, env or another
command (cd web && npm test). Other ecosystems are not primed: their pin
re-runs only with what the host already has.
A patched lockfile is installed before anyone approves the patch. That
install never gets open network: it reaches the package registries alone
(pypi.org and files.pythonhosted.org for uv; registry.npmjs.org and
registry.yarnpkg.com for Node; plus any registry or git host the baseline
lockfile already resolves from) through Skep's filtering proxy, so a patched
lockfile that names any other host fails to install. Node installs run with
lifecycle scripts off and corepack offline: where yarn or pnpm is a
corepack launcher, corepack reads the host's own corepack cache
($COREPACK_HOME, else ~/.cache/node/corepack) and never writes it, so
only a package manager the host already has runs. With none cached the
replay is unavailable and names the fix (corepack prepare yarn@<version> --activate). A patch that changes the package
manager itself is not installed and the replay is unavailable, with the
reason: a new packageManager, or any change to .npmrc, .yarnrc,
.yarnrc.yml, .pnpmfile.cjs, .pnpmfile.mjs, pnpm-workspace.yaml,
.yarn/ or the baseline's yarn-path target.
Registry-only holds only when the supervisor sandbox is on. With the sandbox off, the installers are only told to use the proxy (through their proxy environment); nothing stops a tool that ignores it, and the replay itself runs unconfined on the host.
Known residual: uv still builds source distributions, and the project itself,
during the prime and the re-prime. A build step from a patched lockfile or
pyproject.toml therefore runs before approval, with the same registry-only
network. It cannot write re-verify's shared dependency cache: every install
after the patch applies uses a throwaway cache inside the replay and a sandbox
profile with no shared cache writable, so it downloads from cold.
The dependency caches live under the project's cache root
(~/.skep/cache/projects/<project>/). A sandboxed worker writes only its
worker/ half. Re-verify primes from its reverify/ half, which no worker's
sandbox mounts (a worker in workspace mode has no sandbox, so this does not
hold for it).
The replay of the patch's own commands cannot write either: it reads the Node
cache, gets a throwaway uv cache, and its packages are copies, not links into
the cache.
A node pin in a repository that declares dependencies but has no lockfile runs
unprimed, and the detail says so; an npm, npx, pnpm or yarn pin there is
unavailable ("no lockfile to prime from"). A patch that adds the first
dependencies with a lockfile is primed from it (same rules); without
one it is unavailable, never failed.
Outcomes
| Outcome | Meaning |
|---|---|
passed |
Patch applied and every command exited 0. |
failed |
Patch did not apply, a command timed out, or at least one command exited non-zero. The tail of the output is kept (from pytest's short summary on, when there is one), and the whole output, up to its last 200 KB, is written to reverify.output.txt in the run's evidence directory. |
unavailable |
Skep could not honestly re-run the check: the command was not installed (exit 127), the sandbox could not be built, the dependency install failed or was refused (see above), the worker claimed changed files but left no patch, or there was no command to run. |
not_applicable |
The run changed no files or produced no patch, so there is nothing to re-verify. Shown as nothing to re-verify, without an alarm. |
A run is confirmed only when the worker reported verification passed and
Skep's replay outcome is also passed. A run has one re-verification record.
What Run Shows
skep run (and a resumed run) prints Skep's replay verdict as its
verification line once the replay ran, and the worker's own check below it
as the worker's claim (v127-T3) — an external engine's own check is only
git diff --check, so its "passed" is never shown as the verdict:
verification: NOT CONFIRMED by re-verify (G10) — re-ran npm test → exit 1
worker claim: passed — codex produced a verified patch.
What Review Shows
skep review <task_id> includes the worker claim and the independent replay:
task 019...
state: completed
verification: passed (worker says tests passed)
re-verify: passed [confirmed] (G10): re-ran clean: all exit 0 [command from the project's pinned verify_command]
re-ran ['pytest'] → exit [0]
If replay disagrees with the worker, review marks it loudly:
re-verify: failed [NOT CONFIRMED — DO NOT TRUST] (G10): re-run exit codes [1] (expected all 0); full output: <evidence dir>/reverify.output.txt
That disagreement blocks auto-landing, and so does a confirmed replay of a
known-vacuous check (below). You can still inspect the evidence and approve by
hand; skep review <task_id> --approve then lands the patch and prints a
warning that re-verification failed.
Why This Matters
Without re-verification, a supervisor would still be trusting the agent's own summary. Re-verification changes the trust boundary: the agent supplies a patch, the project supplies the command, and Skep checks whether that patch satisfies that command on a clean copy.
This catches common failures:
- the worker changed files but did not run tests
- the worker reported success after a stale or partial test run
- the patch depends on untracked local files
- the patch does not apply cleanly to the original baseline
- the worker claimed success but the replayed command fails
Current Limits
- Re-verification is only as strong as the command. A pinned command that does
not cover the change confirms a wrong patch. On an unpinned project the
worker chose the command: if it recorded
true, Skep can confirm only thattrueexits successfully. External engines refuse to work without a pin for that reason, and such a run is never auto-landed: every auto-landing rule needs the project's pinnedverify_command, never the worker's own choice, and refuses a replay whose commands are all known-vacuous (true,:,exit 0,test/[,echo,printf,ls,cat,git diffwith no flags or only--check,git status,git log, npm'sno test specifiedplaceholder,sh/bash/zsh -cof those, any of those behindenvorKEY=value; empty), names it on the run'sauto_apply_blockedrecord, and leaves the patch for a human. The pin runs through/bin/sh -c, and Skep reads it the way the shell does: after;or a newline only the last command decides the exit code (uv sync; uv run pytestandcd web; npm testare real checks,pytest; trueis not); a chain joined by&&is vacuous only when every part is (true && pytestis a real check) and one joined by||when any part is (pytest || true); only the-cargument ofsh/bash/zshis read as a nested command (python -c "import app; app.check()"is a real check). A pipe|, a background&, backticks or$(anywhere is refused for auto-landing outright. A patch that changes what the pin runs never auto-lands either. Any touch of a test-runner config file blocks it:conftest.py,pytest.ini,.pytest.ini,pytest.toml,.pytest.toml,tox.ini,setup.cfg,setup.py,noxfile.py,Makefile,.coveragerc,.npmrc,.yarnrc*,jest.config.*,vitest.config.*,vitest.workspace.*,vite.config.*,.mocharc*(in any directory); so does any file a word of the pin names (uv run python scripts/check.pymakesscripts/check.pya control file), and any file named by thepackage.jsonscript the pin runs (npm test→scripts.test→node test/run.js).pyproject.tomlblocks when its[tool.pytest*],[tool.coverage*],[tool.tox],[tool.nox],[tool.poe],[tool.hatch.envs],[tool.pdm.scripts],[project.scripts],[project.gui-scripts], any[project.entry-points.*]table or[build-system]change (a console script namedpytestwould replace the tool the pin runs);package.jsonblocks on any change other than its dependency lists,overrides,resolutionsorversion, so a dependency bump still lands. Skep reads this from the patch, not the worker. This blocks global switches of the test setup, not test edits: a patch that edits the test files themselves still auto-lands if its own tests pass. Re-verification proves the pin passes on the patched tree, not that the patch is good. Known-vacuous is a list of shapes, not proof that any other command can fail. Every save face (setup, preview, phase change, the web policy editor) turns an auto-apply it did not ask for (a phase default or a stored value) off on a missing or known-vacuous pin, with a note, and refuses a request that itself turns auto-apply on with such a pin. - Toolchain mismatch is reported as
unavailable, notfailed. For example, if the command ispytestbut the supervisor environment cannot find it, Skep cannot honestly confirm the run. - The dependency install (uv and Node) reaches the package registries before the offline re-run. Other toolchains are not primed: their dependencies must already be available offline.
- Skep does not re-verify failed or pending runs as successful work. There is no completed patch claim to confirm.
- Long-running verification commands are bounded by the supervisor timeout.
For the broader run lifecycle, see how-it-works.md.