Protect the policy
A pull request cannot relax the policy that judges that same pull request. Workflow and policy changes can be placed behind protected-path rules.
Trust boundary →A required GitHub check for agent-written pull requests
Check the work before it merges.
Agent Vigil checks exact commits with rules and tests selected from the base branch. If required evidence is missing, stale, contradictory, or weakened, the check does not pass.
Hosted installer: direct node --test inference for root Node/npm repositories, or an explicit digest-pinned runner and test command for other toolchains. Local transcript adapters support Codex, Claude Code, Copilot, Cursor, Gemini CLI, Aider, and OpenCode.
Current distribution: GitHub release, versioned Action, Marketplace listing, and npm package v0.24.4 are public. Use npx --yes --package=@sulmusic/agent-vigil@0.24.4 agent-vigil protect --repo . or the immutable GitHub package.
TASK Change src/**. Test locally. Do not publish. ✓ changed-paths 3 files stayed inside src/** ✗ authorized-actions observed release_publish via gh release create ✓ tests-pass trusted command observed 792 passing; 13 skipped. FAIL · sha256:c3128a2c6abc…
PASSThe required checks passed and the evidence agrees with the change.
FAILA required check found a contradiction or policy violation.
INCONCLUSIVEThere is not enough reliable evidence to approve the change.
Where it fits
Use review to find bugs. Use Agent Vigil to enforce evidence.
Code review asks what might be wrong. Agent Vigil asks whether this exact change produced the evidence the repository required: the trusted tests ran, changed tests prove the intended regression, the test suite was not weakened, and the change stayed inside policy. It does not prove the code is correct or replace a maintainer's judgment.
No workflow changes
Paste one public pull request and keep the receipt.
Use the browser evidence desk or the read-only command to observe public GitHub metadata. Neither path fetches source files, adds an app, changes the repository, or authorizes deployment.
npx --yes https://github.com/sulmusic2-star/agent-vigil/releases/download/v0.24.4/sulmusic-agent-vigil-0.24.4.tgz pr-receipt \ https://github.com/OWNER/REPOSITORY/pull/123 \ --tool-ref FULL_AGENT_VIGIL_COMMIT \ --output pr-123.receipt.json npx --yes https://github.com/sulmusic2-star/agent-vigil/releases/download/v0.24.4/sulmusic-agent-vigil-0.24.4.tgz pr-receipt verify pr-123.receipt.json
Open the browser checker
Read the receipt boundary
Install without an npm account
Report a public receipt trial
Missing or failed evidence returns HOLD. CURRENT still leaves every protected action unauthorized.
One-command protection
Install the gate without sending us your code or transcript.
vigil protect writes four reviewable control files: a base policy, pull-request template, evidence workflow, and outcome workflow. The evidence job binds the exact head to the base policy and runs candidate setup and tests in credential-free Linux Docker. The outcome workflow records only the completed-run snapshot.
Check the tests
New skips, bypasses, zeroed coverage, lost tests, empty tests, and constant assertions block. Lower-confidence patterns remain visible for review.
Integrity rules →Keep the run
The retained Value Card keeps verification, review disposition, Actions runtime, and pull-request state visible at completion. Later lifecycle events need a separate observer.
Outcome boundary →After the first green check
A passed change can lose permission to deploy.
Agent Vigil keeps the original result and adds later evidence. A verified revert stops deployment. Another ordinary green check cannot erase that stop. Independent signed repair aimed at the exact revocation can restore permission.
CURRENTFresh verification and merge evidence allow the protected action.
REVOKEDA recorded revert stops the protected action and remains in the history.
CURRENTIndependent signed repair can restore permission without deleting the revocation.
The lab uses synthetic evidence, reads no secrets, and deploys nothing. A successful lab run proves the mechanism, not production use or customer demand.
Failures found in real releases
See what failed, why it failed, and how it was fixed.
Each record names the broken revision, the corrected revision, the maintainer’s decision, and a test that reproduces the original problem. These are Agent Vigil’s own release records, not customer results.
Stale Action artifact returned a false PASS
Malformed evidence reused an earlier run’s receipt. Hosted CI blocked v0.4 until every invocation received a fresh artifact directory.
Exact SHAs and runs →Local bytecode contaminated the package
A broad package allowlist captured publisher-machine state. The release stopped and a planted-contamination replay became a gate.
Exact package controls →Receipt output followed a symlink
An adversarial filesystem fixture overwrote the link target. v0.8 rejects unsafe destinations and replaces outputs atomically, with POSIX mode 0600.
Reproduction and limit →A real check, not a walkthrough
Run the installation and failure replay yourself.
This bounded command creates a disposable repository, installs the base-selected pull-request check, runs the setup doctor, inspects immutable Action and receipt-retention wiring, and replays the three published release failures. The run stops rather than passing if it takes longer than 60 seconds.
npm ci npm run build npm run demo:60s
The three cases are first-party records. No outside repository is currently listed as a verified adopter.
Separate signing authority
Do not sign in the candidate job.
Candidate receipt attestation is disabled in v0.24.4. The separately scheduled Control Proof workflow can ask GitHub to sign planted non-candidate challenges. It does not check out or execute pull-request code.
Exact fileThe Control Proof signature covers the SHA-256 digest of the whole proof file.
Exact runtimeThe proof binds the reviewed Agent Vigil source commit and signer workflow.
Separate jobThe signed workflow runs planted challenges, not candidate repository code.
Published benchmark
Keep the test method and the limits visible.
The test plan was committed before the run. The published results cover 520 paired synthetic cases, 325 planted defects, and 232 merged pull requests believed to be clean. Findings on those pull requests measure review burden; they are not a confirmed false-positive rate.
88.5%Agent Vigil paired synthetic balanced accuracy, with 100% clean specificity.
79.4% vs 75.1%Swarm vs Agent Vigil exact oracle recall; paired p=0.189, not a reliable difference in this run.
146 vs 622Total findings on the same 232 presumed-clean PRs: Agent Vigil vs Swarm.
Cost and outcome records
Record what the change cost and what happened after review.
vigil value keeps the recorded usage, stated cost, maintainer decision, and later outcome together. The example below uses test data.
vigil compare-value compares records only when the task type, transcript adapter, and model set match. It shows the sample size, missing cost records, later failures, review time, and uncertainty. It does not turn unrelated work into a leaderboard.
Controlled trial
Add the exact-pin pull-request gate.
The protection profile works without an agent transcript. It writes the base-selected evidence workflow, calibrated Test Integrity Guard policy, receipt retention, and completed-run outcome workflow for review before commit.
npx --yes https://github.com/sulmusic2-star/agent-vigil/releases/download/v0.24.4/sulmusic-agent-vigil-0.24.4.tgz protect --repo .
v0.24.4 is ready for controlled trials. Its successful protect run prints a doctor command using the same immutable GitHub package, so the handoff does not depend on npm. Review and commit the four prepared control files, merge the setup pull request, then run the printed command.
Root Node/npm repositories can use inferred direct node --test. Other toolchains use --runner common or an organization-owned --runner-image plus an explicit --test-cmd. A passing doctor result verifies the installed files; it does not make the check required in GitHub.
GitHub Action: sulmusic2-star/agent-vigil@29d3a6c8ac4f48baaa3f1702fdd9d297c6d328ee.
A required job name does not bind GitHub to the expected workflow or event. Use an external required-workflow ruleset or App exact-head check for enforcement and merge queues.
Generated hosted execution requires either an inferred direct node --test command or a base-owned digest-pinned image and explicit bounded test command. Missing setup or project dependencies fail closed. Local CLI commands use host privileges and are not sandboxed.