Evidence

How this repository knows that a claim is proven rather than merely referenced: a stable identity for every test, derived from the path a claim already names; a recorded execution carrying the provenance the run itself did not have; a ledger of the latest execution per test; and one derivation that joins the two and says, per claim, exactly what can honestly be said about it. Behaviour as implemented; where this document and the executable disagree, the document is wrong and changes in the same commit. The decision is ADR 40; the rules it serves are project.no-claim-without-test and project.never-reported-is-not-green under .ai/repo/rules/project/.

flowchart TD
  claims["docs/CLAIMS.yaml<br>claim → source, implementation,<br>test (a path)"]
  testid["TestId::of(path)<br>suite:84_distribution_model<br>crate:why"]
  ledger[".ai/repo/evidence/ledger.json<br>latest execution per test:<br>outcome, seconds, commit, tree,<br>digest, time, origin, command"]
  latest["ledger.latest(id)"]
  report["evidence::report()"]
  states["proven · inputs_unchanged · stale · failing<br>not_run · unrunnable · no_test"]
  out["CLI · HTTP · MCP · scripts/evidence-check"]
  claims --> testid --> latest
  ledger --> latest
  latest --> report --> states --> out

The problem

docs/CLAIMS.yaml binds a claim to a test with a path:

- id: distribution-canonical-model
  claim: Every platform, artifact name and installation URL is derived from one model …
  source: docs/DISTRIBUTION.md
  implementation: apps/majordomus-cli/src/distribution/mod.rs
  test: test/cases/84_distribution_model.sh
  status: guaranteed

A path is a declaration that a proof exists somewhere. It is not the proof. scripts/generate-site-data --check verifies that every source, implementation and test path resolves — a check on the reference, not on the claim. Until this subsystem, nobody could answer from the repository whether that case had ever run, against which revision, with what result, or whether the result still applied, and the sentence was rendered on the public site as a guarantee on the strength of a file existing.

This is the same defect ADR 30 named one level up: a test result that cannot go stale is a claim about the past presented as a claim about the present.

The model

Three objects and one derivation, in apps/majordomus-cli/src/evidence/.

The join is derived on every read and is never written down. There is no second place where a claim's proof state is stored, and therefore no place for it to go stale.

Test identity, derived rather than entered

A test's identity is computed from the path a claim already names. Nothing is entered by hand, no field is added to the matrix, and no claim had to be migrated:

Both spellings already existed: test/run.sh names a case by its file stem, and cargo test --test <stem> runs an integration binary by its. TestId::of only names the join.

"Anything else" is an answer, not a failure. A claim may name a template, a fixture, a library or a document — share/install/install.sh.in, test/lib.sh, a nested path under test/cases/ that the runner does not enumerate — and the report says unrunnable rather than counting it as covered. The matrix's own placeholder for nothing, -, quoted or not, reads as no test at all.

A third runner would be a variant of Runner and a branch in TestId::of. It is deliberately a code change rather than a configuration one: a runner nothing executes is a runner whose results nothing can record.

An execution

Every field is provenance. A result with no commit is an anonymous green, which is the one thing a badge must never be derived from.

Only pass proves anything. The runner's word is read case-insensitively (ok, pass, passed; fail, failed; skip, skipped; timeout) and anything unrecognised is error — a result nobody can classify is not a pass. There is deliberately no not_run outcome: that is the absence of an execution, not an outcome one had, and encoding it would let a recorder write "this did not run" and have it counted among the things that did.

origin is provider-neutral. A CI adapter records ci; nothing in the model knows or cares which CI it was.

The runner speaks that vocabulary. A case that cannot meet a precondition calls skip from test/lib.sh, which exits 4; test/run.sh prints skip, writes SKIP into the TSV in both the serial and the parallel phase, and counts it in a third tally beside passed and failed. It is not a failure — the run's exit status is unchanged — and it is not a pass either, which it was until the word existed: a case that declined exited 0, the runner wrote ok, and the ledger recorded the proof of a claim from a run that asserted nothing. The status is not the declaration on its own: skip also writes the file the runner names in MJ_SKIP_MARK, and a case that ends with 4 without it — jq -e fails with 4 when it produced no result, and a case runs under set -e — is a failure, not a skip. The claim is a-skipped-case-is-not-a-proof and the case is test/cases/413_a_skipped_case_is_not_a_proof.sh, which also refuses any case in test/cases/ that still says exit 0 before its last statement, so the old shape cannot come back with a new case.

The seven states

Ranked from strongest to weakest, so a summary that sorts by the state reads as a ranking. Each state carries its own one-sentence meaning in the model, so every surface says the same thing rather than inventing a gloss.

Every state is decided by one function, evidence::freshness, from the recorded run and the revision it is judged at, and the first row that matches wins. E is the commit the run was recorded on; "the presented revision" is what is judged (below); "the inputs" are what the claim names — exactly three paths: its source, its implementation and the test's own source (a rule names its test's source and its own definition). "Changed" is every path that differs between E and the presented revision, and changed' is that set without the ledger's own row.

Only row 10 is proven, and only proven may be rendered as verified on any surface. A stale claim names the paths that changed, or says in its detail why it is stale, so a reader is told what invalidated the run rather than being told to go and look. Not knowing is not proof: an unanswerable comparison is row 6, never row 13.

The presented revision. evidence show judges the working tree: HEAD plus every tracked, staged and untracked change, compared with git diff --name-only E -- together with the untracked files git would show. evidence show --presented HEAD judges the checked-out commit as committed. The commit must be the one checked out — the claims, the tests' sources and their digests are read from the checkout, so a verdict at any other commit would mix two trees — and the ledger is the one committed in it, compared with git diff --name-only E <commit> --. The presented tree is measured ignoring the ledger's working copy, which is not what is judged, and --presented-tree can only weaken that measurement. The report says what it was judged at in presented. This is the reading a site built from a commit needs, and no surface asks for it yet: the evidence publication (issue I1955) and the site build that binds it to build.json (issue I1959) are the later slices that will pass build.json.commit as --presented and a build with build.json.dirty set as --presented-tree dirty. The two measurements differ: build.json.dirty ignores untracked files, and the presented tree counts them. The narrower one cannot make a verdict proven, because the presented tree is measured here whatever the caller declares, and a declared state can only weaken it.

The working ledger's uncommitted executions. Under --presented, the executions the working copy of the ledger holds that the committed ledger does not are listed in presented.uncommitted, and read only through the monotone rule: a supplementary record may withhold proven and never grant it. When one of them failed, timed out or errored, on a clean tree, at a commit that contains E and that the presented commit contains, a proven or inputs_unchanged verdict is capped at stale, with a detail naming the run. The cap is stale and not failing: such a record never decides a verdict, and an uncommitted pass never strengthens one. A working ledger that cannot be read is refused, because a run it may hold cannot be ruled out.

A skip is not a failure. A test that declined to run proved nothing and failed nothing: it is not_run, with a detail saying it declined, and a guarantee resting on it is named by a finding that says its test declined to run. A test that errored or timed out did not decline: it is failing.

Containment. A pass recorded on a commit the presented revision does not contain is a fact about another history — a branch that was never merged, a rewritten one — however empty a diff between the two trees happens to be. It is stale, and its detail names the commit. A commit this clone does not have is neither contained nor not: git cannot answer, which is row 6, and nothing is diffed against it. A ledger row's commit is data, so a word git would read as an option (--output=<file>) is never handed to git: it names no commit, and it is row 6 too.

Aggregation. Where several states make one — a rule over the tests it names — a failing part makes the whole failing, whatever the other parts say, because the ranking puts failing above not_run and the weakest of a failing test and a skipped one would otherwise read not_run and hide the failure. Otherwise the whole is the weakest of the parts that can carry proof, and a part nothing can ever record counts only when it is all there is.

proven and inputs_unchanged are deliberately not the same state

This is the point of the whole subsystem, and it is the thing to preserve in any change to it.

proven means a passing run exists, nothing has changed since it, the run measured the commit it is joined to, and what is presented is that commit's own history, as committed: the presented revision contains the execution's own commit, the diff between the two is empty but for the ledger, the execution's working_tree is clean, and so is the presented tree. It is proof of the tree in front of you — the tree the run measured is the tree you are looking at.

Each condition is needed, and each fails differently. A run recorded while something else was pending sat on its commit without measuring it (ADR 0041: "proven is a passing run recorded against this exact commit with a clean tree"), so it is capped at inputs_unchanged however empty the diff is when the report is taken — a later git checkout of the edited file cannot retroactively make the commit describe what ran.

The ledger's own row is excluded from that diff, because the evidence is about the tree rather than part of what the tests measure. Without that exclusion proven would be unreachable by construction: recording writes the ledger and so dirties the tree, and committing the record moves HEAD past the commit the record names. That was a real defect in the first cut of this design, found in review; test/cases/124_evidence.sh now asserts both halves — that the tree is dirty after a recording, and that every claim is proven anyway — so a regression that made proven depend on a clean tree again would fail.

The same exclusion applies when a run is recorded: a recording whose only pending change is the previous recording's own row is stamped clean, because that row is not something any test measured. Any other pending path is dirty and the execution says so.

inputs_unchanged means a passing run exists and nothing the claim names has moved since it. That is the absence of a known invalidation — not proof at HEAD. Something the claim does not name may have broken the behaviour: a shared library the implementation calls, a schema it reads, a script the case invokes, a dependency bump, a change to test/run.sh itself. The dependency model is deliberately shallow and non-transitive: it is the three paths the claim writes down, and nothing computed from them.

Both alternatives to that choice are worse. Treating a passing run as valid until something anywhere changes is the green badge whose derivation cannot be inspected. Declaring every result stale the instant anything anywhere changes is sound and useless — it would mean no claim is ever proven except in the seconds after a full run, on a repository that pushes to master every few minutes.

So the model reports which of the two it found, and labels the weaker one as weaker everywhere it is shown. The command line prints them differently and prints the sentence that explains the second. A surface that renders both as one tick has reintroduced the defect, whatever the underlying data says.

What inputs_unchanged does not catch, stated plainly:

• a change to a file the claim does not name that breaks the behaviour anyway;
• a change to the runner, the harness, the toolchain or the environment;
• a claim whose implementation field names one file where the behaviour lives in several;
• a behaviour that depends on something outside the repository entirely.

Each of those is a real hole. The answer to all of them is the same and is not a cleverer dependency graph: record a run against HEAD, and the state becomes proven.

Current evidence, for a reader that counts a route as met

A reader that turns a verdict into met or not — an intent's satisfaction criterion is the first — does not draw its own line. evidence::current draws it once, over a verdict of the truth table: proven is current, and inputs_unchanged is current only when the run measured a clean tree and the presented revision is its own (the working tree, whose changes are already in the comparison). Rows 11, 12 and 13 all say inputs_unchanged, and only row 13 on two clean trees is current: a run recorded on a dirty tree is not, because its commit does not describe what ran and no later tree can be matched against it. Currency is a property of the inputs the route names and of the recorded digest, never of commit equality with HEAD, so committing the ledger after a run, or a change the route does not name, leaves current evidence current; everything weaker is not current.

An intent criterion naming a test judges it with the test's source and the source and implementation of every claim that test proves as its inputs (docs/PLANNING.md). A test that proves no claim naming an implementation declares nothing about the code it tests, and its own file is not that code, so it is judged as a route that names no inputs (row 14): its pass is current only while nothing but the ledger has changed since the run, and a change to the code under test, with the test file byte-identical, makes it stale.

Counting inputs_unchanged as met does not make the two strongest states one tick. met is the intent's own question, and every intent criterion carries proof, the verdict behind it: the terminal prints current (proven) or current (inputs_unchanged), and the Cockpit's badge reads current · proven or current · inputs unchanged in different colours. A reader who needs proof at the revision reads proof, never met.

The ledger

.ai/repo/evidence/ledger.json, tracked, one entry per test: the latest execution.

It is shaped like the benchmark baselines under .ai/repo/benchmarks/ on purpose — JSON, committed, written by a tool, read by a gate, reviewed in a diff. The repository already had a precedent for recorded, committed, reviewable measurement, and this is the same kind of artifact for the same reason rather than something new.

JSON, not the layer's YAML. This repository reads a deliberately small YAML subset (SCHEMAS.md), and a canonical file only our own reader can parse is a trap it has fallen into before. The ledger is machine-written and machine-read, so it is JSON: serde on both sides, no subset to respect, and jq works on it.

Latest per test, no history. A run per commit for every test grows without bound, and the question the ledger answers — is this claim proven now? — only ever reads the newest. A history would be a second thing to maintain in service of a question nobody asked here; the durable record of what happened over time is git, over this file.

No captured output. A log of every case is megabytes per run and belongs where the run happened — the CI artifact, the terminal. What is kept is the durable semantic part, which is what a reader needs in order to decide whether to believe the result and how to check it themselves.

Merged per test, never rewritten per run. Recording one test replaces that test's entry and leaves every other alone. This is what makes bash test/run.sh 84_distribution_model worth recording at all: a whole-ledger write would silently delete the evidence for every test the run did not include, turning a one-case run into a repository that has proven one thing — which is worse than one that has proven nothing, because it looks deliberate.

Ordered and stable. Entries are sorted by test id and the file is written with a two-space indent and a trailing newline, so two runs of the same set produce the same file and a diff shows what changed rather than what moved.

Versioned, and refused rather than guessed. The file declares version: 1. A reader that meets a version it does not know refuses with both numbers and says why: a misread ledger is worse than an absent one, because an absent one reports not_run and a misread one could report a pass.

A missing ledger and an empty one are the same answer. Both report every claim as not_run, which is true, and neither is an error — a fresh clone has the first and a repository mid-adoption has the second. The report's ledger summary says which it was, along with how many executions it holds, the newest timestamp in it, and the distinct commits they were recorded against.

Recording a run

The runs already write their results down. test/run.sh writes one TSV row per case — name, result, seconds, phase — when MJ_TEST_REPORT is set. cargo test prints a Running tests/<name>.rs line and a test result: line per integration binary. Neither is durable, neither carries a commit, and neither is joined to anything.

A run measures the checkout it left with evidence stamp, and evidence record reads what the run wrote, carries that measurement into each result with the digest, the time and the origin, and merges it into the ledger.

MJ_TEST_REPORT=suite.tsv bash test/run.sh
majordomus evidence stamp --producer suite --report suite.tsv --out suite.provenance.json
majordomus evidence record --suite suite.tsv --provenance suite.provenance.json

cargo test --manifest-path apps/majordomus-cli/Cargo.toml 2>&1 | tee crate.log
majordomus evidence stamp --producer crate --report crate.log --out crate.provenance.json
majordomus evidence record --crate-output crate.log --provenance crate.provenance.json

Then commit the ledger. A recording that is not committed is a local opinion.

It records; it decides nothing. A case that failed is a case the runner said failed, and a test that appears in no report is left exactly as it was — the report will say not_run for it, which is true.

The run measures; the recorder carries. A recorder that stamped each result with the tree it sees would vouch for a tree it did not see: CI records in a fresh checkout, and a local run may be recorded after its checkout changed. So evidence stamp measures the checkout as the run left it: the commit; the tree, with the evidence ledger ignored and the run's own untracked outputs excluded (--exclude names a file or a directory, never a pattern, the whole checkout or a path holding tracked files, so an exclude never hides a tracked change; a report inside the checkout is one of those outputs); the producer, its toolchain, the recorder's version, the report's digest, the host and the CI run. evidence record --provenance [<producer>=]<file> carries that measurement into the report's executions. When the measured tree was clean, the test's digest is taken from the commit, which holds the bytes the run executed. A report recorded without a measurement carries the tree unknown, which never reads proven. A local or release measurement must name its report (--report), or it cannot say which run it measured. A stamp measures the checkout when it is taken, after the report was written, and not the tree while the run executed: a run on a dirty tree that is cleaned before the stamp reads clean.

Refusals, all of them deliberate:

• nothing to record — no --suite, --crate-output or --coverage was given;
• not a git work tree with a commit — a run recorded there would carry no provenance and prove nothing;
• a malformed report line — refused rather than skipped, because a silently dropped case leaves a claim reporting not_run after a run that did run it, which is a lie in the safe direction and still a lie;
• a result for a test this checkout does not have — named in the recording's unknown list rather than recorded or dropped, because it means the runner and the matrix have diverged;
• a measurement of another commit — the report ran somewhere else: record it in a checkout of the commit it ran;
• a measurement of another report — its digest is not the report's;
• a measurement for a report not given, or keyed to another producer;
• a local or release measurement that names no report;
• a CI report without a measurement — a CI recording carries the measurement its own job made. A job's tree file, which names no report, is accepted for a CI recording until CI stamps with its report.

Every recording is also a typed run. evidence record returns an EvidenceRunRecord (its id, origin, commit, the weakest tree over the reports given, the measurement of each report, totals by outcome and runner, what was absent, dropped or unknown, the executions) and writes it with --run-record <file>. --coverage <summary> joins the summary scripts/rust-coverage --summary-json wrote, bound to the commit and the tree its run measured, with the floors the threshold files state; it is current only for its own commit, and only on clean trees. --ledger local merges into .ai/local/evidence/ledger.json, which the repository ignores: it changes no tracked file and moves no verdict. A run record never decides a verdict; a later surface lets one weaken a verdict only through the one monotone rule (freshness::weakened_by, which run records reach through evidence::weakened_by_records), which can withhold proven and never grant it.

A crate binary states only what ran. cargo test prints ok on the result line of a binary that ran nothing at all, so the recorder reads the counts, not only the word: a binary with a failed test is a failure whatever its word says; one that passed nothing (every test ignored, or none there) or ran only a subset a name filter chose is a skip, the absence of a proof rather than a proof; one whose result line carries no count, or that printed no result line before the next binary, is an error, because the harness could not say what ran. Colour codes are stripped from every line before it is matched, so a coloured run reads as a plain one. What the output held that no claim can name yet is listed, with the reason, in the recording's dropped list (a dropped line in the text output) rather than ignored: the crate's own unit-test binary (Running unittests src/lib.rs), its doctests (Doc-tests <crate>), a binary outside tests/, and a test result: line with no Running line before it, which is never credited to the previous binary.

In CI

Half the wiring is already there. The suite job of .github/workflows/validate.yml runs bash test/run.sh with MJ_TEST_REPORT: suite.tsv and uploads that TSV as the ci-metrics-suite artifact; the macos job does the same. The recording step is one line after the suite step:

- if: always()
  run: majordomus evidence record --suite suite.tsv --provenance suite=suite-tree.json --origin ci

What is not wired, and is a target rather than a description: nothing runs that step today, and a ledger written in a CI checkout is thrown away with the runner. Landing CI's evidence in the repository needs a commit, which a pull-request run must not make; the shape that works is a push-to-master job that records and commits the ledger in its own commit, the way the benchmark baseline is written. Until that exists, the ledger is written by whoever runs the suite locally and commits it, and the states are honest about the result either way.

The capabilities

One declaration in src/capability/builtin/evidence.rs; every surface is derived from it (CAPABILITIES.md).

These five are subcommands of the Rust executable, and majordomus is the one program an installed tool puts on PATH. The shell tool has a command of the same name, which records what a task owes (majordomus evidence --covers <token>, --gate <id>; CLI.md) and takes options only. So a word in the first position is never the shell tool's: the launcher hands show, claim, proves, record and stamp to the executable with every argument unchanged, the global --repo included on either side of the command, and the exit code is then the executable's. Which words those are is read from the executable's own declaration, never listed in the launcher, and majordomus evidence --help prints both halves. Through 0.22.0 the word reached the option parser instead, which answered evidence: unknown option record; case 2998_evidence_reaches_the_subcommands_the_executable_owns holds every line this page prints against the launcher, from a checkout and from an installed layout.

evidence show takes --state, --status and --findings as filters and --check, which exits 10 when any claim declares a guarantee the evidence does not support. The tallies are computed before the filter: a filtered answer says how much of the matrix it examined, never how much it returned.

None of them caches. The ledger is a file that changes outside the process, and a cached answer would be exactly the stale evidence this subsystem exists to name.

evidence.record is the only one that writes, and it is not reachable over the network. It declares a command line and nothing else. This executable's MCP and HTTP surfaces are read-only — a property of the server, not an accident of what has been built — so the recorder is a developer capability: offered to whoever runs the executable and to nobody over a socket. It is declared as a capability rather than hand-written as a command so that it stays inside the registry, the command graph and the generated reference. It is also waived from the benchmark, with the reason recorded: timing it in a loop would rewrite the repository's evidence.

evidence.stamp writes nothing, and it is command-line only too: it reads a report whose path its caller names, and a network client must not choose a file this executable reads.

Navigating in both directions

A relation that can only be walked one way is half a relation.

majordomus evidence claim distribution-canonical-model
majordomus evidence proves suite:84_distribution_model
majordomus evidence proves test/cases/84_distribution_model.sh   # the path resolves to the same test

evidence claim answers what proves this claim: the state and its meaning, the execution behind it, the paths that changed since, the command that reproduces it, and the other claims the same test proves — a test shared by twelve claims is a test whose failure is twelve findings, and a reader of one should be able to see the other eleven without searching.

evidence proves answers what does this test prove: its latest execution, whether its source still hashes to what that execution recorded, the command that runs it again, and every claim that names it with the state each is in.

Both read the same derivation, so the two directions cannot disagree. A claim the matrix does not declare is a not-found rather than an empty answer: a typo that read as "this claim has no evidence" is the one answer these capabilities must never give.

Subjects: what a verdict is about

A claim is one thing a reader asks about, and not the only one. A reader also asks whether a feature is proven, what tests a command, what proves a rule. Each of those is a subject: one declaration the repository already makes, named by a key <kind>:<id> — claim:<id>, rule:<id> (the rule's id without its version), feature:<id>, command:<path> (the words of a command joined by one space, command:commit plan), capability:<id>, mcp:<tool> and use_case:<id>. The subject index says, for every one of them, which subjects it is made of and which tests it reaches; nothing lists either by hand.

A command word is a public command of the shell tool, a documented example path of the executable or a capability's command-line path, and the subject names the programs it belongs to. Those are the words of the command graph (majordomus commands graph): every command of the executable that can be run carries a documented example and a group that only holds commands carries none, so a group such as evidence is not a word of the executable, and a case holds the command subjects and their programs equal to the graph's. A word both programs answer to is one subject naming both, with the advisory finding command_in_two_programs: its header routes prove the shell tool and its example route proves the executable. The members form a graph with no cycles — a use case never includes the features that name it, and nothing includes a feature — and a reference that resolves to nothing is skipped here and reported by the product validation.

A subject's verdict is the one aggregation the claims and the rules already use, over its parts: a failing part makes the subject failing; otherwise the verdict is the weakest of the parts that can carry proof; otherwise the weakest of all its parts; and a subject with no part reads no_test. The totals of its parts, by state, are always beside the verdict, so one not_run part is never hidden behind it. A subject is judged over its own routes and its members' own verdicts, never over its members' raw routes. It is judged at the presented revision, as evidence show --presented judges a claim: a claim route is the claim join's proof, a command's routes go through the same truth table and the same monotone rule over the working ledger's uncommitted run, and the rules report, which judges the working tree, is capped by the presented tree, so a rule-derived proven becomes inputs unchanged when the presented tree is not its commit, as every other route does. A surface that reads records beyond the tracked ledger may weaken a route before anything is aggregated, and never strengthen it: the route keeps the weaker of the two states.

A rule subject reads its rule proof's state through one declared mapping:

A scenario and a mechanism are listed so that a reader sees them, and neither ever decides a verdict: the ledger holds no scenario run, and a mechanism is a gate or a script that refuses violations rather than a run that passed. A feature whose members reach no test a runner drives carries the finding feature_without_evidence. The subject findings are advisory until the gate slice holds them.

The committed index, site/data/registry/evidence-subjects.json, is written by majordomus generate site and carries structure only: no ledger row, no commit and no time, so recording a run never makes it stale, and generate --check refuses a copy that differs from the derivation. Shell and jq only join it. It replaces or holds equal the shell derivations of subject to test that came before it:

• the command pages (scripts/generate-site-data) read the first covers and negative line of every case, as the index does, and a case holds the two equal until the pages join the index instead;
• the command-coverage doctrine (lib/commands.sh) reads the same first line;
• the command-furnished gate and the use-case impact trace match a command anywhere in any covers line between word boundaries, so they would read a second header of a kind, and a command inside a longer word (capability:commit.plan holds plan); a case holds every case to one header of each kind, and its covers line to naming the same public commands under both readings, so they cannot yet disagree; the gate slice moves them onto the index;
• the capability pages (scripts/lib/executable-site.jq) give every capability the claims implemented in its module's file, and a case holds them equal to the index;
• the doctrine pages name the first test of each rule, and a case holds every one of them to a route or a mechanism of that rule.

jq '.subjects["feature:evidence"]' site/data/registry/evidence-subjects.json
majordomus generate site --check

ADR 0087 (proposed) records the decision to index evidence by subject.

The gate

scripts/evidence-check renders the executable's own answer and ratchets it. It derives nothing of its own — a gate with a second opinion about what is proven would be a second model of proof.

scripts/evidence-check              # report; 0 clean, 10 with regressions
scripts/evidence-check --json       # the report, from the capability itself
scripts/evidence-check --strict     # every unsupported guarantee fails; the baseline is ignored
scripts/evidence-check --baseline   # rewrite the baseline from what is found now
scripts/evidence-check --repo PATH  # judge that repository instead of this one

Only guaranteed claims are judged. advisory states that enforcement is not observable from outside, planned that nothing implements it, and rejected that nothing will; demanding a current proof of those would be demanding proof of a thing the claim already says is not there. A guaranteed claim is supported by proven or inputs_unchanged, and by nothing else; stale, failing, not_run, unrunnable and no_test are findings, each with the reason and the command that would settle it.

stale is the one of those that looks like support and is not, and it was counted as support until it was measured: 14 guarantees rested on it, against 0 proven. A stale run is a pass — passing() says so, and the summary must keep it apart from a failure — but it is a pass of a subject that has since moved: the claim's implementation or its test changed after the run offered as its proof, so what passed is not what the claim now names. Accepting it made the gate accept, as support for a guarantee, a measurement of something else, which is the whole defect this subsystem exists to name one level down. The floor is inputs_unchanged: weaker than proven, but at least a pass of this subject.

It is advisory today, and this is a choice with an end. The ledger starts empty, so on the day the gate arrives every guaranteed claim is not_run — true, and as a blocking gate it would mean nothing could be merged by anybody until a full suite run had been recorded. So the debt that existed at that moment is written to .ai/repo/evidence-baseline.txt by --baseline, never by hand.

What the ratchet refuses is not membership of that list. It was, and that was a defect rather than a strictness: a claim written after the baseline is absent from it for the only reason a claim can be absent from a snapshot of the past — it did not exist yet — and the gate reported every such claim as having "lost the evidence that supported them", evidence it had never had. Every newly merged guarantee reddened the trunk for every branch that merged after it. A ratchet whose alarm fires on growth rather than on regression measures the wrong thing: it taxes writing claims down, which is what the matrix exists to reward.

So the baseline records the state per claim — both halves, a + line for a guarantee a run supported and a bare line for one no run supported — and two things are refused:

And one thing is deliberately not refused: a guaranteed claim the baseline never knew, arriving with no recorded run. That is new debt — reported, counted, and not a regression — because refusing it is the broken behaviour above. It is admitted to the baseline by --baseline, in its own commit, with the reason, and the ratchet holds it from then on. The limit is stated rather than hidden: a claim promoted from advisory to guaranteed without a run is new debt too, and is also only reported.

It still tightens in one direction only. The unsupported half shrinks by recording a run that proves a claim and rewriting the file — never by adding a line. --strict refuses every unsupported guarantee; it is the end state, and CI switches to it when the unsupported half is empty.

The gate is named in .ai/repo/ci/gates.yaml as evidence-check, in the structure job, with always: true — so the verdict arrives on every push rather than when somebody remembers to run it, which is the other half of project.never-reported-is-not-green. The evidence path class names what can change the answer: the ledger, the baseline, the gate script, the claims matrix, the generated claim pages and this document.

proven is rare here, and that is the model working

The ledger is a tracked file, and recording writes it into the tree. So a recording made at HEAD leaves the tree dirty, and committing the ledger moves HEAD past the commit the executions name — so a rule that asked for a clean tree at HEAD would never be satisfied here at all.

That is why the rule is the diff rather than HEAD, and why the ledger's own row is excluded from it. proven is reachable in a repository that tracks its own evidence, which is the only arrangement in which the evidence is worth anything to anybody but the person who ran the tests. test/cases/124_evidence.sh proves it on a fixture whose ledger is tracked exactly as this repository's is.

Recorded in CI

The ledger above holds whatever was recorded into the tree. CI's runs are recorded as well, and a validating run never commits its rows: they are kept where the run happened, as a run record. A trunk run's rows are committed later, by a separate pull request (ADR 0080), because a run cannot commit what it proved without adding a commit after the one it proved, on a trunk that moves faster than the suite finishes. That recording pull request is not wired yet. The decision is ADR 68, as ADR 0087 amends it.

flowchart LR
  suite["suite job, four shards<br>suite.tsv · suite-tree.json, joined"]
  crate["rust job, three lanes<br>cargo-test-1..3.txt · crate-tree.json, joined"]
  cov["coverage job<br>coverage.json"]
  collect["evidence job<br>scripts/ci/evidence-collect"]
  artifact["artifact `evidence`<br>report · ledger · coverage · manifest"]
  pages["pages.yml<br>scripts/pages evidence"]
  site["/evidence/<br>current · stale · unknown · unavailable"]
  suite --> collect
  crate --> collect
  cov --> collect
  collect --> artifact --> pages --> site
  collect -. "commit still the tip: dispatch" .-> pages

What the collector derives first. Before it records anything, scripts/ci/evidence-collect measures the checkout it runs in and derives the report through majordomus evidence show, as report.txt and report.json. The reports it is handed and the directory it writes are under the runner's temporary directory, outside that checkout. The report is therefore the tracked ledger's verdict at the run's commit, the answer a clean checkout of that commit gives, and the manifest's report_tree says whether the checkout was clean when it was derived.

What the run's rows are. Only then does it record the suite's and the crate's results through majordomus evidence record --origin ci, and keep the ledger that now holds them as ledger.json. Those rows are a run record. They are counted in the manifest and decide no verdict, in the artifact or on the site. A trunk run's rows become verdicts only when ADR 0080's recording pull request lands them in the tracked ledger. The collector also summarises the coverage export through scripts/rust-coverage --summary-json.

What the producers and the recorder measure. Each job that ran tests measures its own checkout right after its run and hands the measurement over beside its report: suite-tree.json from the suite job, crate-tree.json from the rust job. Each excludes by name the outputs its own run names, at the paths where that run writes them, and nothing else: the suite its report, at the root; the rust job its timings and its artifact directory, which scripts/rust-check writes in the crate's directory, where it runs. The measurement lists what it excluded. The recorder refuses a CI report without the measurement its own job made, so the collector passes each job's tree file as that report's provenance, and each runner's rows carry its own job's measurement (the manifest keeps the rows' trees as rows_working_tree). The manifest's totals, its working_tree (the weakest tree over the recorded reports) and the suite and crate words of absent are read from the recording's run record rather than computed a second time; it is never read from the recorder's checkout, which is clean by construction. A report whose job measured a commit other than the one the collector runs on, or whose job left no measurement, is refused: it is not recorded, and it is named.

The collector writes manifest.json, naming:

• the commit that ran (for a pull request, GitHub's merge commit), and head_sha, the head it was built from, carried beside it and never in its place;
• the event and the run;
• the outcomes of that run's executions;
• the producers' measurements, the run record's working_tree, and the report_tree;
• which reports were absent, a report that yielded no execution included, and which were refused, with the reason.

An absent report is named, never counted as a pass. The job keeps the directory as the artifact evidence. It is not a gate: it reads jobs that have already decided, and a red suite is exactly the evidence worth keeping.

An execution names its run. A ci recording made inside a GitHub Actions environment stamps every execution with run: the provider, the run's identifier, its attempt, the workflow, the job and the address the provider gave. RunRef::from_env is the only place that knows the environment's names. A local recording names no run, even inside a CI shell, and rows recorded before runs were named have no run at all.

"run": {
  "provider": "github_actions",
  "id": "<run id>",
  "attempt": 1,
  "workflow": "validate",
  "job": "evidence",
  "url": "https://github.com/korczis/prismatic-majordomus/actions/runs/<run id>/attempts/1"
}

What a publication says about it. Before the build, scripts/pages evidence finds the evidence artifact recorded against the commit being published or its nearest ancestor. It chooses the nearest ancestor by history, not the newest upload. It writes site/data/evidence.json, which is never committed. It publishes the report as it was derived, and reads the manifest only to decide whether the run confirms it: a reason can withhold current, and never changes the report. ADR 0087 defines the states it says:

Whatever withholds current is listed under the sentence. None of the states stops a publication. What does stop one is the UI snapshot showing tests as not run (I2786):

• The run's ledger, in place. The snapshot job first puts the same artifact's ledger.json in the checkout's ledger path (--ledger-out), so the screens judge the run's rows at the published commit. This is ADR 0087's 2026-10-17 amendment, until the recording branch exists.
• The check. scripts/pages published-evidence then asks evidence.report and refuses with exit 10 when more claims read not run than .ai/repo/evidence/published-not-run.txt declares. Rule project.published-evidence-is-what-ci-recorded holds this. When a master run's evidence is kept while its commit is still the tip, the job dispatches pages.yml, and that publication says current when the run confirms the report. When master has moved on, the newer publication already carries the evidence as stale and the newer run will refresh it.

scripts/pages evidence                          # this commit, fetched with gh
scripts/pages evidence --from <dir> --out FILE  # a gathered directory, offline
scripts/pages evidence --from <dir> --out FILE --ledger-out .ai/repo/evidence/ledger.json  # and its ledger, in place
scripts/pages published-evidence --bin <majordomus>  # refuse a UI that would show tests as not run
cat cargo-test-*.txt > cargo-test.txt   # the rust job's three lanes, joined
scripts/ci/evidence-collect --out <dir> --suite suite.tsv --suite-tree suite-tree.json \
  --crate-output cargo-test.txt --crate-tree crate-tree.json --coverage coverage.json

The behavioural proof is test/cases/357_ci_records_evidence.sh.

What this is not

It is not tamper-proof, and it does not pretend to be. The ledger is a tracked file. A person can edit "outcome": "pass" into it, and what a reviewer sees is the diff. What the recorded digest buys is staleness detection that survives a revert — an execution whose test source no longer hashes to the recorded digest did not run the test that is there now, whatever the commit says. Cryptographic ceremony against someone who can already commit to the repository would be ceremony with no threat model. The honest security property is the one this repository relies on everywhere else: a fabricated result is a reviewable line in a commit rather than an untraceable green.

Note the exact reach of that digest: it is reported by evidence proves as digest_matches and is not an input to the proof state, which is decided by the git comparison. A tree where the two disagree is a tree where something was recorded against a commit that does not describe it, and the working_tree: dirty field is usually the reason.

It is not the execution control plane. src/execution/ and EXECUTIONS.md are about capability executions of a running server — in-flight work, progress, typed events, gone with the process. This is about test runs: durable, committed, and read long after the process that produced them exited. Two different subjects that share an English word.

It is not the obligation evidence of lib/evidence.sh. That subsystem (ADR 30) is about what a task owes before it may be called completed, and its ledger is the append-only task ledger. The two share an idea — evidence that can go stale — and nothing else. Neither reads the other's records.

It does not judge results. Nothing here re-runs a test, re-interprets an outcome, or decides that a failure was environmental. The recorder records what the runner said.

It does not make a claim true. A claim with a proven state is a claim whose test passed against this commit. Whether the test tests the claim is a question for review, and project.no-claim-without-test is the rule that holds it.

When something fails

Stability