Skip to content

Coverage of the Rust crate is measured with test code out of the denominator, on every change that can affect the crate and on every push to master, and the build fails under the floor in scripts/rust-coverage-threshold or under the session/continuity domain's own floor in scripts/session-coverage-threshold

The crate's coverage is a number with a committed floor, and a push that drops under the floor does not go green. Two floors, each one integer in a file of its own: scripts/rust-coverage-threshold over the whole executable, and scripts/session-coverage-threshold over the canonical session/continuity domain declared in scripts/session-coverage-domain. Both are read by every place that measures — the coverage job in CI, scripts/rust-check when cargo-llvm-cov is installed, and just coverage — so there is no second copy of either number to drift.

declared statusdeclared status: guaranteed Declared deterministic and blocking, implemented, and naming the behavioural test that is to prove it. Whether a recorded run supports it is the evidence verdict, shown beside it.

recorded evidencerecorded evidence: not run Declared guaranteed, and not supported by a recorded run: the test is named, and no passing run of the current inputs is recorded. how evidence is recorded

cargo-llvm-cov over all targets, exported as JSON and post-processed so that tests/, benches/, examples/ and every inline `#[cfg(test)]` item leave the fraction on both sides. --ignore-filename-regex cannot do the second, which is why this is a script and not a flag. Two floors, each one integer in a file of its own, each read by CI, by scripts/rust-check and by just coverage, and each a ratchet — the coverage measured when it was set, which may rise and never fall; the crate floor's bound is FLOOR_BOUND in scripts/ci/rust-command-check (rule project.rust-command-tested-in-file) and case 77 reads it there. The domain the second measures is declared in scripts/session-coverage-domain and the script refuses a domain naming a file the export does not carry, so narrowing it to raise the number is a visible edit. Branch coverage proper needs a nightly toolchain; region coverage stands in for it and is reported under that name. The coverage job runs when the plan selects the rust-coverage gate, which the rust and share classes of .ai/repo/ci/gates.yaml select and every full plan does.

What it means

The crate's coverage is a number with a committed floor, and a push that drops under the floor does not go green. Two floors, each one integer in a file of its own: scripts/rust-coverage-threshold over the whole executable, and scripts/session-coverage-threshold over the canonical session/continuity domain declared in scripts/session-coverage-domain. Both are read by every place that measures — the coverage job in CI, scripts/rust-check when cargo-llvm-cov is installed, and just coverage — so there is no second copy of either number to drift.

The measurement excludes test code. That is the part worth stating out loud, because for a long time it did not.

How it works

scripts/rust-coverage runs cargo llvm-cov --all-targets --no-fail-fast --json once and post-processes the export. A file under tests/, benches/ or examples/, and build.rs, is dropped whole; inside a src/ file, every #[cfg(test)] item is dropped by line range, brace-matched from its opening line to the brace that closes it, and so is every #[test] and #[bench] function. What remains is the code under test, and the fraction is computed over that on both sides.

The previous gate was cargo llvm-cov --all-targets --summary-only --fail-under-lines "$(cat scripts/rust-coverage-threshold)", with no --ignore-filename-regex. --all-targets instruments the integration suites and the benchmarks as source files of their own, and this crate keeps its unit tests inline as #[cfg(test)] mod tests inside src/*.rs, where no filename regex can reach them. Test code is nearly always fully executed by the run that measures it, so every test written raised the percentage whether or not it covered anything and every test deleted lowered it. A threshold over that mixture cannot be reasoned about, and the obvious way to hold it is to write tests of the tests.

The second floor exists because a crate-wide number says nothing about any particular subsystem. The session and continuity domain — the code a repository's memory of its own work depends on — can sit at zero while the crate-wide figure stays green. scripts/session-coverage-domain names the files it covers, one per line, and the script refuses a domain that names a file the export does not carry: narrowing the domain to raise the number is an edit a reviewer sees rather than a silent change of subject.

Both floors are ratchets: each is the coverage its subject measured when it was set, a lower bound that may rise and never fall. The crate floor's bound is FLOOR_BOUND in scripts/ci/rust-command-check, the gate of the blocking rule project.rust-command-tested-in-file, which refuses a floor under it; the domain floor's bound is held in the case below. Neither floor stands above what the crate has measured, because a floor no run meets fails every push alike and says nothing about any change; new uncovered code is refused regardless by the changed-code gate, project.new-code-is-covered.

test/cases/77_rust_evidence.sh checks that both files hold one integer, that neither is under its bound, that the coverage job runs the script rather than calling cargo llvm-cov --summary-only again, that scripts/rust-check and just coverage do the same, and that every file the domain names exists.

How to see it

cat scripts/rust-coverage-threshold scripts/session-coverage-threshold
cat scripts/session-coverage-domain
scripts/rust-coverage --report        # measure and print, hold nothing
just coverage                         # cargo-llvm-cov: cargo install cargo-llvm-cov; rustup component add llvm-tools-preview

The report prints lines, functions and regions for the crate and for the domain, and, beside the crate's line figure, the figure the old gate measured — the same run counted with the test code left in — so that the difference between the two is visible rather than asserted.

What it does not cover

Coverage is measured on Linux in CI, once per run that selects the rust-coverage gate (the rust and share classes of .ai/repo/ci/gates.yaml, and every full plan); a change that cannot reach the crate does not recompile it with instrumentation. Branch coverage proper is not measured: cargo llvm-cov --branch needs a nightly toolchain and this repository builds on stable, so region coverage is what stands in for it and the report says regions rather than branches. The crate-wide floor is on lines only; the domain floor is on lines, functions and regions together. A file under the crate floor is not refused as long as the crate as a whole is above it.

The shell half of the tool — lib/*.sh, which owns the session lifecycle, the ledger and the record stores — is not in this measurement and has no floor. scripts/shell-coverage measures it, exactly for functions and as a stated floor for lines, and is an instrument rather than a gate; the reason it is not one is in its own header.

Why it exists

The operator asked for coverage near 100 percent and for that to be enforced for the future. A threshold that counts the tests in its own denominator answers a different question from the one that was asked: it says how much of the crate plus its tests ran, which rises when tests are added and falls when they are deleted, whatever they cover. Subtracting the test code makes the number mean what it says, and it costs something — the honest figure is lower than the one the old gate printed. The rule project.rust-cli-evidence states the requirement; this claim ties the numbers to the gate.

Detail rendered from docs/claims/rust-coverage-floor.md.

From the statement to the proof

  1. 1 · the public statement

    Coverage of the Rust crate is measured with test code out of the denominator, on every change that can affect the crate and on every push to master, and the build fails under the floor in scripts/rust-coverage-threshold or under the session/continuity domain's own floor in scripts/session-coverage-threshold

    Made on Benchmarks.

  2. 2 · the canonical claim

    rust-coverage-floordeclared status: guaranteed in docs/CLAIMS.yaml

    Defined in .ai/repo/rules/project/rust-cli-evidence.v1.md

  3. 3 · the implementation

    scripts/rust-coverage

  4. 4 · the test and the scenarios

    Test: test/cases/77_rust_evidence.sh

  5. 5 · the recorded runs

    recorded evidence: not run how evidence is recorded

Verify it yourself

The test runs in a disposable temporary repository and asserts the behaviour, not a string in the source.

from a clone of the repository
bash test/run.sh 77_rust_evidence

Related claims same implementation

Searches the titles and descriptions of every page this site publishes.