test-evidence — Every verifiable claim and widget resolves to revision-bound test evidence, on every surface

•ACTIVE 16 of 32 issues done

ready

4

blocked

12

active

0

verify

0

done

16

cancelled

0

Problem

Claims bind to tests by path, and only `majordomus evidence show` asks whether a claim is proven, always at the working tree of whoever runs it. The site cannot ask that question about the commit it was built from, and no widget on it names the subject that would prove what it shows: where a page says verified, the word was placed by hand. CI runs the suite and the crate tests on every merge and keeps none of it in the ledger. A test that declined to run reads `failing`, while a case that declines by exiting zero is recorded as a pass. The rules module holds a second copy of the freshness derivation, the entry preflight proposed in PR #578 counts a run as current by a filter of its own with no containment test, and ADR 0068 and ADR 0080 disagree about where CI evidence goes. Each surface that wants to say verified derives it differently or not at all.

Outcome

Evidence is indexed by subject and judged at the presented revision, as ADR 0087 records. Every claim, rule, feature, command, capability, MCP tool and use case is a subject of one derived index, and every widget that states behaviour names one. Every surface reads its verdict from the tracked ledger alone, through one freshness function and one aggregation, at the revision it presents: the CLI and HTTP and MCP at the working tree or the checked-out commit, the site at the commit it was built from. A trunk run reaches the ledger through a reviewed recording pull request, and a supplementary record (a run record, coverage, a local row) can only weaken a verdict, naming itself when it does. Verified is rendered only for the one proven row of the freshness table, on every surface.

Current state

On master at 5a0770c437 the tracked ledger holds local recordings only, and how many claims reach each state is what `majordomus evidence show --format json | jq .totals` prints. There is no subject index; the command-to-test join is derived in shell more than once (`scripts/generate-site-data`, `lib/commands.sh`, `scripts/ci/command-furnished`). `rules/mod.rs` carries its own `changed_since` and its own state mapping beside `evidence/mod.rs`. The decisions this milestone builds on are proposed and unlanded: ADR 0068 on PR #577 (CI evidence as an artifact and a published dataset), ADR 0080 on PR #629 (the recording branch, and the skip word), ADR 0071 on PR #600 (delivery dimensions, with a private containment implementation); PR #597 adds the rules corpus verdict and PR #578 the entry preflight.

Desired state

A reader of any page of the site follows a widget's badge to the evidence of its subject and sees the verdict at the commit the page was built from, with the run that produced it and every record that weakened it. `majordomus evidence subject <key>` answers the same question from a terminal, over HTTP and over MCP, and the Cockpit shows the same verdict. A merge to master is followed by a recording pull request that lands its run's executions in the ledger. A claim then reads `proven` at a trunk commit that nothing but ledger-only commits separates from the recorded one, and `inputs_unchanged` or `stale`, naming what changed, where other merges landed in between; whether the process makes the first the rule is the owner's decision (ADR 0087, Consequences). A skipped test reads `not_run` with its reason, and no surface anywhere reads greener than the ledger for the revision it presents.

Scope

• .ai/repo/adrs
• .ai/repo/project
• .ai/repo/evidence
• .ai/repo/evidence-baseline.txt
• .ai/repo/features/evidence.md
• .ai/repo/ci
• apps/majordomus-cli/src/evidence
• apps/majordomus-cli/src/rules
• apps/majordomus-cli/src/git
• apps/majordomus-cli/src/capability/builtin/evidence.rs
• apps/majordomus-cli/src/commands/evidence.rs
• apps/majordomus-cli/src/cockpit
• apps/majordomus-cli/src/environment/preflight.rs
• apps/majordomus-cli/src/redaction.rs
• .github/workflows
• scripts/ci
• scripts/pages
• scripts/generate-site-data
• scripts/evidence-check
• share/design/tokens.yaml
• site/templates
• site/data/registry/evidence-subjects.json
• docs/EVIDENCE.md
• docs/CLAIMS.yaml
• docs/claims
• test/cases

Out of scope

• Accepting ADR 0041, 0068, 0080 or 0087; acceptance is the owner's act
• Bumping the version; a pull request that moves the public surface inherits the version already on master by merging it
• Judging a commit other than the checked-out one or the site's build commit
• MCP resource templates per subject; evidence is reached through tool calls
• Making the required `ci` check depend on the evidence job
• New shell automation (ADR 0069)
• A regex crate or any other new dependency for the redactor
• Stamping use-case evidence with a commit in the committed catalogue

Acceptance criteria

• Model: one freshness function, one presented revision, one containment and one aggregation judge every verdict, a skip reads `not_run`, and one derived index names every subject; proven by test/cases/502_evidence_is_judged_at_the_presented_revision.sh, test/cases/507_evidence_subjects_are_derived.sh and test/cases/501_verdict_words_are_status_words.sh
• Producer: every recorded run carries the commit and tree it measured, a crate binary that ran nothing is a skip, CI's report is the tracked ledger's verdict derived before the run records, and a trunk run reaches the ledger through one recording pull request, stopping at a commit the ledger covers and recording a ledger-only commit over an unrecorded merge; proven by test/cases/505_a_recording_carries_what_its_run_measured.sh, test/cases/506_a_crate_binary_that_ran_nothing_is_not_a_pass.sh, test/cases/357_ci_records_evidence.sh, test/cases/510_ci_evidence_reaches_the_ledger_through_a_recording_branch.sh and test/cases/511_every_runner_report_carries_its_revision_and_reason.sh
• Surfaces: the CLI, HTTP, MCP, the Cockpit and the entry preflight answer by subject from the one authority, and a supplementary record only weakens; proven by test/cases/508_evidence_answers_by_subject.sh, test/cases/509_supplementary_records_only_weaken.sh, test/cases/513_cockpit_shows_evidence.sh, test/cases/514_every_test_line_is_read.sh, test/cases/515_a_local_run_stays_local.sh and test/cases/529_the_preflight_reads_the_one_verdict.sh
• Site: every widget names its subject, every subject has an evidence route, the publication is bound to its build and allow-listed, and published text is redacted once; proven by test/cases/504_public_text_is_redacted_once.sh, test/cases/512_evidence_publication_is_a_projection.sh, test/cases/516_evidence_routes_render_from_the_subject_index.sh, test/cases/517_a_panel_subject_is_joined_from_its_route.sh, test/cases/518_evidence_publication_is_bound_to_its_build.sh and test/cases/520_every_widget_names_its_proof.sh
• Policy: subject findings refuse, the evidence baseline is rewritten in the same change, and the validator reads every widget; proven by test/cases/522_evidence_check_reads_widgets.sh and test/cases/523_evidence_rules_are_gated.sh
• Tests: no surface is greener than the ledger, the badge works in a browser at phone width, tampered evidence is refused, and coverage is shown with its commit; proven by test/cases/524_no_surface_is_greener_than_the_ledger.sh, test/cases/525_an_evidence_badge_works_in_a_browser.sh, test/cases/526_tampered_evidence_is_refused.sh and test/cases/527_coverage_is_shown_with_its_commit.sh
• Docs: the evidence reference names what the code declares and is held to the tree; proven by test/cases/528_the_evidence_reference_is_held_to_the_tree.sh
• Deploy: a deploy verifies the evidence routes it publishes, and the evidence survives artifact expiry on gh-pages; proven by test/cases/521_deploy_verifies_evidence_routes.sh, test/cases/519_evidence_survives_artifact_expiry.sh and the live page https://majordomus.dev/evidence/

Validation

• bin/majordomus adr check
• bin/majordomus plan validate
• bin/majordomus-cli evidence show --check
• scripts/evidence-check
• scripts/ci/claim-proof-check --strict
• bash test/run.sh
• cargo test --workspace --all-features
• majordomus generate --check
• scripts/ci/pages-check

Evidence required

• one_authority_one_freshness
• subjects_derived_once
• runs_measured_where_they_ran
• no_surface_greener_than_the_ledger
• every_widget_names_its_subject
• publication_bound_to_its_build

Risks

• Until the recording branch runs, the tracked ledger holds only local rows and the site renders mostly `not_run`. That is the true state; the badges follow the CI wiring for that reason.
• The decisions this milestone builds on are proposed on open pull requests (#577, #578, #597, #600, #629). A slice stacked on one of them waits for it, and the landing order is the owner's.
• `proven` at a trunk revision needs a quiet trunk: a recording costs a full suite run and then the required checks, and a code merge landing in that window leaves claims `inputs_unchanged` or `stale` at the landing commit. Until the owner decides whether the process makes `proven` reachable or `required_tests_current` binds to a reachable state, on a busy trunk the delivery dimension rarely passes and the site rarely renders verified (ADR 0087, Consequences).
• The recording pull request needs a token whose events start the required workflow. Until the owner chooses between dispatching validate.yml and an App token, the CI wiring slice cannot land.
• Every landing makes the open slices' pull requests conflict on derived files; each is refreshed by merging master with the derived merge driver, never by a rebase.

Issues

Each one is an execution contract with its own acceptance criteria, validation command and required evidence. The wave column is the layer of the dependency graph it sits in.

The graph

flowchart LR
    I1945["I1945<br/>The test-evidence decision and its plan are recorded"]:::done
    I1946["I1946<br/>The verdict words are status words of the design system"]:::done
    I1947["I1947<br/>Every feature reaches a test, and every use case has a feature"]:::done
    I1948["I1948<br/>Evidence is judged at the presented revision, by one freshness function"]:::done
    I1949["I1949<br/>CI evidence is recorded clean, and its report is the tracked ledger's verdict"]:::done
    I1950["I1950<br/>Text a run printed is redacted by one rule set"]:::done
    I1951["I1951<br/>Evidence runs are typed, and a recording carries what its run measured"]:::ready
    I1952["I1952<br/>Evidence subjects are derived once, in Rust"]:::done
    I1953["I1953<br/>Evidence answers by subject, on every surface"]:::blocked
    I1954["I1954<br/>Every runner reaches the ledger, through the recording branch"]:::blocked
    I1955["I1955<br/>The evidence publication is an allow-listed projection"]:::blocked
    I1956["I1956<br/>The Cockpit shows evidence by subject"]:::blocked
    I1957["I1957<br/>Every test line is read, and a local run stays local"]:::blocked
    I1958["I1958<br/>Evidence has one route per subject, and every panel is joined to its subject"]:::blocked
    I1959["I1959<br/>The evidence publication is typed, bound to its build, and kept on gh-pages"]:::blocked
    I1960["I1960<br/>Every widget names its proof"]:::blocked
    I1961["I1961<br/>A deploy verifies the evidence routes it publishes"]:::blocked
    I1962["I1962<br/>The evidence gates read subject findings and widgets"]:::blocked
    I1963["I1963<br/>Evidence is tested end to end"]:::blocked
    I1964["I1964<br/>The evidence reference is documented and held to the tree"]:::blocked
    I1965["I1965<br/>The entry preflight reads the one verdict"]:::ready
    I2282["I2282<br/>The record pull request's dispatched verdict does not satisfy branch protection"]:::done
    I2296["I2296<br/>The executor reopens a record pull request no run was started for"]:::done
    I2298["I2298<br/>A pull request stacked on one that landed merges on the verdict its head already has"]:::done
    I2303["I2303<br/>A release publishes its metadata to the site before its record lands, and the site never goes back a release"]:::done
    I2396["I2396<br/>The CI critical path of 2026-10-10 is measured, recorded, and explained before anything is changed"]:::done
    I2416["I2416<br/>The ci verdict asks every selected gate for a receipt, and a gate whose step never ran is red"]:::done
    I2786["I2786<br/>The published UI reads the evidence CI recorded, and a publication that would show tests as not run is refused"]:::done
    I2795["I2795<br/>The trunk's recorded run reaches the tracked evidence ledger, and the baseline is tightened to it"]:::done
    I2845["I2845<br/>a run cancelled midway is not the publication's evidence while a complete one is near"]:::ready
    I2853["I2853<br/>a Pages deploy on a fresh master is not refused for claims newer than the ledger it reads"]:::done
    I3013["I3013<br/>The evidence job runs only when a planned lane ran tests, so a records-only pull request is not red"]:::ready
    I1946 --> I1958
    I1946 --> I1960
    I1947 --> I1952
    I1948 --> I1951
    I1948 --> I1952
    I1948 --> I1965
    I1949 --> I1951
    I1950 --> I1955
    I1951 --> I1953
    I1951 --> I1954
    I1952 --> I1953
    I1952 --> I1958
    I1952 --> I1962
    I1953 --> I1955
    I1953 --> I1956
    I1953 --> I1962
    I1954 --> I1959
    I1955 --> I1957
    I1955 --> I1958
    I1955 --> I1959
    I1956 --> I1963
    I1958 --> I1960
    I1958 --> I1961
    I1959 --> I1960
    I1959 --> I1961
    I1959 --> I1963
    I1960 --> I1962
    I1960 --> I1963
    I1962 --> I1964
    I1963 --> I1964
    classDef done stroke:#16a34a,fill:#dcfce7,color:#14532d,stroke-width:2px
    classDef active stroke:#2563eb,fill:#eff6ff,color:#1e3a8a,stroke-width:2px
    classDef verify stroke:#7c3aed,fill:#f5f3ff,color:#4c1d95,stroke-width:2px
    classDef ready stroke:#0891b2,fill:#ecfeff,color:#164e63,stroke-width:2px
    classDef planned stroke:#0891b2,fill:#ecfeff,color:#164e63,stroke-width:2px
    classDef blocked stroke:#b45309,fill:#fffbeb,color:#78350f,stroke-width:2px
    classDef cancelled stroke:#6b7280,fill:#f9fafb,color:#374151,stroke-width:2px
    classDef superseded stroke:#6b7280,fill:#f9fafb,color:#374151,stroke-width:2px

Canonical record: .ai/repo/project/milestones/test-evidence.yaml. Read it back with majordomus plan show test-evidence.