I3013 — The evidence job runs only when a planned lane ran tests, so a records-only pull request is not red
validate.yml's evidence job runs when the plan selected a lane that runs tests and leaves a report (the shell suite, or the crate's tests under rust_check) and that lane ran, instead of whenever the rust job succeeded. A records-only plan, whose rust lanes only build the executable and run the registry checks (rust_integration), then skips the evidence job rather than failing it at evidence-collect's exit 12.
READY wave 0 · p1 · implementation profile · parallel safe
Ready. Every dependency is done, so majordomus plan start I3013 will be accepted.
Objective
validate.yml's evidence job runs when the plan selected a lane that runs tests and leaves a report (the shell suite, or the crate's tests under rust_check) and that lane ran, instead of whenever the rust job succeeded. A records-only plan, whose rust lanes only build the executable and run the registry checks (rust_integration), then skips the evidence job rather than failing it at evidence-collect's exit 12.
Why
On 2026-10-20 the records pull requests #1745, #1746 and #1749 (validate run 39483394648, job 118855710606) all ended with evidence red: 'evidence-collect: no report to record: the suite and the crate both left none', exit 12 in the gather step. Their plan ran rust-check --integration in lane 3, which writes no cargo test output, ran nothing in lanes 1 and 2, and skipped the suite. The job's condition (validate.yml, the evidence job's if) fires on needs.rust.result == success, so it ran with nothing to record and failed. A red check that says nothing about the change trains everyone to read red as noise, and the landing seat has to explain it on every records batch.
Current state
The evidence job's if is always() && (suite succeeded or failed, or rust succeeded or failed). rust succeeds on an integration-only plan, which runs no test, so the job runs and scripts/ci/evidence-collect exits 12.
Desired state
The evidence job's if names the plan outputs that make a lane leave a report (shell_suite, rust_check) together with that lane's result. A plan that selects neither skips the job. evidence-collect keeps refusing a run where a planned lane should have left a report and left none.
Scope
- .github/workflows/validate.yml
- test/cases/3402_a_records_only_plan_skips_the_evidence_job.sh
- .ai/repo/project/issues/I3013.yaml
- site
Out of scope
- Making evidence-collect pass with nothing to record: a planned lane that left no report stays a failure
- The rust lanes' modes and what rust-check --integration runs
Dependencies
None. This issue is a root of the graph.
Acceptance criteria
- The evidence job's if requires that the plan selected shell_suite or rust_check and that the matching job succeeded or failed
- A case evaluates the job's condition for a records-only plan (shell_suite false, rust_check false, rust_integration true, rust success) and finds it false, and for a suite or crate-test plan finds it true
- A records-only pull request's validate run shows evidence skipped, not failed
Validation
- bash test/run.sh 3402_a_records_only_plan_skips_the_evidence_job
- bash test/run.sh 26_ci_wiring
- majordomus plan validate
- scripts/derive-check
Evidence required
- records_only_plan_case
Evidence
None recorded. Every token above needs a command or an artifact behind it before this issue can be completed; narrative is refused.
Timeline
- started
- —
- verified
- —
- completed
- —
Those three fields, the evidence above and the state of the dependencies are all the status is made of. There is no status field to disagree with them.
Canonical record: .ai/repo/project/issues/I3013.yaml. Read it back with majordomus plan show I3013.