Skip to content
Cockpit0.29.0
A snapshot: read at commit 707a04c7ce8 on 2026-10-18T00:06:01Z by majordomus 0.29.0. Nothing on this page is read again. See it live

I0826

bench --check refuses a regression by policy

Where it stands

devtask.issue
Readiness
complete completion is recorded and every required evidence token is present
Status
DONE
Can start
not now
Milestone
M004
Milestone status
ACTIVE
Wave
4
Waiting on
none
Holds back
I0830I0831I0835I0838
Behind it in all
5
Evidence
1 of 1

Moves

plan.transition
A move writes the repository: it stamps one field of this issue's record and appends one event to the ledger, and a commit will carry both.
The same moves from a terminal

majordomus plan start I0826

majordomus plan verify I0826

majordomus plan done I0826

What it is

Objective
Compare a fresh run against the baseline under the policy's benchmark.regression thresholds (relative p50, p95, p99) and exit non-zero naming the command, the metric, both values and the threshold.
Why
Without a gate the next slow validator arrives unnoticed, exactly as the current ones did.
Now
No comparison exists.
Wanted
Deterministic report: metric, baseline, current, absolute and relative delta, threshold, result; incompatible baselines are reported as not comparable rather than as a number.
Done when
unknown: the record declares no `completion`
Risk
Thresholds live in the policy and are read like every other policy key: declared, no reader-side default.

Acceptance

  • A baseline edited to make the current run a regression makes the check exit non-zero with the named metric
  • A compatible run within thresholds passes
  • A baseline with another schema version is not comparable
Validation
bash test/run.sh 82_bench_check
Evidence required
regression_refused
Evidence present
regression_refused

Scope

Touches
lib/bench.shshare/allow/policy.txtshare/skeleton/policy.yaml.ai/repo/policy.yamltest/cases/82_bench_check.sh
Not
Significance testing
Depends on
I0825
Priority
p0
Profile
implementation
Parallel safe
false
Owner
unknown: the record declares no `owner`

The work so far

Branches
none
Merged
none
Commits
0
Trunk
master
Sessions
none
Created
2026-09-05
Updated
2026-09-05
Started
2026-09-05T03:47:33Z
Verified
2026-09-05T03:47:33Z
Completed
2026-09-05T03:47:33Z

Synchronisation

Adapter
scripts/github-sync
Repository
unknown: the derived plan's header carries no repository
External issue
unknown: a live GitHub state; this executable makes no network call, so only `scripts/github-sync --check` can answer it
State there
unknown: a live GitHub state; this executable makes no network call, so only `scripts/github-sync --check` can answer it

Findings

Nothing is wrong with these records.