Skip to content

I2115 — The slowest case costs a fraction of its budget, and its budget is never raised to follow runner load again

Cut case 95_executable_reference to a fraction of its cost instead of raising the slowest-case budget again. Measure first where its time goes: it runs scripts/generate-site-data eight times over a full fixture copied by fixture_repo. Then reduce what each run regenerates or how many runs the case needs, keeping every assertion it makes, so the case proves the same things in a fraction of the time.

READY wave 0 · p1 · implementation profile · runs alone

Part of ci-critical-path — The critical path from a change to its verdict, and from a merge to its verified deploy, runs under a minute warm.

Ready. Every dependency is done, so majordomus plan start I2115 will be accepted.

Objective

Cut case 95_executable_reference to a fraction of its cost instead of raising the slowest-case budget again. Measure first where its time goes: it runs scripts/generate-site-data eight times over a full fixture copied by fixture_repo. Then reduce what each run regenerates or how many runs the case needs, keeping every assertion it makes, so the case proves the same things in a fraction of the time.

Why

The case has cost 2069, 2895, 3115, 3168, 3305, 3352 and, on 2026-10-09, 3468 s, moving with runner load; the slowest-case budget was re-derived from it twice, and when it went 18 s over 3450 s it failed the verdict of #831, a pull request that does not touch what it tests. A budget that sits within runner jitter of its case fails unrelated work.

Current state

test/cases/95_executable_reference.sh calls scripts/generate-site-data on eight lines over a fixture_repo copy of the tree; .ai/repo/ci/suite-durations.tsv records 2068 s for it; .ai/repo/ci/suite.yaml budgets the slowest case at 3450 s.

Desired state

The case proves every relation it proves today, costs well under half its budget on a loaded runner, and the slowest-case budget is lowered toward the new worst rather than raised. Amended on 2026-10-18 by the derive-performance plan: the change generalises from case 95 to every case: no case on the feedback lane exceeds its declared p95 (proposed 30 s unit, 120 s integration, 5 min acceptance), a longer one is decomposed or moves to the proof lane, and the budgets are derived from .ai/repo/ci/suite-durations.tsv (I2716), never pinned.

Scope

  • test/cases/95_executable_reference.sh
  • test/lib.sh
  • scripts/generate-site-data
  • .ai/repo/ci/suite.yaml
  • .ai/repo/ci/suite-durations.tsv
  • test/cases
  • .ai/repo/ci/gates.yaml

Dependencies

None. This issue is a root of the graph.

Acceptance criteria

  • Every assertion of case 95 holds after the change, and a mutation of each relation it proves still fails it
  • The case's cost on three GitHub runs, named, is under half of the slowest-case budget
  • The slowest-case budget is lowered or unchanged, never raised
  • No case on the feedback lane exceeds its declared p95 budget, and a case that would is decomposed or moved to the proof lane
  • Every case budget is derived from .ai/repo/ci/suite-durations.tsv and none is a pinned number

Validation

  • bash test/run.sh 95_executable_reference
  • scripts/ci/suite-budget

Evidence required

  • the_slowest_case_is_cut

Evidence

None recorded. Every token above needs a command or an artifact behind it before this issue can be completed; narrative is refused.

Timeline

started
—
verified
—
completed
—

Those three fields, the evidence above and the state of the dependencies are all the status is made of. There is no status field to disagree with them.

Canonical record: .ai/repo/project/issues/I2115.yaml. Read it back with majordomus plan show I2115.

Searches the titles and descriptions of every page this site publishes.