Skip to content

I3370 — The corpus suite is registered and committed before its first run, and the T2 protocol names its registration digest

Complete .ai/repo/benchmarks/economics/suites/corpus.yaml as a live registered suite: the 20 tasks by stratum, the control baseline and the treatment arm the owner chooses, the model, harness and per-session budget, the pinned release, repetitions meeting the methodology's minimum, acceptance under I3133's pairing as primary outcome with wall time and spend secondary, the exact test and the MDE, the run-order seed and the stopping rule with the budget ceiling the owner consented to. Register the T2 paired protocol in .ai/repo/benchmarks/proof-plan.yaml with the registration digest.

BLOCKED wave 4 · p1 · implementation profile · parallel safe

Part of the-corpus-is-registered-before-it-runs — A corpus of 20 tasks, stratified by a difficulty proxy measured and registered before the new tasks are written, is registered with its analysis, run order, stopping rule and a pinned release.

Blocked. This issue cannot start until I3352, I3368, I3369, I3375, I3133 are done. The status is derived from that, not declared.

Objective

Complete .ai/repo/benchmarks/economics/suites/corpus.yaml as a live registered suite: the 20 tasks by stratum, the control baseline and the treatment arm the owner chooses, the model, harness and per-session budget, the pinned release, repetitions meeting the methodology's minimum, acceptance under I3133's pairing as primary outcome with wall time and spend secondary, the exact test and the MDE, the run-order seed and the stopping rule with the budget ceiling the owner consented to. Register the T2 paired protocol in .ai/repo/benchmarks/proof-plan.yaml with the registration digest.

Why

T2 of the owner's report of 2026-10-19 requires the split to be fixed in advance; a registration committed before any run of its suite, carried by every run's digest, is the evidence. It depends on I3133 because the acceptance pairing is methodology version 2's.

Current state

git ls-files .ai/repo/benchmarks/economics/suites lists context.yaml and pilot.yaml; economics summary reports suite pilot with 0 runs and no registered suite.

Desired state

economics summary lists suite corpus with every pair missing; economics proof shows T2 as not_run with reason no counted record.

Scope

  • .ai/repo/benchmarks/economics/suites
  • .ai/repo/benchmarks
  • docs/generated/economics.json
  • docs/generated/economics.md
  • docs/generated/economics.yaml
  • site/data/generated/economics.json

Out of scope

  • Running the suite (I3372)
  • Changing the methodology or a task

Dependencies

What waits on this

Acceptance criteria

  • The registration lists 7 small, 7 medium and 6 large tasks and loads
  • The T2 protocol validates with the registration digest
  • The budget ceiling cites a recorded owner decision
  • majordomus generate --check passes

Validation

  • majordomus economics summary
  • majordomus economics proof
  • majordomus economics references --work-dir \"$(mktemp -d)\"
  • majordomus generate --check
  • cargo test --manifest-path apps/majordomus-cli/Cargo.toml --test economics

Evidence required

  • registration_committed_before_first_run

Evidence

None recorded. Every token above needs a command or an artifact behind it before this issue can be completed; narrative is refused.

Timeline

started
—
verified
—
completed
—

Those three fields, the evidence above and the state of the dependencies are all the status is made of. There is no status field to disagree with them.

Canonical record: .ai/repo/project/issues/I3370.yaml. Read it back with majordomus plan show I3370.

Searches the titles and descriptions of every page this site publishes.