the-corpus-is-registered-before-it-runs — A corpus of 20 tasks, stratified by a difficulty proxy measured and registered before the new tasks are written, is registered with its analysis, run order, stopping rule and a pinned release

✕BLOCKED 0 of 6 issues done

ready

0

blocked

6

active

0

verify

0

done

0

cancelled

0

Problem

The owner's report of 2026-10-19 (T2) asks for 20 comparable tasks split by difficulty in advance. The economics corpus has seven tasks, 4 small and 3 medium by their declared complexity and none large (grep complexity .ai/repo/benchmarks/economics/tasks/*/task.yaml). The suite declaration (model.rs:437) registers no strata, outcomes, analysis, run order or stopping rule. Difficulty is only the author's declaration. The treatment runs the checkout's own bin, lib and share (suites/pilot.yaml freshness_inputs), which the mesh changes many times a day, so a run spanning days would go stale before its pairs complete.

Outcome

suites/corpus.yaml is a registration committed before the first run: 20 tasks in three strata whose cut-offs on a measured proxy were registered before the new tasks existed, each task proven failable and passable and reviewed by someone other than its author, a seeded interleaved run order, a stopping rule, and a treatment pinned to one released version.

Current state

suites/pilot.yaml:10-14 fixes model claude-sonnet-5, repetitions 2 and 5 USD per session against methodology.yaml:57 min_repetitions 3. methodology.yaml:43 lists majordomus revision among the comparability keys. runner.rs:103 RunOptions has a per-session timeout and no suite ceiling. wc -l test/fixtures/economics/reference/*.patch gives patches of 11 to 92 lines.

Desired state

suites/corpus.yaml loads under the registration validation; economics references proves all 20 tasks; the T2 protocol in the proof plan names the suite's registration digest.

Scope

• apps/majordomus-cli/src/economics/model.rs
• apps/majordomus-cli/src/economics/mod.rs
• apps/majordomus-cli/src/economics/runner.rs
• apps/majordomus-cli/tests/economics.rs
• .ai/repo/benchmarks/economics/suites
• .ai/repo/benchmarks/economics/README.md
• .ai/repo/benchmarks
• docs/ECONOMICS.md
• site/content/docs/economics.md
• docs/generated/economics.json
• docs/generated/economics.md
• docs/generated/economics.yaml
• site/data/generated/economics.json

Out of scope

• The acceptance pairing and finish-contract arm of methodology version 2 (I3133)
• Per-mechanism arms (I3298)
• Running the corpus (a-registered-corpus-judges-majordomus)
• Tasks from the T1 repository; a later registration may add them, never this one mid-run

Acceptance criteria

• majordomus economics references proves each of the 20 registered tasks
• The registration is committed before the first run of the suite, and every run of it carries the registration digest and the pinned release
• A task whose measured proxy disagrees with its stratum is refused on load

Validation

• majordomus plan validate

Risks

• The corpus is one synthetic fixture written by the treatment's builders; the published result says so
• Case 430 runs economics references over every task, so 13 more tasks lengthen it
• I3133 must apply its completion declaration to the 13 new prompts if it lands after them
• I3367, I3374 and I3375 edit model.rs in sequence; their depends_on order them

Issues

Each one is an execution contract with its own acceptance criteria, validation command and required evidence. The wave column is the layer of the dependency graph it sits in.

The graph

flowchart LR
    I3367["I3367<br/>A suite can be registered: strata, outcomes, acceptance rule, analysis, MDE, a seeded interleaved run order and the stopping rule are declared data the load validates and the runner honours"]:::blocked
    I3368["I3368<br/>Six large tasks join the economics corpus, each failing on its start, passing on its reference and reviewed by someone other than its author"]:::blocked
    I3369["I3369<br/>Four medium and three small tasks join the economics corpus, each failing on its start, passing on its reference and reviewed by someone other than its author"]:::blocked
    I3370["I3370<br/>The corpus suite is registered and committed before its first run, and the T2 protocol names its registration digest"]:::blocked
    I3374["I3374<br/>Task difficulty is a proxy measured from each task's reference, and the stratum cut-offs are registered before any new task is written"]:::blocked
    I3375["I3375<br/>A registered suite pins its treatment to one released version, so landings on master during the runs leave them current"]:::blocked
    I3367 --> I3374
    I3368 --> I3370
    I3369 --> I3370
    I3374 --> I3368
    I3374 --> I3369
    I3374 --> I3375
    I3375 --> I3370
    classDef done stroke:#16a34a,fill:#dcfce7,color:#14532d,stroke-width:2px
    classDef active stroke:#2563eb,fill:#eff6ff,color:#1e3a8a,stroke-width:2px
    classDef verify stroke:#7c3aed,fill:#f5f3ff,color:#4c1d95,stroke-width:2px
    classDef ready stroke:#0891b2,fill:#ecfeff,color:#164e63,stroke-width:2px
    classDef planned stroke:#0891b2,fill:#ecfeff,color:#164e63,stroke-width:2px
    classDef blocked stroke:#b45309,fill:#fffbeb,color:#78350f,stroke-width:2px
    classDef cancelled stroke:#6b7280,fill:#f9fafb,color:#374151,stroke-width:2px
    classDef superseded stroke:#6b7280,fill:#f9fafb,color:#374151,stroke-width:2px

Canonical record: .ai/repo/project/milestones/the-corpus-is-registered-before-it-runs.yaml. Read it back with majordomus plan show the-corpus-is-registered-before-it-runs.