the-mesh-is-measured-against-one-agent — Coordinated workers on the mesh are compared with a single agent on the same batch tasks, at quality held by a registered non-inferiority guard
The runner runs a coordinated-workers arm; batch tasks compose existing corpus tasks; a registered suite compares it with a single agent; the verdict on wall time is allowed only under a non-inferiority guard on accepted share, and is published on the T4 row.
BLOCKED 0 of 5 issues done
- ready
- 0
- blocked
- 5
- active
- 0
- verify
- 0
- done
- 0
- cancelled
- 0
Problem
T4 of the owner's report of 2026-10-19 asks to show a speed-up of the mesh against a single agent at controlled quality. The economics runner runs one job as sequential sessions in one workspace (runner.rs:1-26); I3298 only declares topology arms as data (its acceptance: economics summary lists every arm with status untested); no task is a set of issues several workers could share; no analysis holds a speed-up to a quality guard.
Outcome
The runner runs a coordinated-workers arm; batch tasks compose existing corpus tasks; a registered suite compares it with a single agent; the verdict on wall time is allowed only under a non-inferiority guard on accepted share, and is published on the T4 row.
Current state
I3298 is READY at p3 in every-sessions-usage-is-observed with scope apps/majordomus-cli/src/economics. runner.rs:103 RunOptions has no concurrency. The mesh lab (test/mesh-lab/run:187-245) proves claims, handovers, partition and crash at the mesh layer with scripted nodes, not with agents doing work.
Desired state
suites/mesh.yaml registered with its T4 protocol; the arm runs N sessions in worktrees of one workspace sharing a server; the guarded verdict is published.
Scope
- apps/majordomus-cli/src/economics/runner.rs
- apps/majordomus-cli/src/economics/model.rs
- apps/majordomus-cli/src/economics/stats.rs
- apps/majordomus-cli/src/economics/mod.rs
- apps/majordomus-cli/tests/economics.rs
- .ai/repo/benchmarks/economics/suites
- .ai/repo/benchmarks/economics/tasks
- .ai/repo/benchmarks/economics/runs
- .ai/repo/benchmarks
- docs/ECONOMICS.md
- docs/generated/economics.json
- docs/generated/economics.md
- docs/generated/economics.yaml
- site/data/generated/economics.json
Out of scope
- Cross-machine mesh runs; the arm runs on one host, and the published result says so
- Lease-based assignment (I3198)
- Per-mechanism attribution (I3298)
Acceptance criteria
- A replaying harness runs the coordinated arm end to end without a provider
- No favourable verdict is printed while the accepted-share guard fails
- The T4 row publishes the verdict with every lost or invalid pair
Validation
- majordomus plan validate
Risks
- Concurrent live sessions multiply account usage; the budget ceiling is an owner decision
- On one host the workers share CPU and disk; the result is a one-host result
- Merge conflicts between workers are part of the treatment's cost and are counted, not excluded
Issues
Each one is an execution contract with its own acceptance criteria, validation command and required evidence. The wave column is the layer of the dependency graph it sits in.
| issue | status | wave | depends on | title |
|---|---|---|---|---|
| I3385 | BLOCKED | 6 | I3371 | A registered suite's primary verdict can be held to a non-inferiority guard, so a speed-up counts only when accepted share is not worse by more than a registered margin |
| I3386 | BLOCKED | 4 | I3375, I3298 | The economics runner runs one job as several concurrent workers in worktrees of one workspace sharing the treatment's server, and records the job's wall time to an integrated, accepted tree |
| I3387 | BLOCKED | 2 | I3367 | Batch tasks compose existing corpus tasks into sets of issues several workers can share, with their dependencies and the union of their hidden tests |
| I3388 | BLOCKED | 7 | I3385, I3386, I3387, I3352 | The mesh suite is registered before its first run, and the T4 protocol names its registration digest |
| I3389 | BLOCKED | 8 | I3388, I3354 | The mesh suite is run in both arms and its guarded verdict is published with its losses |
The graph
flowchart LR
I3385["I3385<br/>A registered suite's primary verdict can be held to a non-inferiority guard, so a speed-up counts only when accepted share is not worse by more than a registered margin"]:::blocked
I3386["I3386<br/>The economics runner runs one job as several concurrent workers in worktrees of one workspace sharing the treatment's server, and records the job's wall time to an integrated, accepted tree"]:::blocked
I3387["I3387<br/>Batch tasks compose existing corpus tasks into sets of issues several workers can share, with their dependencies and the union of their hidden tests"]:::blocked
I3388["I3388<br/>The mesh suite is registered before its first run, and the T4 protocol names its registration digest"]:::blocked
I3389["I3389<br/>The mesh suite is run in both arms and its guarded verdict is published with its losses"]:::blocked
I3385 --> I3388
I3386 --> I3388
I3387 --> I3388
I3388 --> I3389
classDef done stroke:#16a34a,fill:#dcfce7,color:#14532d,stroke-width:2px
classDef active stroke:#2563eb,fill:#eff6ff,color:#1e3a8a,stroke-width:2px
classDef verify stroke:#7c3aed,fill:#f5f3ff,color:#4c1d95,stroke-width:2px
classDef ready stroke:#0891b2,fill:#ecfeff,color:#164e63,stroke-width:2px
classDef planned stroke:#0891b2,fill:#ecfeff,color:#164e63,stroke-width:2px
classDef blocked stroke:#b45309,fill:#fffbeb,color:#78350f,stroke-width:2px
classDef cancelled stroke:#6b7280,fill:#f9fafb,color:#374151,stroke-width:2px
classDef superseded stroke:#6b7280,fill:#f9fafb,color:#374151,stroke-width:2pxLinked from this diagram (5)
- I3385 A registered suite's primary verdict can be held to a non-inferiority guard, so a speed-up counts only when accepted share is not worse by more than a registered margin
- I3386 The economics runner runs one job as several concurrent workers in worktrees of one workspace sharing the treatment's server, and records the job's wall time to an integrated, accepted tree
- I3387 Batch tasks compose existing corpus tasks into sets of issues several workers can share, with their dependencies and the union of their hidden tests
- I3388 The mesh suite is registered before its first run, and the T4 protocol names its registration digest
- I3389 The mesh suite is run in both arms and its guarded verdict is published with its losses
Canonical record: .ai/repo/project/milestones/the-mesh-is-measured-against-one-agent.yaml. Read it back with majordomus plan show the-mesh-is-measured-against-one-agent.