I0819 — The project model is loaded with one flatten per file and no per-file lookups
Make mj_project_load and mj_project_unknown_keys read each milestone and issue once, checking ids and unknown keys inside awk rather than with a subprocess per file and per key.
DONE wave 1 · p0 · implementation profile · runs alone
Part of M004 — Performance is executable evidence, and the hot path does no canonical work twice.
Objective
Make mj_project_load and mj_project_unknown_keys read each milestone and issue once, checking ids and unknown keys inside awk rather than with a subprocess per file and per key.
Why
The project validator costs 3.5 s of doctor and plan validate costs 4.4 s on 73 records; the count grows with the plan.
Current state
Every record is flattened, then read back with mj_yget per check, then re-read to build raw.tsv.
Desired state
One flatten and one awk per record, unknown keys computed by one awk over the flat directory against the allow-lists.
Scope
- lib/project.sh
- lib/doctor.sh
- lib/plan.sh
- test/cases/40_project_model.sh
- test/cases/41_project_status.sh
- test/cases/43_plan_command.sh
Out of scope
- Any change to the model semantics in project.awk
Dependencies
What waits on this
- I0820READY Catalogue, DAG, context and command-surface validators read their inputs once
- I0822BLOCKED Every canonical file is flattened at most once per command
- I0828BLOCKED doctor under a declared budget, and the hook inherits it
- I0832READY generate-site-data instrumented and its hot spots removed
- I0836READY A synthetic repository generator for scaling cases
- I0842BLOCKED Before and after evidence for every optimisation
Acceptance criteria
- validate:project under 400 ms and plan validate under 1 s on this repository
- Cases 40 to 48 pass unchanged
Validation
- MJ_TIMING=1 bin/majordomus plan validate
- bash test/run.sh 40_project_model
Evidence required
- batch_reads
Evidence
| covers | type | command | result | at commit |
|---|---|---|---|---|
| batch_reads | test | MJ_TIMING=1 bin/majordomus plan validate | mj_yaml_unknown_keys is one cut and one grep per file instead of one grep per key, and the record id check rides on the awk pass that emits the model rows; on this repository validate:project went from 3483 ms to 452 ms, doctor from 10.0 s to 3.9 s and plan validate from 4.4 s to 1.1 s with the same findings; an issue whose id disagrees with its filename is still refused by name; cases 02, 16, 24, 40, 41, 43, 44, 45 and 47 pass unchanged | next |
Risk
Unknown-key detection moved into awk must keep the same allow-list semantics; case 44 mutates a record and expects the refusal.
Timeline
- started
- 2026-09-05T02:06:00Z
- verified
- 2026-09-05T02:16:00Z
- completed
- 2026-09-05T02:16:00Z
Those three fields, the evidence above and the state of the dependencies are all the status is made of. There is no status field to disagree with them.
Canonical record: .ai/repo/project/issues/I0819.yaml. Read it back with majordomus plan show I0819.