Skip to content

M004 — Performance is executable evidence, and the hot path does no canonical work twice

Every public command can be benchmarked from the command registry with cold and warm distributions, the phases and work counters of any run can be printed, the caches and batch reads that remove repeated work preserve every observable result, an accepted baseline lives in the repository and a check refuses a regression by policy, and the pre-commit hook is fast enough that nobody routes around it.

ACTIVE 19 of 45 issues done

ready
14
blocked
11
active
0
verify
1
done
19
cancelled
0

Problem

The shell tool is slow enough to be avoided: doctor takes ten seconds and runs on every commit through the pre-commit hook, watch takes the same, knowledge nodes six seconds, plan validate four, the site-data check and a single test case half a minute, and the suite half an hour. Nobody could say which phase paid for it, because nothing measured phases; the two validators that cost two thirds of doctor are loops of one awk process per key over files that were already flattened once. The Rust executable measured at the same time is fast, so the slowness is the shell tool's own.

Outcome

Every public command can be benchmarked from the command registry with cold and warm distributions, the phases and work counters of any run can be printed, the caches and batch reads that remove repeated work preserve every observable result, an accepted baseline lives in the repository and a check refuses a regression by policy, and the pre-commit hook is fast enough that nobody routes around it.

Current state

MJ_TIMING prints per-validator phases and two work counters (feature/performance-truth 77f5d79). No benchmark command, no baseline, no regression check, no parse cache, no performance doctrine; the numbers above are the measured baseline on this repository.

Desired state

majordomus bench derives its targets from share/commands.yaml, records p50, p90, p95, p99, min, max, mean and stddev per command and mode, persists local runs under .ai/local/benchmarks/, compares against .ai/repo/benchmarks/baseline.json under the policy's benchmark block, and exits non-zero on a regression. Registry, catalogue, project-model and coverage validators read each flat file once. Flattened YAML is cached under .ai/local/cache/ by content hash. Project rules state the doctrine, docs/PERFORMANCE.md explains it, and CI runs the structural checks.

Scope

  • lib
  • bin/majordomus
  • share/commands.yaml
  • share/allow
  • share/standard/majordomus/
  • share/skeleton
  • .ai/repo/rules/project
  • .ai/repo/benchmarks
  • docs
  • scripts
  • test/cases
  • test/lib.sh
  • .github/workflows/validate.yml

Out of scope

  • Any change to apps/majordomus-cli, whose measured latency is not the problem
  • A metrics database, a dashboard, a tracing backend or any telemetry service
  • Hot reload or filesystem watching
  • A persistent capability result cache across processes beyond the flatten cache
  • Benchmarking external services

Acceptance criteria

  • Every public command in share/commands.yaml is a benchmark target without a second list anywhere
  • Cold and warm distributions are recorded separately with p50, p95 and p99 and never mixed
  • A local run is persisted under .ai/local/benchmarks/ and never treated as a baseline
  • The committed baseline is updated only by an explicit command on a clean tree
  • bench --check exits non-zero on a regression over the policy thresholds and names the command, the metric, both values and the threshold
  • Every canonical file is flattened at most once per command, proved by the work counters in a case
  • A cache hit and a cache miss produce byte-identical command output for every read-only command, proved by a case
  • doctor on this repository runs under the budget the policy declares, and the pre-commit hook inherits it
  • The performance rules are project rules under .ai/repo/rules/project, and docs/PERFORMANCE.md is the reference the README points at

Validation

  • bash test/run.sh
  • bin/majordomus doctor
  • bin/majordomus watch
  • bin/majordomus bench --check
  • scripts/generate-site-data --check
  • shellcheck -x -s bash bin/majordomus lib/*.sh

Evidence required

  • phase_timing
  • batch_reads
  • flatten_cache
  • bench_targets_derived
  • bench_persisted
  • baseline_explicit
  • regression_refused
  • doctor_under_budget
  • docs_integrated

Risks

  • A cache is a second thing that can be believed. The flatten cache is keyed by the content hash of the file it caches, so a stale entry cannot be read for a changed file; it can only cost a miss.
  • Wall-clock thresholds are noisy across machines. The regression policy is conservative and relative, the structural counters are exact, and only the counters gate CI strictly.
  • Batching reads changes how validators are written. Each rewrite is proved by the case that already exists for the validator, run before and after, plus the counter case.

Issues

Each one is an execution contract with its own acceptance criteria, validation command and required evidence. The wave column is the layer of the dependency graph it sits in.

issuestatuswavedepends ontitle
I0817 DONE 0 — Phase timing and work counters behind MJ_TIMING
I0818 DONE 1 I0817 The command registry is read in one pass
I0819 DONE 1 I0817 The project model is loaded with one flatten per file and no per-file lookups
I0820 READY 2 I0818, I0819 Catalogue, DAG, context and command-surface validators read their inputs once
I0821 VERIFY 1 I0817 Flattened YAML is cached under .ai/local/cache by content hash
I0822 BLOCKED 3 I0818, I0819, I0820 Every canonical file is flattened at most once per command
I0823 DONE 1 I0817 majordomus bench: targets derived from the command registry
I0824 DONE 2 I0823 Bench persists local runs and keeps a latest projection
I0825 DONE 3 I0824 An explicit committed baseline
I0826 DONE 4 I0825 bench --check refuses a regression by policy
I0827 DONE 2 I0823 The benchmark policy block
I0828 BLOCKED 3 I0818, I0819, I0820, I0827 doctor under a declared budget, and the hook inherits it
I0829 DONE 0 — Performance doctrine as project rules
I0830 DONE 5 I0823, I0826, I0829 docs/PERFORMANCE.md, README, CONTRIBUTING and CLI reference
I0831 BLOCKED 5 I0822, I0826 Performance claims at their proven status
I0832 DONE 2 I0817, I0819 generate-site-data instrumented and its hot spots removed
I0833 DONE 1 I0817 knowledge nodes and edges read each source once
I0834 BLOCKED 4 I0828 The test runner reports per-case time and the suite budget
I0835 BLOCKED 5 I0822, I0826 CI runs the structural performance checks
I0836 DONE 2 I0819 A synthetic repository generator for scaling cases
I0837 DONE 1 I0817 Timing report as JSON
I0838 READY 5 I0826 bench compare and the top regressions report
I0839 BLOCKED 4 I0828 watch inherits every doctor fix and its own budget
I0840 BLOCKED 3 I0820, I0821 context and its subcommands under a second
I0841 BLOCKED 2 I0821 Cache equivalence proved by property-shaped cases
I0842 BLOCKED 3 I0818, I0819, I0820, I0832, I0833 Before and after evidence for every optimisation
I0843 BLOCKED 6 I0830, I0831, I0835, I0841, I0842 The performance workflow is discoverable and complete
I2068 DONE 0 — A verdict that depends on the runner says so, and one cold sample is not a regression
I2660 READY 0 — The site is published as an artifact by the job that built it, so a ready site waits for no second workflow
I2661 READY 0 — A publication that did not arrive is a finding the next reader meets, never a green step
I2662 READY 0 — Validation leaves the deployment a runner: a branch's checks never hold the pool a publication or a release record needs
I2663 READY 3 I2489 The published UI is not read from nothing on every deployment
I2664 DONE 0 — A run that has already failed is cancelled by itself, and a red on the trunk is fixed before anything else is prepared
I2709 READY 1 I2664 A red on the trunk has one named carrier, and nobody else may land until it is paid
I2710 DONE 0 — A release run never waits behind pull-request validates for a job slot
I2711 DONE 0 — The pages job judges production only after the deployment of its own commit has arrived
I2716 DONE 0 — A case derives its budgets and lists from what it tests, never pins them
I2717 READY 0 — A release record pull request gets a verdict that can pass
I2846 READY 0 — A hand-pinned budget in a case is reported unless it states its margin
I3018 READY 0 — The intent workspace borrows the plan instead of copying it on every request
I3448 READY 0 — A deploy of master keeps every pre-release's metadata the site already serves
I3761 BLOCKED 1 I2660 The Pages run of a release head publishes with fresh evidence, or publishes the site with the evidence lag stated, and never fails on a missing snapshot artifact
I3799 READY 0 — The Pages publication budgets are re-derived from the runs that publish the language copies
I4200 READY 0 — A warm UI screen borrows what its sources answered and writes its page once, so a plan of any size is not copied per request
I4223 READY 0 — The pre-commit generation gate answers inside its bound, and a failed case's reason reaches an artifact

The graph

M004 dependency graphfrom .ai/repo/project
flowchart LR
    I0817["I0817<br/>Phase timing and work counters behind MJ_TIMING"]:::done
    I0818["I0818<br/>The command registry is read in one pass"]:::done
    I0819["I0819<br/>The project model is loaded with one flatten per file and no per-file lookups"]:::done
    I0820["I0820<br/>Catalogue, DAG, context and command-surface validators read their inputs once"]:::ready
    I0821["I0821<br/>Flattened YAML is cached under .ai/local/cache by content hash"]:::verify
    I0822["I0822<br/>Every canonical file is flattened at most once per command"]:::blocked
    I0823["I0823<br/>majordomus bench: targets derived from the command registry"]:::done
    I0824["I0824<br/>Bench persists local runs and keeps a latest projection"]:::done
    I0825["I0825<br/>An explicit committed baseline"]:::done
    I0826["I0826<br/>bench --check refuses a regression by policy"]:::done
    I0827["I0827<br/>The benchmark policy block"]:::done
    I0828["I0828<br/>doctor under a declared budget, and the hook inherits it"]:::blocked
    I0829["I0829<br/>Performance doctrine as project rules"]:::done
    I0830["I0830<br/>docs/PERFORMANCE.md, README, CONTRIBUTING and CLI reference"]:::done
    I0831["I0831<br/>Performance claims at their proven status"]:::blocked
    I0832["I0832<br/>generate-site-data instrumented and its hot spots removed"]:::done
    I0833["I0833<br/>knowledge nodes and edges read each source once"]:::done
    I0834["I0834<br/>The test runner reports per-case time and the suite budget"]:::blocked
    I0835["I0835<br/>CI runs the structural performance checks"]:::blocked
    I0836["I0836<br/>A synthetic repository generator for scaling cases"]:::done
    I0837["I0837<br/>Timing report as JSON"]:::done
    I0838["I0838<br/>bench compare and the top regressions report"]:::ready
    I0839["I0839<br/>watch inherits every doctor fix and its own budget"]:::blocked
    I0840["I0840<br/>context and its subcommands under a second"]:::blocked
    I0841["I0841<br/>Cache equivalence proved by property-shaped cases"]:::blocked
    I0842["I0842<br/>Before and after evidence for every optimisation"]:::blocked
    I0843["I0843<br/>The performance workflow is discoverable and complete"]:::blocked
    I2068["I2068<br/>A verdict that depends on the runner says so, and one cold sample is not a regression"]:::done
    I2660["I2660<br/>The site is published as an artifact by the job that built it, so a ready site waits for no second workflow"]:::ready
    I2661["I2661<br/>A publication that did not arrive is a finding the next reader meets, never a green step"]:::ready
    I2662["I2662<br/>Validation leaves the deployment a runner: a branch's checks never hold the pool a publication or a release record needs"]:::ready
    I2663["I2663<br/>The published UI is not read from nothing on every deployment"]:::ready
    I2664["I2664<br/>A run that has already failed is cancelled by itself, and a red on the trunk is fixed before anything else is prepared"]:::done
    I2709["I2709<br/>A red on the trunk has one named carrier, and nobody else may land until it is paid"]:::ready
    I2710["I2710<br/>A release run never waits behind pull-request validates for a job slot"]:::done
    I2711["I2711<br/>The pages job judges production only after the deployment of its own commit has arrived"]:::done
    I2716["I2716<br/>A case derives its budgets and lists from what it tests, never pins them"]:::done
    I2717["I2717<br/>A release record pull request gets a verdict that can pass"]:::ready
    I2846["I2846<br/>A hand-pinned budget in a case is reported unless it states its margin"]:::ready
    I3018["I3018<br/>The intent workspace borrows the plan instead of copying it on every request"]:::ready
    I3448["I3448<br/>A deploy of master keeps every pre-release's metadata the site already serves"]:::ready
    I3761["I3761<br/>The Pages run of a release head publishes with fresh evidence, or publishes the site with the evidence lag stated, and never fails on a missing snapshot artifact"]:::blocked
    I3799["I3799<br/>The Pages publication budgets are re-derived from the runs that publish the language copies"]:::ready
    I4200["I4200<br/>A warm UI screen borrows what its sources answered and writes its page once, so a plan of any size is not copied per request"]:::ready
    I4223["I4223<br/>The pre-commit generation gate answers inside its bound, and a failed case's reason reaches an artifact"]:::ready
    I0817 --> I0818
    I0817 --> I0819
    I0817 --> I0821
    I0817 --> I0823
    I0817 --> I0832
    I0817 --> I0833
    I0817 --> I0837
    I0818 --> I0820
    I0818 --> I0822
    I0818 --> I0828
    I0818 --> I0842
    I0819 --> I0820
    I0819 --> I0822
    I0819 --> I0828
    I0819 --> I0832
    I0819 --> I0836
    I0819 --> I0842
    I0820 --> I0822
    I0820 --> I0828
    I0820 --> I0840
    I0820 --> I0842
    I0821 --> I0840
    I0821 --> I0841
    I0822 --> I0831
    I0822 --> I0835
    I0823 --> I0824
    I0823 --> I0827
    I0823 --> I0830
    I0824 --> I0825
    I0825 --> I0826
    I0826 --> I0830
    I0826 --> I0831
    I0826 --> I0835
    I0826 --> I0838
    I0827 --> I0828
    I0828 --> I0834
    I0828 --> I0839
    I0829 --> I0830
    I0830 --> I0843
    I0831 --> I0843
    I0832 --> I0842
    I0833 --> I0842
    I0835 --> I0843
    I0841 --> I0843
    I0842 --> I0843
    I2660 --> I3761
    I2664 --> I2709
    classDef done stroke:#16a34a,fill:#dcfce7,color:#14532d,stroke-width:2px
    classDef active stroke:#2563eb,fill:#eff6ff,color:#1e3a8a,stroke-width:2px
    classDef verify stroke:#7c3aed,fill:#f5f3ff,color:#4c1d95,stroke-width:2px
    classDef ready stroke:#0891b2,fill:#ecfeff,color:#164e63,stroke-width:2px
    classDef planned stroke:#0891b2,fill:#ecfeff,color:#164e63,stroke-width:2px
    classDef blocked stroke:#b45309,fill:#fffbeb,color:#78350f,stroke-width:2px
    classDef cancelled stroke:#6b7280,fill:#f9fafb,color:#374151,stroke-width:2px
    classDef superseded stroke:#6b7280,fill:#f9fafb,color:#374151,stroke-width:2px
Linked from this diagram (45)

Canonical record: .ai/repo/project/milestones/M004.yaml. Read it back with majordomus plan show M004.

Searches the titles and descriptions of every page this site publishes.