Skip to content

Prove the tool is fast, not assume it, and refuse a regression

Measure every public command and every capability of the executable against a committed baseline, and let the check fail on a regression rather than a reviewer noticing later.

described Performanceintermediatemaintainerreviewerwrites state

Situation

A refactor made doctor slower and nobody measured; a hot path in the executable rebuilds the index on every request and the only evidence is a feeling that MCP calls got sluggish. Performance claims exist in the README and nowhere else.

What you run

  • bench: samples every public command of the shell tool, cold and warm, after warm-up runs, and writes a result document under the local half; --check refuses a regression larger than the fractions the policy declares
  • doctor: reports its own wall time against the budget in the policy, as a warning that names the key and never as the exit code

Scenario

setup: installed-wired
given:
  - 'installed and wired, nothing measured yet'
steps:
  - id: the-targets
    run: ['bench', '--list']
    note: 'every public command of the registry is a target; nothing is listed by hand'
    expect:
      exit: 0
      stdout_contains: ['^doctor +read-only', '^start +state-mutating']
  - id: measure-one
    run: ['bench', 'version', '--samples', '1', '--warmup', '0', '--mode', 'cold', '--no-save']
    note: 'one cold run, not saved'
    expect:
      exit: 0
      stdout_contains: ['^version +cold +ok +1 ']
  - id: within-budget
    run: ['doctor']
    note: 'doctor reports its own wall time against the policy budget'
    expect:
      exit: 0
      stdout_contains: ['budget', 'doctor: 0 failure']
then:
  - 'a latency figure comes from a recorded run, never from prose'

Outcome

Every externally callable operation is a benchmark target derived from the registry, directly and over each transport it is exposed on, with a coverage table that CI checks is complete. Hundreds of MCP or HTTP requests rebuild nothing, and the counters prove it. The hot paths of the executable carry criterion benchmarks that build on every push, and the thresholds and budgets are policy, measured and changed with a run.

No scenario yet

This use case is described, not proved: it names its commands and rules, and nothing executes it. It cannot be more than described until a scenario is added.

Why the tool knows this

The chain from this page to the code: the use case names commands, rules and claims; each rule is dispatched from the registry and each claim names the test that proves it; the scenario above executed the commands. Nothing in this chain is prose about intent.

The rules that make it hold

Guarantees exercised

Related use cases

Computed from what they share: claims, rules, commands, category, applications. Nobody maintains this list.

The moments this answers