Module context 

The deterministic suite: what the context compiler considers and what it selects, counted.

No model is called. For every seed (an issue of the repository’s plan) the compiler (majordomus devcontext) is asked for the context of that work under its default budget. Every distinct file it reached is read from disk and counted in bytes and in tokens of a named tokenizer; a file two entries share is counted once, because a reader would read it once.

The denominator is deliberately the narrow one. A candidate is a file the compiler judged relevant: selected, or left out only because the budget ran out. Files it reached and judged irrelevant (below the relevance floor, too deep, stale, superseded) are counted as considered and reported by reason, but never divided by: a ratio against everything a graph walk touched would describe the walk, not the selection.

What this measures is selection: of what the compiler found relevant, how much it put in front of the worker. It is not what a session consumed, and it is never reported as total token savings — the metric that carries it says so in its own not field.

use majordomus_cli::economics::context::{count_tokens, TOKENIZER_ENCODING};
assert_eq!(TOKENIZER_ENCODING, "o200k_base");
assert_eq!(count_tokens(""), 0);
assert!(count_tokens("Majordomus counts what it selects.") > 3);

Constants

TOKENIZER_ENCODING

The encoding every count of this suite is in.

TOKENIZER_IMPLEMENTATION

The implementation and its version, as Cargo.lock resolved the pin in Cargo.toml: the build script reads it from the lockfile, so the number is stated in one place and a record can never name a version other than the one compiled in.

Functions

count_tokens

Tokens of text in TOKENIZER_ENCODING, special-token text counted as ordinary text.

measure

Measure a context suite over every seed, as one record. Seeds the compiler refuses are kept in the record’s refused list, each with the compiler’s reason, and returned beside it as well so that the caller can report them: a ratio over the seeds that compiled must never be read as one over every seed the suite names.

measure_seed

Measure one seed: compile its context, and count what was considered and selected.

seeds

The seeds of a context suite: the ids of the plan’s issues, from the file names under the suite’s seeds directory, sorted.

tokenizer

The tokenizer’s identity, as recorded with every count.