Skip to main content

Module model

Module model 

Source
Expand description

The typed records of the economics subsystem: the declarations it reads (methodology, suites, tasks), the raw facts it records (runs), and the answer it derives (summary).

Raw facts and derived answers are different types on purpose. A run holds what the provider reported and what the success gates found, and nothing computed from them; a pair, a metric and a summary are recomputed from runs on every read, so a change to how a number is derived never needs the runs rewritten.

Every public type is prefixed Economics: the OpenAPI document keeps one flat namespace of schema names, and Token* and Evidence* already belong to other subsystems.

use majordomus_cli::economics::model::EconomicsClass;
assert!(EconomicsClass::Observed.is_measurement());
assert!(!EconomicsClass::Counterfactual.is_measurement());
assert_eq!(EconomicsClass::Derived.weakest(EconomicsClass::Estimated), EconomicsClass::Estimated);

Structs§

EconomicsCheck
One success gate’s verdict on a run.
EconomicsClaimBinding
A published claim bound to the metric that must support it.
EconomicsClassDefinition
One measurement class as the methodology defines it.
EconomicsContextRun
One recorded measurement of the context suite: every seed the compiler compiled, counted, and every seed it refused, named with its reason rather than dropped.
EconomicsContextSeed
Token counts of one context-compilation seed.
EconomicsContextView
The context suite’s latest measurement, in brief.
EconomicsDistribution
The shape of a set of per-pair values.
EconomicsExclusion
One run excluded under the outlier rule.
EconomicsExplainInput
The input of economics.explain: the id of the one metric to explain. Nothing else is accepted, so a misspelt field is refused rather than silently ignored.
EconomicsExplanation
The answer of economics.explain: one metric, and everything it rests on.
EconomicsGateDefinition
One gate a run must pass to count as a success.
EconomicsHarness
The harness that ran a session, as it identified itself.
EconomicsHistoryPoint
A recorded measurement over time: one entry per recorded context run.
EconomicsHypothesis
A statement the benchmark exists to test. Never a result.
EconomicsInterval
A bootstrap interval, with everything needed to reproduce it.
EconomicsMethodology
.ai/repo/benchmarks/economics/methodology.yaml: what is measured, how, and when a result may be published.
EconomicsMethodologyChange
One entry of the methodology’s change log.
EconomicsMetric
One metric: its value, where it comes from, and how much it can bear.
EconomicsModelTotal
A model’s usage over a whole session, as the harness totalled it (modelUsage), including models the harness called on its own account.
EconomicsOrientation
What a session did before its first edit: the cost of getting oriented.
EconomicsOutcome
Whether the run did the task.
EconomicsOutlierPolicy
The rule for excluding runs, and the runs it excluded.
EconomicsPair
A control run and its treatment run, and what follows from them.
EconomicsPairCounts
Pair counts of a suite, so that nothing is dropped out of sight.
EconomicsPairing
How runs are paired and when a pair is comparable and valid.
EconomicsPublicationPolicy
When a quantitative total-token claim may be published. Every threshold must hold.
EconomicsQuery
The input of economics.summary: optional narrowing of the pairs and metrics. The verdict is always about all the evidence, whatever is narrowed, and a narrowed metric is never verified: the publication rule was evaluated over all of it, not over a slice.
EconomicsRequest
The token usage of one provider request, provider-native field names kept.
EconomicsRevision
The repository revision a run was recorded at.
EconomicsRun
One recorded run: a task, one variant, one repetition, every session of it. Raw facts only; nothing here is derived from another field.
EconomicsRunList
The answer of economics.runs: the raw facts every metric is computed from.
EconomicsRunView
One run in brief, with its derived totals.
EconomicsRunsQuery
The input of economics.runs: optional filters on suite, task and variant, all of which must match. An empty query lists every recorded run, and an unknown field is refused.
EconomicsSegment
A metric’s value within one segment of the pairs.
EconomicsSession
One session of a run: one harness invocation on one prompt, with every provider request it made, the harness’s own totals and the tools it called. A count the harness did not report is absent rather than zero.
EconomicsStatisticsPolicy
How per-pair results are aggregated into a headline value and its interval. Every parameter of the bootstrap is declared here, its generator seed included, so the same runs always give the same interval.
EconomicsSuite
A suite, as .ai/repo/benchmarks/economics/suites/<id>.yaml declares it: what is run, with which model and arms, and which tracked paths decide whether its evidence is still current. A live suite fills the session fields and a context suite the seed and tokenizer fields; the other kind’s fields stay absent and are not written back.
EconomicsSuiteView
A suite and the state of its evidence.
EconomicsSummary
The answer of economics.summary: every metric, where each comes from, and the one statement the evidence allows.
EconomicsTask
A task, as .ai/repo/benchmarks/economics/tasks/<id>/task.yaml declares it: the fixture a run starts from, the prompt of each session in order, and the hidden acceptance tests and verify command that decide whether the run succeeded.
EconomicsTaskSession
One session of a task: the prompt file it starts from.
EconomicsTokenizer
The identity of a tokenizer, recorded with every count it produced.
EconomicsUsage
A run’s totals, derived from its raw usage.
EconomicsVariant
One arm of the experiment: what the session runs with, and what it runs without.
EconomicsVerdict
The one sentence the evidence allows, and why it allows no stronger one.

Enums§

EconomicsClass
How a number came to be. The classes are ordered from strongest to weakest, and a value computed from several inputs is never stronger than the weakest of them.
EconomicsFreshness
How recorded evidence stands against the repository as it is now.
EconomicsMetricStatus
Where a metric stands against the publication rule: whether its value may back a claim, is a result too thin to claim anything, or was never measured at all.
EconomicsPairStatus
Why a pair does or does not count.