economics.explain
Jedna metrika a vše, na čem stojí: její vzorec, třída měření a co ta třída znamená, dvojice a běhy za ní (včetně neplatných), vyloučené běhy, sady a revize a příkazy, které ji reprodukují. Repozitář, který nedeklaruje žádnou metodiku, nemá žádnou metriku a na jakékoli id se odpoví `present: false` a uvedením důvodu, neodmítne se.
Jedna metrika a vše, na čem stojí: její vzorec, třída měření a co ta třída znamená, dvojice a běhy za ní (včetně neplatných), vyloučené běhy, sady a revize a příkazy, které ji reprodukují. Repozitář, který nedeklaruje žádnou metodiku, nemá žádnou metriku a na jakékoli id se odpoví `present: false` a uvedením důvodu, neodmítne se.
- query
- behaviorally_verified
- module economics
- #economics
- #evidence
Zpřístupnění
Kde je tato jedna definice dosažitelná. Absence je explicitní: povrch, který není uveden, ji nenese.
| povrch | jako |
|---|---|
| MCP | nástroj majordomus_economics_explain |
| HTTP | GET /api/v1/economics/explain operationId economics.explain |
| příkazová řádka | majordomus economics explain |
Schémata
Kanonický vstup a výstup z typů Rustu; schéma nástroje MCP a parametry a odpovědi OpenAPI se odvozují z nich.
input · EconomicsExplainInput
Vstup
economics.explain: id jedné metriky k vysvětlení. Nic jiného se nepřijímá, takže překlepnuté pole je odmítnuto, nikoli tiše ignorováno.
use majordomus_cli::economics::model::EconomicsExplainInput;
let i: EconomicsExplainInput =
serde_json::from_value(serde_json::json!({ "metric": "context_reduction_ratio" })).unwrap();
assert_eq!(i.metric, "context_reduction_ratio");
assert!(serde_json::from_value::<EconomicsExplainInput>(serde_json::json!({})).is_err());
let extra = serde_json::json!({ "metric": "context_reduction_ratio", "suite": "context" });
assert!(serde_json::from_value::<EconomicsExplainInput>(extra).is_err());| vlastnost | typ | povinné | popis |
|---|---|---|---|
| metrika | string | ano | Id metriky tak, jak je vypisuje economics.summary. |
JSON Schema
{
"additionalProperties": false,
"description": "The input of `economics.explain`: the id of the one metric to explain. Nothing else is\naccepted, so a misspelt field is refused rather than silently ignored.\n\n```\nuse majordomus_cli::economics::model::EconomicsExplainInput;\nlet i: EconomicsExplainInput =\n serde_json::from_value(serde_json::json!({ \"metric\": \"context_reduction_ratio\" })).unwrap();\nassert_eq!(i.metric, \"context_reduction_ratio\");\nassert!(serde_json::from_value::<EconomicsExplainInput>(serde_json::json!({})).is_err());\nlet extra = serde_json::json!({ \"metric\": \"context_reduction_ratio\", \"suite\": \"context\" });\nassert!(serde_json::from_value::<EconomicsExplainInput>(extra).is_err());\n```",
"properties": {
"metric": {
"description": "The metric's id, as `economics.summary` lists it.",
"type": "string"
}
},
"required": [
"metric"
],
"title": "EconomicsExplainInput",
"type": "object"
}výstup · EconomicsExplanation
Odpověď
economics.explain: jedna metrika a vše, na čem stojí.
Repozitář, který nedeklaruje metodiku měření výkonu, nic neměří, takže nemá metriku k vysvětlení: odpovídá se mu tak, jak odpovídá economics.summary — present: false, tvrzení proč, žádná metrika a žádné číslo — a neodmítá se, protože repozitář, který sám sebe neměří, není chybný požadavek. Id metriky, které deklarovaná metodika nemá, se stále odmítá.
use majordomus_cli::economics::model::{EconomicsClass, EconomicsExplanation};
let e: EconomicsExplanation = serde_json::from_value(serde_json::json!({
"present": true,
"metric": { "id": "context_reduction_ratio", "title": "Context reduction",
"class": "derived", "inputs": "counted", "unit": "ratio",
"formula": "median over seeds of 1 - selected_tokens / candidate_tokens",
"status": "not_measured", "n": 0, "suite": "context" },
"class_meaning": "Arithmetic over observed or counted values.", "methodology": 1,
"suites": [], "pairs": [], "excluded": [], "variants": [],
"reproduce": ["majordomus economics measure --suite context"]
}))
.unwrap();
assert_eq!(e.metric.unwrap().inputs, Some(EconomicsClass::Counted));
assert!(e.pairs.is_empty(), "a counted metric rests on seeds, not on pairs");
let none: EconomicsExplanation = serde_json::from_value(serde_json::json!({
"present": false, "statement": "No verified total-token-savings claim is available",
"class_meaning": "", "suites": [], "pairs": [], "excluded": [], "variants": [],
"reproduce": []
}))
.unwrap();
assert!(none.metric.is_none() && none.methodology.is_none(), "nothing declared, no number");| vlastnost | typ | povinné | popis |
|---|---|---|---|
| class_meaning | string | ano | Co jeho třída znamená, slovy metodiky. |
| vyloučeno | array | ano | Běhy vyloučené podle pravidla odlehlých hodnot. |
| metodika | integer | null | ne | Verze metodiky; chybí, pokud žádná není deklarována. |
| metrika | jedno ze 2 | ne | Metrika; chybí, když není deklarována žádná metodika. |
| dvojice | array | ano | Dvojice, na kterých stojí, platné či ne, pokud stojí na dvojicích. |
| přítomno | boolean | ano | Zda je zde vůbec deklarována nějaká metodika. |
| reprodukovat | array | ano | Jak to reprodukovat. |
| výrok | string | null | ne | Proč není co vysvětlovat, když je present false. |
| sady | array | ano | Sady, ze kterých čerpá. |
| varianty | array | ano | Porovnávané varianty. |
JSON Schema
{
"$defs": {
"EconomicsClass": {
"description": "How a number came to be. The classes are ordered from strongest to weakest, and a value\ncomputed from several inputs is never stronger than the weakest of them.\n\n```\nuse majordomus_cli::economics::model::EconomicsClass;\nassert_eq!(serde_json::to_value(EconomicsClass::Counterfactual).unwrap(), \"counterfactual\");\nlet parsed: EconomicsClass = serde_json::from_value(\"observed\".into()).unwrap();\nassert!(parsed < EconomicsClass::Estimated, \"a stronger class sorts first\");\n```",
"oneOf": [
{
"const": "observed",
"description": "Returned by the provider or its harness at run time and recorded unmodified.",
"type": "string"
},
{
"const": "counted",
"description": "Computed deterministically from bytes on disk with a named, pinned tokenizer.",
"type": "string"
},
{
"const": "derived",
"description": "Arithmetic over observed or counted values.",
"type": "string"
},
{
"const": "estimated",
"description": "A number standing in for one nobody measured. Never compared, never published.",
"type": "string"
},
{
"const": "counterfactual",
"description": "What a mechanism would have avoided, modelled rather than observed.",
"type": "string"
}
]
},
"EconomicsDistribution": {
"description": "The shape of a set of per-pair values.\n\n```\nuse majordomus_cli::economics::model::EconomicsDistribution;\nlet d: EconomicsDistribution = serde_json::from_value(serde_json::json!({\n \"n\": 1, \"mean\": 0.4, \"median\": 0.4, \"p25\": 0.4, \"p75\": 0.4, \"p95\": 0.4,\n \"min\": 0.4, \"max\": 0.4\n}))\n.unwrap();\nassert!(d.sd.is_none(), \"one value has no sample standard deviation\");\nassert!(d.min <= d.p25 && d.p25 <= d.median && d.median <= d.p75 && d.p75 <= d.max);\n```",
"properties": {
"max": {
"description": "Largest.",
"format": "double",
"type": "number"
},
"mean": {
"description": "Arithmetic mean.",
"format": "double",
"type": "number"
},
"median": {
"description": "Median.",
"format": "double",
"type": "number"
},
"min": {
"description": "Smallest.",
"format": "double",
"type": "number"
},
"n": {
"description": "How many values.",
"format": "uint",
"minimum": 0,
"type": "integer"
},
"p25": {
"description": "25th percentile.",
"format": "double",
"type": "number"
},
"p75": {
"description": "75th percentile.",
"format": "double",
"type": "number"
},
"p95": {
"description": "95th percentile.",
"format": "double",
"type": "number"
},
"sd": {
"description": "Sample standard deviation; absent for one value.",
"format": "double",
"type": [
"number",
"null"
]
}
},
"required": [
"n",
"mean",
"median",
"p25",
"p75",
"p95",
"min",
"max"
],
"type": "object"
},
"EconomicsExclusion": {
"description": "One run excluded under the outlier rule.\n\n```\nuse majordomus_cli::economics::model::{EconomicsExclusion, EconomicsOutlierPolicy};\nlet o: EconomicsOutlierPolicy = serde_json::from_value(serde_json::json!({\n \"rule\": \"A run is excluded only by naming it here with a reason.\",\n \"excluded\": [{ \"run\": \"vat-rounding--baseline--r2\", \"reason\": \"provider outage\" }]\n}))\n.unwrap();\nlet first: &EconomicsExclusion = &o.excluded[0];\nassert_eq!(first.run, \"vat-rounding--baseline--r2\");\nlet unexplained = serde_json::json!({ \"run\": \"vat-rounding--baseline--r1\" });\nassert!(serde_json::from_value::<EconomicsExclusion>(unexplained).is_err());\n```",
"properties": {
"reason": {
"description": "Why.",
"type": "string"
},
"run": {
"description": "The run's id.",
"type": "string"
}
},
"required": [
"run",
"reason"
],
"type": "object"
},
"EconomicsFreshness": {
"description": "How recorded evidence stands against the repository as it is now.\n\n```\nuse majordomus_cli::economics::model::EconomicsFreshness;\nassert_eq!(serde_json::to_value(EconomicsFreshness::NoEvidence).unwrap(), \"no_evidence\");\nlet f: EconomicsFreshness = serde_json::from_value(\"incompatible\".into()).unwrap();\nassert_eq!(f, EconomicsFreshness::Incompatible);\n```",
"oneOf": [
{
"const": "current",
"description": "Recorded under this methodology, and nothing it depends on changed since.",
"type": "string"
},
{
"const": "stale",
"description": "Recorded under this methodology, but a mechanism it measured changed since.",
"type": "string"
},
{
"const": "incompatible",
"description": "Recorded under another methodology version: not comparable at all.",
"type": "string"
},
{
"const": "no_evidence",
"description": "Nothing recorded.",
"type": "string"
}
]
},
"EconomicsInterval": {
"description": "A bootstrap interval, with everything needed to reproduce it.\n\n```\nuse majordomus_cli::economics::model::EconomicsInterval;\nlet i: EconomicsInterval = serde_json::from_value(serde_json::json!({\n \"level_bp\": 9500, \"low\": 0.21, \"high\": 0.38,\n \"method\": \"bootstrap percentile of the median\", \"resamples\": 10000, \"seed\": 20260924\n}))\n.unwrap();\nassert!(i.low <= i.high);\nassert_eq!(serde_json::to_value(&i).unwrap()[\"seed\"], 20260924, \"the seed travels with it\");\n```",
"properties": {
"high": {
"description": "Upper bound.",
"format": "double",
"type": "number"
},
"level_bp": {
"description": "Confidence in basis points.",
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"low": {
"description": "Lower bound.",
"format": "double",
"type": "number"
},
"method": {
"description": "How it was computed: a cluster bootstrap over tasks for every metric over pairs or\nruns, whose repetitions of one task are not independent, and an independent\nbootstrap over seeds for the context suite.",
"type": "string"
},
"resamples": {
"description": "Resamples.",
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"seed": {
"description": "Generator seed.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"level_bp",
"low",
"high",
"method",
"resamples",
"seed"
],
"type": "object"
},
"EconomicsMetric": {
"description": "One metric: its value, where it comes from, and how much it can bear.\n\n```\nuse majordomus_cli::economics::model::{EconomicsClass, EconomicsMetric, EconomicsMetricStatus};\nlet m: EconomicsMetric = serde_json::from_value(serde_json::json!({\n \"id\": \"effective_token_reduction\", \"title\": \"Effective token reduction\",\n \"class\": \"derived\", \"inputs\": \"observed\", \"unit\": \"ratio\",\n \"formula\": \"median of 1 - treatment / control\", \"status\": \"not_measured\", \"n\": 0,\n \"suite\": \"pilot\"\n}))\n.unwrap();\nassert_eq!(m.inputs, Some(EconomicsClass::Observed));\nassert!(m.value.is_none() && m.status == EconomicsMetricStatus::NotMeasured);\nassert!(serde_json::to_value(&m).unwrap().get(\"value\").is_none(), \"no value, not a zero\");\n```",
"properties": {
"class": {
"$ref": "#/$defs/EconomicsClass",
"description": "The measurement class."
},
"distribution": {
"anyOf": [
{
"$ref": "#/$defs/EconomicsDistribution"
},
{
"type": "null"
}
],
"description": "Their distribution."
},
"evidence": {
"default": [],
"description": "The runs, pairs or seeds it rests on.",
"items": {
"type": "string"
},
"type": "array"
},
"formula": {
"description": "The formula, in words.",
"type": "string"
},
"id": {
"description": "The metric's id.",
"type": "string"
},
"inputs": {
"anyOf": [
{
"$ref": "#/$defs/EconomicsClass"
},
{
"type": "null"
}
],
"description": "For a derived metric, the class of what it was derived from: a reduction derived from\nobserved usage is not the same evidence as one derived from counted bytes."
},
"interval": {
"anyOf": [
{
"$ref": "#/$defs/EconomicsInterval"
},
{
"type": "null"
}
],
"description": "The interval, when enough values exist."
},
"n": {
"description": "How many values it rests on.",
"format": "uint",
"minimum": 0,
"type": "integer"
},
"not": {
"description": "What the metric is not, where it is easily mistaken for something else.",
"type": [
"string",
"null"
]
},
"status": {
"$ref": "#/$defs/EconomicsMetricStatus",
"description": "Where it stands."
},
"suite": {
"description": "The suite.",
"type": "string"
},
"title": {
"description": "A name.",
"type": "string"
},
"unit": {
"description": "`ratio`, `tokens`, `calls`.",
"type": "string"
},
"value": {
"description": "The headline value; absent when nothing was measured.",
"format": "double",
"type": [
"number",
"null"
]
},
"warnings": {
"default": [],
"description": "What a reader must know before using it.",
"items": {
"type": "string"
},
"type": "array"
}
},
"required": [
"id",
"title",
"class",
"unit",
"formula",
"status",
"n",
"suite"
],
"type": "object"
},
"EconomicsMetricStatus": {
"description": "Where a metric stands against the publication rule: whether its value may back a claim,\nis a result too thin to claim anything, or was never measured at all.\n\n```\nuse majordomus_cli::economics::model::EconomicsMetricStatus;\nassert_eq!(serde_json::to_value(EconomicsMetricStatus::NotMeasured).unwrap(), \"not_measured\");\nlet s: EconomicsMetricStatus = serde_json::from_value(\"preliminary\".into()).unwrap();\nassert_ne!(s, EconomicsMetricStatus::Verified, \"a preliminary result is not a claim\");\n```",
"oneOf": [
{
"const": "verified",
"description": "Measured, and meets the publication rule. Only the primary metric over all the\nevidence can be: the rule is evaluated for it and for nothing else, so no secondary\nmetric and no narrowed slice is ever verified.",
"type": "string"
},
{
"const": "preliminary",
"description": "Measured, but no claim may rest on it: below the publication rule, narrowed by a\nquery, or a metric the rule was never evaluated for. A result, not a claim.",
"type": "string"
},
{
"const": "measured",
"description": "Measured deterministically; the publication rule for sampled data does not apply.",
"type": "string"
},
{
"const": "not_measured",
"description": "Nothing recorded to compute it from.",
"type": "string"
}
]
},
"EconomicsPair": {
"description": "A control run and its treatment run, and what follows from them.\n\n```\nuse majordomus_cli::economics::model::{EconomicsPair, EconomicsPairStatus};\nlet p: EconomicsPair = serde_json::from_value(serde_json::json!({\n \"suite\": \"pilot\", \"task\": \"vat-rounding\", \"category\": \"bugfix\", \"sessions\": 1,\n \"repetition\": 2, \"control\": \"vat-rounding--baseline--r2\", \"status\": \"missing\",\n \"reasons\": [\"no treatment run recorded\"]\n}))\n.unwrap();\nassert_eq!(p.status, EconomicsPairStatus::Missing);\nassert!(p.treatment.is_none() && p.token_reduction.is_none(), \"no side, no reduction\");\n```",
"properties": {
"category": {
"description": "The task's category.",
"type": "string"
},
"continuation_reduction": {
"description": "The same on the tokens of the last session only: what continuing cost.",
"format": "double",
"type": [
"number",
"null"
]
},
"control": {
"description": "The control run's id, when recorded.",
"type": [
"string",
"null"
]
},
"control_usage": {
"anyOf": [
{
"$ref": "#/$defs/EconomicsUsage"
},
{
"type": "null"
}
],
"description": "The control's totals."
},
"cost_reduction": {
"description": "The same on the harness's cost projection.",
"format": "double",
"type": [
"number",
"null"
]
},
"first_request_overhead": {
"description": "First-request input of the treatment minus that of the control: what the treatment\nadds before the model has done anything.",
"format": "int64",
"type": [
"integer",
"null"
]
},
"reasons": {
"default": [],
"description": "Why not, when it does not.",
"items": {
"type": "string"
},
"type": "array"
},
"repetition": {
"description": "The repetition.",
"format": "uint32",
"minimum": 0,
"type": "integer"
},
"sessions": {
"description": "Sessions the task spans.",
"format": "uint",
"minimum": 0,
"type": "integer"
},
"status": {
"$ref": "#/$defs/EconomicsPairStatus",
"description": "Whether it counts."
},
"suite": {
"description": "The suite.",
"type": "string"
},
"task": {
"description": "The task.",
"type": "string"
},
"token_reduction": {
"description": "`1 - treatment / control` on total tokens. Negative: the treatment cost more.\nPresent on a valid pair, and on an excluded pair that would otherwise be valid, so\nthat the outlier rule's effect can be shown both ways; only valid pairs enter a\nmetric other than `effective_token_reduction.including_excluded`. The same holds for\nevery reduction and overhead below.",
"format": "double",
"type": [
"number",
"null"
]
},
"tool_call_reduction": {
"description": "The same on tool calls.",
"format": "double",
"type": [
"number",
"null"
]
},
"treatment": {
"description": "The treatment run's id, when recorded.",
"type": [
"string",
"null"
]
},
"treatment_usage": {
"anyOf": [
{
"$ref": "#/$defs/EconomicsUsage"
},
{
"type": "null"
}
],
"description": "The treatment's totals."
}
},
"required": [
"suite",
"task",
"category",
"sessions",
"repetition",
"status"
],
"type": "object"
},
"EconomicsPairCounts": {
"description": "Pair counts of a suite, so that nothing is dropped out of sight.\n\n```\nuse majordomus_cli::economics::model::EconomicsPairCounts;\nlet c = EconomicsPairCounts {\n declared: 14, attempted: 12, valid: 9, control_failed: 1,\n treatment_failed: 1, both_failed: 0, other: 1,\n};\nlet accounted = c.valid + c.control_failed + c.treatment_failed + c.both_failed + c.other;\nassert_eq!(accounted, c.attempted, \"every attempted pair is in exactly one bucket\");\nassert_eq!(EconomicsPairCounts::default().declared, 0);\n```",
"properties": {
"attempted": {
"description": "Pairs with at least one run recorded.",
"format": "uint",
"minimum": 0,
"type": "integer"
},
"both_failed": {
"description": "Pairs where both failed.",
"format": "uint",
"minimum": 0,
"type": "integer"
},
"control_failed": {
"description": "Pairs whose control failed.",
"format": "uint",
"minimum": 0,
"type": "integer"
},
"declared": {
"description": "Pairs the suite declares.",
"format": "uint",
"minimum": 0,
"type": "integer"
},
"other": {
"description": "Pairs not comparable, missing a side, lacking usage, or excluded.",
"format": "uint",
"minimum": 0,
"type": "integer"
},
"treatment_failed": {
"description": "Pairs whose treatment failed.",
"format": "uint",
"minimum": 0,
"type": "integer"
},
"valid": {
"description": "Pairs that count.",
"format": "uint",
"minimum": 0,
"type": "integer"
}
},
"required": [
"declared",
"attempted",
"valid",
"control_failed",
"treatment_failed",
"both_failed",
"other"
],
"type": "object"
},
"EconomicsPairStatus": {
"description": "Why a pair does or does not count.\n\n```\nuse majordomus_cli::economics::model::EconomicsPairStatus;\nassert_eq!(serde_json::to_value(EconomicsPairStatus::ControlFailed).unwrap(), \"control_failed\");\nlet s: EconomicsPairStatus = serde_json::from_value(\"usage_unavailable\".into()).unwrap();\nassert_ne!(s, EconomicsPairStatus::Valid);\n```",
"oneOf": [
{
"const": "valid",
"description": "Both runs succeeded, are comparable and report usage.",
"type": "string"
},
{
"const": "control_failed",
"description": "The control run failed a success gate.",
"type": "string"
},
{
"const": "treatment_failed",
"description": "The treatment run failed a success gate.",
"type": "string"
},
{
"const": "both_failed",
"description": "Both failed.",
"type": "string"
},
{
"const": "usage_unavailable",
"description": "A run reports no usage to compare.",
"type": "string"
},
{
"const": "incomparable",
"description": "The runs differ in something the methodology requires them to share.",
"type": "string"
},
{
"const": "missing",
"description": "One side was never recorded.",
"type": "string"
},
{
"const": "excluded",
"description": "A run is excluded under the outlier rule.",
"type": "string"
}
]
},
"EconomicsSuiteView": {
"description": "A suite and the state of its evidence.\n\n```\nuse majordomus_cli::economics::model::{\n EconomicsFreshness, EconomicsPairCounts, EconomicsSuiteView,\n};\nlet v = EconomicsSuiteView {\n id: \"pilot\".into(), kind: \"live\".into(), version: 1, title: \"Pilot\".into(),\n model: Some(\"claude-sonnet-5\".into()), freshness: EconomicsFreshness::NoEvidence,\n freshness_detail: Some(\"no run recorded\".into()), runs: 0,\n pairs: EconomicsPairCounts { declared: 14, ..EconomicsPairCounts::default() },\n revisions: vec![], harnesses: vec![], models_reported: vec![],\n};\nlet json = serde_json::to_value(&v).unwrap();\nassert_eq!(json[\"freshness\"], \"no_evidence\");\nassert_eq!(json[\"pairs\"][\"declared\"], 14, \"declared pairs show even when none ran\");\n```",
"properties": {
"freshness": {
"$ref": "#/$defs/EconomicsFreshness",
"description": "Where its evidence stands."
},
"freshness_detail": {
"description": "Why, when not current.",
"type": [
"string",
"null"
]
},
"harnesses": {
"default": [],
"description": "The harness versions that ran it.",
"items": {
"type": "string"
},
"type": "array"
},
"id": {
"description": "The suite.",
"type": "string"
},
"kind": {
"description": "`live` or `context`.",
"type": "string"
},
"model": {
"description": "The model, for a live suite.",
"type": [
"string",
"null"
]
},
"models_reported": {
"default": [],
"description": "The models the harness reported.",
"items": {
"type": "string"
},
"type": "array"
},
"pairs": {
"$ref": "#/$defs/EconomicsPairCounts",
"description": "Pair counts."
},
"revisions": {
"default": [],
"description": "The revisions the evidence was recorded at.",
"items": {
"type": "string"
},
"type": "array"
},
"runs": {
"description": "Runs recorded.",
"format": "uint",
"minimum": 0,
"type": "integer"
},
"title": {
"description": "A name.",
"type": "string"
},
"version": {
"description": "Its version.",
"format": "uint32",
"minimum": 0,
"type": "integer"
}
},
"required": [
"id",
"kind",
"version",
"title",
"freshness",
"runs",
"pairs"
],
"type": "object"
},
"EconomicsUsage": {
"description": "A run's totals, derived from its raw usage.\n\n```\nuse majordomus_cli::economics::model::EconomicsUsage;\nlet u = EconomicsUsage {\n input_uncached: 10, cache_write: 90, cache_read: 900, output: 50,\n total_input: 1000, total: 1050, requests: 3, ..EconomicsUsage::default()\n};\nassert_eq!(u.total_input, u.input_uncached + u.cache_write + u.cache_read);\nlet json = serde_json::to_value(&u).unwrap();\nassert!(json.get(\"reasoning\").is_none() && json.get(\"cost_microusd\").is_none());\n```",
"properties": {
"cache_read": {
"description": "Input read from the cache.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"cache_write": {
"description": "Input written to the cache.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"cost_microusd": {
"description": "The harness's cost projection, in millionths of a dollar, when every session had one.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"input_uncached": {
"description": "Input neither read from nor written to the cache.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"output": {
"description": "Output, reasoning included.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"reasoning": {
"description": "Reasoning tokens, when every request reported them.",
"format": "uint64",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"requests": {
"description": "Provider requests.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"tool_calls": {
"description": "Tool calls.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"total": {
"description": "All input and output.",
"format": "uint64",
"minimum": 0,
"type": "integer"
},
"total_input": {
"description": "All input.",
"format": "uint64",
"minimum": 0,
"type": "integer"
}
},
"required": [
"input_uncached",
"cache_write",
"cache_read",
"output",
"total_input",
"total",
"requests",
"tool_calls"
],
"type": "object"
},
"EconomicsVariant": {
"description": "One arm of the experiment: what the session runs with, and what it runs without.\n\n```\nuse majordomus_cli::economics::model::EconomicsVariant;\nlet v: EconomicsVariant = serde_json::from_value(serde_json::json!({\n \"id\": \"baseline\", \"role\": \"control\", \"title\": \"Without Majordomus\",\n \"description\": \"The fixture repository as a competent team keeps it.\"\n}))\n.unwrap();\nassert_eq!(v.role, \"control\");\nassert!(v.excludes.is_empty(), \"an arm that names no exclusion excludes nothing\");\n```",
"properties": {
"description": {
"description": "Exactly what the arm consists of.",
"type": "string"
},
"excludes": {
"default": [],
"description": "What the arm deliberately does not include.",
"items": {
"type": "string"
},
"type": "array"
},
"id": {
"description": "`baseline`, `majordomus`, ...",
"type": "string"
},
"role": {
"description": "`control` or `treatment`.",
"type": "string"
},
"title": {
"description": "A short name.",
"type": "string"
}
},
"required": [
"id",
"role",
"title",
"description"
],
"type": "object"
}
},
"description": "The answer of `economics.explain`: one metric, and everything it rests on.\n\nA repository that declares no benchmark methodology measures nothing, so it has no\nmetric to explain: it is answered as `economics.summary` answers it — `present: false`,\nthe statement why, no metric and no number — rather than refused, because a repository\nthat does not benchmark itself is not a malformed request. A metric id that a declared\nmethodology does not have is still refused.\n\n```\nuse majordomus_cli::economics::model::{EconomicsClass, EconomicsExplanation};\nlet e: EconomicsExplanation = serde_json::from_value(serde_json::json!({\n \"present\": true,\n \"metric\": { \"id\": \"context_reduction_ratio\", \"title\": \"Context reduction\",\n \"class\": \"derived\", \"inputs\": \"counted\", \"unit\": \"ratio\",\n \"formula\": \"median over seeds of 1 - selected_tokens / candidate_tokens\",\n \"status\": \"not_measured\", \"n\": 0, \"suite\": \"context\" },\n \"class_meaning\": \"Arithmetic over observed or counted values.\", \"methodology\": 1,\n \"suites\": [], \"pairs\": [], \"excluded\": [], \"variants\": [],\n \"reproduce\": [\"majordomus economics measure --suite context\"]\n}))\n.unwrap();\nassert_eq!(e.metric.unwrap().inputs, Some(EconomicsClass::Counted));\nassert!(e.pairs.is_empty(), \"a counted metric rests on seeds, not on pairs\");\n\nlet none: EconomicsExplanation = serde_json::from_value(serde_json::json!({\n \"present\": false, \"statement\": \"No verified total-token-savings claim is available\",\n \"class_meaning\": \"\", \"suites\": [], \"pairs\": [], \"excluded\": [], \"variants\": [],\n \"reproduce\": []\n}))\n.unwrap();\nassert!(none.metric.is_none() && none.methodology.is_none(), \"nothing declared, no number\");\n```",
"properties": {
"class_meaning": {
"description": "What its class means, in the methodology's words.",
"type": "string"
},
"excluded": {
"description": "Runs excluded under the outlier rule.",
"items": {
"$ref": "#/$defs/EconomicsExclusion"
},
"type": "array"
},
"methodology": {
"description": "The methodology version; absent when none is declared.",
"format": "uint32",
"minimum": 0,
"type": [
"integer",
"null"
]
},
"metric": {
"anyOf": [
{
"$ref": "#/$defs/EconomicsMetric"
},
{
"type": "null"
}
],
"description": "The metric; absent when no methodology is declared."
},
"pairs": {
"description": "The pairs it rests on, valid or not, when it rests on pairs.",
"items": {
"$ref": "#/$defs/EconomicsPair"
},
"type": "array"
},
"present": {
"description": "Whether a methodology is declared here at all.",
"type": "boolean"
},
"reproduce": {
"description": "How to reproduce it.",
"items": {
"type": "string"
},
"type": "array"
},
"statement": {
"description": "Why there is nothing to explain, when `present` is false.",
"type": [
"string",
"null"
]
},
"suites": {
"description": "The suites it draws on.",
"items": {
"$ref": "#/$defs/EconomicsSuiteView"
},
"type": "array"
},
"variants": {
"description": "The arms compared.",
"items": {
"$ref": "#/$defs/EconomicsVariant"
},
"type": "array"
}
},
"required": [
"present",
"class_meaning",
"suites",
"pairs",
"excluded",
"variants",
"reproduce"
],
"title": "EconomicsExplanation",
"type": "object"
}Politiky
- benchmark
- povinné — cíl na každém transportu, který expozice deklaruje; případy jsou ty ze vstupního typu
- cache
- process — až 32 položek v procesu, 5 s každá, vymezené otiskem registru
Cíle benchmarku
Odvozeno z registru pro tento repozitář: jeden požadavek na transport a případy, které poskytuje typ vstupu. Celá matice je na stránce měření výkonu.