Skip to main content

explain

Function explain 

Source
pub fn explain(
    root: &Path,
    metric: &str,
) -> Result<EconomicsExplanation, String>
Expand description

Explain one metric: its value, formula and class, and every pair, run, suite and exclusion it rests on, with the commands that reproduce it. Err names the metrics that exist when metric is not one of them. A repository that declares no methodology has no metric at all, and is answered with present: false and the statement the summary makes, for any id: never an error, and never a number.

A metric over pairs lists the valid pairs it rests on and the invalid pairs of the same suites, so that what was left out is in view; a metric over runs (completion rate, tokens per completed task, transcript resume) lists every pair one of its runs belongs to, failed ones included; a metric over context seeds lists no pair.

use majordomus_cli::economics::{explain, EFFECTIVE_TOKEN_REDUCTION};
use majordomus_cli::economics::model::EconomicsMetricStatus;
let dir = tempfile::tempdir().unwrap();
// a methodology and one live suite `pilot`, with no run recorded
let e = explain(dir.path(), EFFECTIVE_TOKEN_REDUCTION).unwrap();
assert_eq!(e.metric.unwrap().status, EconomicsMetricStatus::NotMeasured);
assert!(e.reproduce.iter().any(|c| c.starts_with("majordomus economics run --suite pilot")));
let refused = explain(dir.path(), "tokens_saved").unwrap_err();
assert!(refused.contains(EFFECTIVE_TOKEN_REDUCTION), "the refusal lists what exists");

// a repository that benchmarks nothing: an answer, not a refusal, and no number in it
let bare = tempfile::tempdir().unwrap();
let none = explain(bare.path(), EFFECTIVE_TOKEN_REDUCTION).unwrap();
assert!(!none.present && none.metric.is_none() && none.methodology.is_none());
assert!(none.statement.unwrap().contains("no benchmark methodology is declared"));