pub struct EconomicsPair {Show 16 fields
pub suite: String,
pub task: String,
pub category: String,
pub sessions: usize,
pub repetition: u32,
pub control: Option<String>,
pub treatment: Option<String>,
pub status: EconomicsPairStatus,
pub reasons: Vec<String>,
pub control_usage: Option<EconomicsUsage>,
pub treatment_usage: Option<EconomicsUsage>,
pub token_reduction: Option<f64>,
pub cost_reduction: Option<f64>,
pub tool_call_reduction: Option<f64>,
pub continuation_reduction: Option<f64>,
pub first_request_overhead: Option<i64>,
}Expand description
A control run and its treatment run, and what follows from them.
use majordomus_cli::economics::model::{EconomicsPair, EconomicsPairStatus};
let p: EconomicsPair = serde_json::from_value(serde_json::json!({
"suite": "pilot", "task": "vat-rounding", "category": "bugfix", "sessions": 1,
"repetition": 2, "control": "vat-rounding--baseline--r2", "status": "missing",
"reasons": ["no treatment run recorded"]
}))
.unwrap();
assert_eq!(p.status, EconomicsPairStatus::Missing);
assert!(p.treatment.is_none() && p.token_reduction.is_none(), "no side, no reduction");Fields§
§suite: StringThe suite.
task: StringThe task.
category: StringThe task’s category.
sessions: usizeSessions the task spans.
repetition: u32The repetition.
control: Option<String>The control run’s id, when recorded.
treatment: Option<String>The treatment run’s id, when recorded.
status: EconomicsPairStatusWhether it counts.
reasons: Vec<String>Why not, when it does not.
control_usage: Option<EconomicsUsage>The control’s totals.
treatment_usage: Option<EconomicsUsage>The treatment’s totals.
token_reduction: Option<f64>1 - treatment / control on total tokens. Negative: the treatment cost more.
Present on a valid pair, and on an excluded pair that would otherwise be valid, so
that the outlier rule’s effect can be shown both ways; only valid pairs enter a
metric other than effective_token_reduction.including_excluded. The same holds for
every reduction and overhead below.
cost_reduction: Option<f64>The same on the harness’s cost projection.
tool_call_reduction: Option<f64>The same on tool calls.
continuation_reduction: Option<f64>The same on the tokens of the last session only: what continuing cost.
first_request_overhead: Option<i64>First-request input of the treatment minus that of the control: what the treatment adds before the model has done anything.
Trait Implementations§
Source§impl Clone for EconomicsPair
impl Clone for EconomicsPair
Source§fn clone(&self) -> EconomicsPair
fn clone(&self) -> EconomicsPair
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for EconomicsPair
impl Debug for EconomicsPair
Source§impl<'de> Deserialize<'de> for EconomicsPair
impl<'de> Deserialize<'de> for EconomicsPair
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Source§impl JsonSchema for EconomicsPair
impl JsonSchema for EconomicsPair
Source§fn schema_id() -> Cow<'static, str>
fn schema_id() -> Cow<'static, str>
Source§fn json_schema(generator: &mut SchemaGenerator) -> Schema
fn json_schema(generator: &mut SchemaGenerator) -> Schema
Source§fn inline_schema() -> bool
fn inline_schema() -> bool
$ref keyword. Read moreSource§impl PartialEq for EconomicsPair
impl PartialEq for EconomicsPair
Source§fn eq(&self, other: &EconomicsPair) -> bool
fn eq(&self, other: &EconomicsPair) -> bool
self and other values to be equal, and is used by ==.