pub struct EconomicsSuite {Show 16 fields
pub schema: String,
pub id: String,
pub version: u32,
pub kind: String,
pub title: String,
pub provider: Option<String>,
pub harness: Option<String>,
pub model: Option<String>,
pub repetitions: Option<u32>,
pub control: Option<String>,
pub treatment: Option<String>,
pub max_budget_usd_per_session: Option<u32>,
pub tasks: Vec<String>,
pub seeds: Option<String>,
pub tokenizer: Option<String>,
pub freshness_inputs: Vec<String>,
}Expand description
A suite, as .ai/repo/benchmarks/economics/suites/<id>.yaml declares it: what is run,
with which model and arms, and which tracked paths decide whether its evidence is still
current. A live suite fills the session fields and a context suite the seed and
tokenizer fields; the other kind’s fields stay absent and are not written back.
use majordomus_cli::economics::model::EconomicsSuite;
let s: EconomicsSuite = serde_json::from_value(serde_json::json!({
"schema": "economics-suite/v1", "id": "context", "version": 1, "kind": "context",
"title": "Context selection, counted", "tokenizer": "o200k_base",
"freshness_inputs": ["apps/majordomus-cli/src/devcontext"]
}))
.unwrap();
assert!(s.model.is_none() && s.tasks.is_empty(), "a context suite runs no model");
let back = serde_json::to_value(&s).unwrap();
assert!(back.get("model").is_none(), "an absent live field is not written as null");Fields§
§schema: Stringeconomics-suite/v1.
id: StringThe suite’s id.
version: u32The suite’s own version.
kind: Stringlive (model sessions) or context (deterministic, no model).
title: StringA name.
provider: Option<String>The provider, for a live suite.
harness: Option<String>The harness that runs sessions, for a live suite.
model: Option<String>The model requested, for a live suite.
repetitions: Option<u32>Repetitions of every task and variant.
control: Option<String>The control variant’s id.
treatment: Option<String>The treatment variant’s id.
max_budget_usd_per_session: Option<u32>The spending cap the harness is given per session.
tasks: Vec<String>The task ids, for a live suite.
seeds: Option<String>Where the seeds come from, for a context suite.
tokenizer: Option<String>The tokenizer, for a context suite.
freshness_inputs: Vec<String>The tracked paths whose content decides whether recorded evidence is still current.
Trait Implementations§
Source§impl Clone for EconomicsSuite
impl Clone for EconomicsSuite
Source§fn clone(&self) -> EconomicsSuite
fn clone(&self) -> EconomicsSuite
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for EconomicsSuite
impl Debug for EconomicsSuite
Source§impl<'de> Deserialize<'de> for EconomicsSuite
impl<'de> Deserialize<'de> for EconomicsSuite
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Source§impl JsonSchema for EconomicsSuite
impl JsonSchema for EconomicsSuite
Source§fn schema_id() -> Cow<'static, str>
fn schema_id() -> Cow<'static, str>
Source§fn json_schema(generator: &mut SchemaGenerator) -> Schema
fn json_schema(generator: &mut SchemaGenerator) -> Schema
Source§fn inline_schema() -> bool
fn inline_schema() -> bool
$ref keyword. Read moreSource§impl PartialEq for EconomicsSuite
impl PartialEq for EconomicsSuite
Source§fn eq(&self, other: &EconomicsSuite) -> bool
fn eq(&self, other: &EconomicsSuite) -> bool
self and other values to be equal, and is used by ==.