pub fn bootstrap_median_clustered(
clusters: &[Vec<f64>],
level_bp: u32,
resamples: u32,
seed: u64,
min_n: usize,
) -> Option<EconomicsInterval>Expand description
A percentile bootstrap interval of the median over clustered values: each resample draws as many clusters as there are, with replacement, pools every value of each drawn cluster, and takes the median of the pool.
The repetitions of one task share its fixture, its prompt and its difficulty, so they are not independent observations: resampling them one by one would treat four runs of one easy task as four tasks and report an interval narrower than the evidence allows. Resampling whole clusters (tasks) keeps what varies between tasks in the interval.
Deterministic: non-finite values are dropped, each cluster is sorted, empty clusters
are dropped, and the clusters are sorted by their sorted values before any draw, so the
interval depends on the clusters and the seed, never on the order they were listed in.
None below min_n values in total, with fewer than two clusters (a single task has
no between-task variation to resample), or with no resamples.
use majordomus_cli::economics::stats::{bootstrap_median_clustered, CLUSTER_METHOD};
let tasks = vec![vec![0.1, 0.12], vec![0.3, 0.31, 0.29], vec![0.5, 0.52]];
let a = bootstrap_median_clustered(&tasks, 9500, 2000, 7, 5).unwrap();
let shuffled = vec![vec![0.52, 0.5], vec![0.1, 0.12], vec![0.29, 0.31, 0.3]];
assert_eq!(a, bootstrap_median_clustered(&shuffled, 9500, 2000, 7, 5).unwrap());
assert!(a.low <= 0.3 && 0.3 <= a.high);
assert_eq!(a.method, CLUSTER_METHOD);
let one_task = vec![vec![0.1, 0.2, 0.3, 0.4, 0.5, 0.6]];
assert!(bootstrap_median_clustered(&one_task, 9500, 2000, 7, 5).is_none());