Module run 

What a run is made of, beyond the executions it recorded.

A recording reads the reports a run wrote: the suite’s TSV, cargo test’s output, a coverage summary. Every entry of a run belongs to exactly one of those reports, and EvidenceProducer is the word that names which. Not everything a report holds is something a claim can name: the crate’s own unit-test binary and its doctests ran, and no claim of docs/CLAIMS.yaml can point at one of their tests yet. Such an entry is listed as an EvidenceDropped, with the report it came from and the reason, rather than ignored, because a recording that silently threw part of a run away reads as a run that was smaller than it was.

What a run is made of

A producer word reads back as the producer it names and nothing else reads at all; a cargo test output with the crate’s unit tests and one integration binary yields one binary a claim can name and one dropped entry that says why the other is not one:

use majordomus_cli::evidence::{read_crate_output, EvidenceProducer, Outcome};

for p in [EvidenceProducer::Suite, EvidenceProducer::Crate, EvidenceProducer::Coverage] {
    assert_eq!(EvidenceProducer::parse(p.as_str()), Some(p));
}
assert_eq!(EvidenceProducer::parse("cargo"), None, "a word no report is named by");

let out = "     Running unittests src/lib.rs (target/debug/deps/majordomus_cli-1)\n\
           test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; \
           finished in 0.50s\n\
                Running tests/alpha.rs (target/debug/deps/alpha-1)\n\
           test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; \
           finished in 1.40s\n";
let read = read_crate_output(out);
assert_eq!(read.binaries.len(), 1);
assert_eq!(read.binaries[0].name, "alpha");
assert_eq!(read.binaries[0].outcome, Outcome::Pass);

assert_eq!(read.dropped.len(), 1, "the unit tests ran, and are listed rather than ignored");
let dropped = &read.dropped[0];
assert_eq!(dropped.producer, EvidenceProducer::Crate);
assert_eq!(dropped.what, "unittests src/lib.rs");
assert!(dropped.reason.contains("no claim can name"), "{}", dropped.reason);

A recording, as a typed run

Every recording returns an EvidenceRunRecord: its commit, the tree each report’s run measured, its totals, what was absent, dropped or unknown, and the executions, each of which maps back to the record’s id:

use majordomus_cli::evidence::{record, EvidenceProducer, EvidenceRunRecord, Origin, RecordRequest, TreeState};
use std::process::Command;

let root = tempfile::tempdir().unwrap();
let git = |args: &[&str]| {
    Command::new("git").arg("-C").arg(root.path()).args(args).output().unwrap()
};
git(&["init", "-q"]);
git(&["config", "user.email", "t@example.com"]);
git(&["config", "user.name", "t"]);
std::fs::create_dir_all(root.path().join("test/cases")).unwrap();
std::fs::write(root.path().join("test/cases/01_a.sh"), "echo a\n").unwrap();
git(&["add", "-A"]);
git(&["commit", "-qm", "init"]);

let reports = tempfile::tempdir().unwrap();
let tsv = reports.path().join("run.tsv");
std::fs::write(&tsv, "01_a\tok\t1\tparallel\n").unwrap();
let out = reports.path().join("run.json");
let req = RecordRequest {
    suite: Some(tsv),
    run_record: Some(out.clone()),
    ..RecordRequest::new(Origin::Local)
};
let run = record(root.path(), &req).unwrap().run_record;

// no measurement came with the report, so the run's tree is not vouched for
assert_eq!(run.working_tree, TreeState::Unknown);
assert_eq!(run.totals.executions, 1);
assert_eq!(run.absent[0].producer, EvidenceProducer::Crate);
let written: EvidenceRunRecord =
    serde_json::from_str(&std::fs::read_to_string(&out).unwrap()).unwrap();
assert_eq!(written, run);
assert_eq!(EvidenceRunRecord::id_of(&run.executions[0]), run.id);

Structs

EvidenceAbsent

A producer that gave the run nothing, and why.

EvidenceDropped

Something a report held that no claim can name yet, listed instead of ignored.

EvidenceRunRecord

One recording, as a typed run: what it recorded, against what, measured how, and what the reports held that it could not record.

EvidenceRunTotals

Counts over a run’s executions: how many, of each outcome, of each runner. A zero is absent rather than 0.

Enums

EvidenceProducer

Which report an entry of a run belongs to: the suite’s TSV, cargo test’s output, or a coverage summary.

Constants

RUN_RECORD_SCHEMA

The version of the EvidenceRunRecord document.

Functions

run_id

The id of a recording: <origin>:<provider>:<run id>:<attempt> for a CI run, else <origin>:<commit[..12]>:<at without '-' and ':'>.

weaken_by_records

A judgement, weakened by the run records that ran the same test: the second source of the one monotone rule, crate::evidence::freshness::weakened_by.

weakened_by_records

weaken_by_records against a checkout: the presented commit is the one presented names, or HEAD for the working tree (none when HEAD is unborn), and containment is crate::git::contains in root.