evidence.test

{
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "test": {
      "type": "string",
      "description": "`suite:84_distribution_model`, `crate:why`, or the path itself\n(`test/cases/84_distribution_model.sh`), which is resolved to the same identity."
    }
  },
  "required": [
    "test"
  ],
  "description": "One test, by its stable identity or by the path a claim names it with.\n\nBoth spellings reach the same test. `docs/CLAIMS.yaml` names a path, the ledger keys on\nan identity, and a caller holding one of them should not have to know the other; the\nhandler resolves a path to the identity and reads the same derivation either way.\n\n```\nuse majordomus_cli::capability::builtin::evidence::EvidenceTestInput;\nuse majordomus_cli::evidence::{Runner, TestId};\n\nlet by_path: EvidenceTestInput =\n    serde_json::from_str(r#\"{\"test\": \"test/cases/84_distribution_model.sh\"}\"#).unwrap();\nlet id = TestId::of(&by_path.test).unwrap();\nassert_eq!(id.as_string(), \"suite:84_distribution_model\");\nassert_eq!(id.runner, Runner::Suite);\n\n// an identity is not a path: `TestId::of` reads paths, and the handler parses the\n// `<runner>:<name>` form itself\nlet by_id: EvidenceTestInput = serde_json::from_str(r#\"{\"test\": \"crate:why\"}\"#).unwrap();\nassert!(TestId::of(&by_id.test).is_none());\n```",
  "title": "EvidenceTestInput"
}

{
  "type": "object",
  "properties": {
    "test": {
      "type": "string",
      "description": "The stable identity: `suite:<case>` or `crate:<binary>`."
    },
    "runner": {
      "$ref": "#/$defs/EvidenceRunner",
      "description": "Which runner owns it."
    },
    "source": {
      "type": "string",
      "description": "Its own source, repository-relative."
    },
    "present": {
      "type": "boolean",
      "description": "Whether that source is in this checkout."
    },
    "reproduce": {
      "type": "string",
      "description": "The command that runs this one test."
    },
    "execution": {
      "anyOf": [
        {
          "$ref": "#/$defs/EvidenceExecution"
        },
        {
          "type": "null"
        }
      ],
      "description": "The latest recorded execution, when there is one."
    },
    "digest_matches": {
      "type": [
        "boolean",
        "null"
      ],
      "description": "Whether the test's source still hashes to what the execution recorded. `None` when\nthere is no execution, or the source is not there to hash."
    },
    "proves": {
      "type": "array",
      "items": {
        "$ref": "#/$defs/EvidenceClaimProof"
      },
      "description": "Every claim that names this test, with the state each is in."
    }
  },
  "required": [
    "test",
    "runner",
    "source",
    "present",
    "reproduce",
    "proves"
  ],
  "description": "One test, and what it proves.\n\nThe other direction of `ClaimEvidence`, read from the test's end: its identity, its\nsource and whether that source is still the one that ran, the command that runs it\nagain, and every claim that names it.\n\n```\nuse majordomus_cli::capability::builtin::evidence::TestEvidence;\nuse majordomus_cli::evidence::TestId;\n\nlet id = TestId::of(\"apps/majordomus-cli/tests/why.rs\").unwrap();\nlet answer = TestEvidence {\n    test: id.as_string(),\n    runner: id.runner,\n    source: id.source(),\n    present: true,\n    reproduce: id.reproduce(),\n    execution: None,\n    digest_matches: None,\n    proves: Vec::new(),\n};\nassert_eq!(answer.test, \"crate:why\");\nassert_eq!(answer.reproduce, \"cargo test --test why\");\n\n// nothing recorded: the two provenance keys are absent rather than null, so \"no\n// execution\" cannot be read as an execution that said nothing\nlet json = serde_json::to_value(&answer).unwrap();\nassert!(json.get(\"execution\").is_none() && json.get(\"digest_matches\").is_none());\nassert_eq!(json[\"proves\"], serde_json::json!([]));\n```",
  "title": "TestEvidence",
  "$defs": {
    "EvidenceRunner": {
      "oneOf": [
        {
          "type": "string",
          "const": "suite",
          "description": "A behavioural case under `test/cases/`, run by `test/run.sh`."
        },
        {
          "type": "string",
          "const": "crate",
          "description": "An integration test binary under `apps/majordomus-cli/tests/`, run by `cargo test`."
        }
      ],
      "description": "Which runner produced a result, and therefore how the test is named and re-run.\n\nThe repository has exactly two that a claim may name today. A third would be a variant\nhere and a branch in `TestId::of`; a claim naming something neither runner produces is\nreported as `ProofState::Unrunnable` rather than silently counted as covered.\n\n```\nuse majordomus_cli::evidence::{Runner, TestId};\n\n// which runner owns a test is read off where the test lives, never declared\nassert_eq!(TestId::of(\"test/cases/07_scope.sh\").unwrap().runner, Runner::Suite);\nassert_eq!(\n    TestId::of(\"apps/majordomus-cli/tests/why.rs\").unwrap().runner,\n    Runner::Crate\n);\n// and a path under neither directory belongs to no runner at all\nassert!(TestId::of(\"test/lib.sh\").is_none());\n```\n# Example\n\n```\nuse majordomus_cli::evidence::Runner;\nassert_eq!(Runner::Suite.prefix(), \"suite\");\nassert_eq!(Runner::Crate.prefix(), \"crate\");\n```"
    },
    "EvidenceExecution": {
      "type": "object",
      "properties": {
        "test": {
          "type": "string",
          "description": "The test, in `TestId::as_string` form."
        },
        "runner": {
          "$ref": "#/$defs/EvidenceRunner",
          "description": "Which runner produced it."
        },
        "source": {
          "type": "string",
          "description": "The test's own source, repository-relative."
        },
        "outcome": {
          "$ref": "#/$defs/EvidenceOutcome",
          "description": "What it said."
        },
        "seconds": {
          "type": "integer",
          "format": "uint64",
          "minimum": 0,
          "description": "How long it took, in whole seconds."
        },
        "commit": {
          "type": "string",
          "description": "The full commit id the run was made against."
        },
        "working_tree": {
          "type": "string",
          "description": "`clean`, `dirty` or `unknown`: whether the tree the run measured was the commit."
        },
        "digest": {
          "type": "string",
          "description": "`sha256:<hex>` of the test's own source as it was when the run was recorded. An\nexecution whose test no longer hashes to this did not run the test that is there now."
        },
        "at": {
          "type": "string",
          "description": "When it was recorded, RFC 3339, UTC."
        },
        "origin": {
          "$ref": "#/$defs/EvidenceOrigin",
          "description": "Where the run happened."
        },
        "command": {
          "type": "string",
          "description": "The exact command that runs this one test again."
        },
        "run": {
          "anyOf": [
            {
              "$ref": "#/$defs/EvidenceRun"
            },
            {
              "type": "null"
            }
          ],
          "description": "The continuous-integration run the execution was recorded in, when it was recorded in\none that could name itself. Absent for a local run, and absent from every row recorded\nbefore runs were named, which is why the ledger's version did not change."
        }
      },
      "required": [
        "test",
        "runner",
        "source",
        "outcome",
        "seconds",
        "commit",
        "working_tree",
        "digest",
        "at",
        "origin",
        "command"
      ],
      "description": "One recorded run of one test: the whole of what this repository durably remembers about\nit.\n\nEvery field is provenance. A result with no commit is an anonymous green, which is the\nthing a badge must never be derived from.\n\n```\nuse majordomus_cli::evidence::{digest_of, Execution, Origin, Outcome, Runner, TestId};\n\nlet id = TestId::of(\"apps/majordomus-cli/tests/why.rs\").unwrap();\nlet run = Execution {\n    test: id.as_string(),\n    runner: Runner::Crate,\n    source: id.source(),\n    outcome: Outcome::Pass,\n    seconds: 12,\n    commit: \"0\".repeat(40),\n    working_tree: \"clean\".into(),\n    digest: digest_of(b\"fn main() {}\"),\n    at: \"2026-09-11T00:00:00Z\".into(),\n    origin: Origin::Ci,\n    command: id.reproduce(),\n    run: None,\n};\n\n// it is a value in a tracked JSON file, so it has to survive the file unchanged\nlet back: Execution = serde_json::from_str(&serde_json::to_string(&run).unwrap()).unwrap();\nassert_eq!(back, run);\nassert_eq!(back.test, \"crate:why\");\nassert_eq!(back.command, \"cargo test --test why\");\n```\n# Example\n\nOne recorded run, with the provenance the runner did not carry: which commit, which\ntree state, the digest of the test's own source, when, and how to run it again.\n\n```\nuse majordomus_cli::evidence::{Execution, Origin, Outcome, Runner};\nlet e = Execution {\n    test: \"suite:07_scope\".into(),\n    runner: Runner::Suite,\n    source: \"test/cases/07_scope.sh\".into(),\n    outcome: Outcome::Pass,\n    seconds: 3,\n    commit: \"a04b65c9\".into(),\n    working_tree: \"clean\".into(),\n    digest: \"sha256:00\".into(),\n    at: \"2026-09-11T00:00:00Z\".into(),\n    origin: Origin::Local,\n    command: \"bash test/run.sh 07_scope\".into(),\n    run: None,\n};\nassert!(e.outcome.proves());\n```"
    },
    "EvidenceOutcome": {
      "oneOf": [
        {
          "type": "string",
          "const": "pass",
          "description": "It passed."
        },
        {
          "type": "string",
          "const": "fail",
          "description": "It failed."
        },
        {
          "type": "string",
          "const": "skip",
          "description": "It declined to run and said so (an unmet precondition the case reports itself)."
        },
        {
          "type": "string",
          "const": "timeout",
          "description": "The bound fired before it finished."
        },
        {
          "type": "string",
          "const": "error",
          "description": "The harness could not run it at all."
        }
      ],
      "description": "What a run said about one test.\n\n`NotRun` is deliberately absent: it is the absence of an execution, not an outcome one\nhad. Encoding it here would let a recorder write \"this did not run\" and have it counted\namong the things that did.\n\n```\nuse majordomus_cli::evidence::Outcome;\n\nassert!(Outcome::parse(\"ok\").proves());\nassert!(!Outcome::parse(\"timeout\").proves());\n// exactly one of the five is evidence of anything; the other four are reasons there\n// is none, kept apart because \"it declined\" and \"the harness broke\" are different bugs\nassert_eq!(Outcome::parse(\"skipped\"), Outcome::Skip);\nassert_eq!(Outcome::parse(\"nonsense\"), Outcome::Error);\n```\n# Example\n\n```\nuse majordomus_cli::evidence::Outcome;\nassert!(Outcome::parse(\"ok\").proves());\nassert_eq!(Outcome::parse(\"wat\"), Outcome::Error);\n```"
    },
    "EvidenceOrigin": {
      "oneOf": [
        {
          "type": "string",
          "const": "local",
          "description": "Someone ran it on their own machine."
        },
        {
          "type": "string",
          "const": "ci",
          "description": "A continuous-integration run."
        },
        {
          "type": "string",
          "const": "release",
          "description": "A release run."
        }
      ],
      "description": "Where a run happened. The canonical model is provider-neutral: a CI adapter records\n`Ci`, and nothing here knows or cares which CI it was.\n\n```\nuse majordomus_cli::evidence::Origin;\n\nassert_eq!(Origin::parse(\"CI\"), Some(Origin::Ci));\nassert_eq!(Origin::parse(\"local\"), Some(Origin::Local));\n// naming the provider is the adapter's business; the ledger records the kind of run,\n// so a repository that changes CI does not change what its old evidence says\nassert!(Origin::parse(\"github-actions\").is_none());\n```\n# Example\n\n```\nuse majordomus_cli::evidence::Origin;\nassert_eq!(Origin::parse(\"ci\"), Some(Origin::Ci));\nassert_eq!(Origin::parse(\"nowhere\"), None);\n```"
    },
    "EvidenceRun": {
      "type": "object",
      "properties": {
        "provider": {
          "type": "string",
          "description": "The provider that ran it, as the adapter that recorded it names it."
        },
        "id": {
          "type": "string",
          "description": "The provider's identifier of the run."
        },
        "attempt": {
          "type": "integer",
          "format": "uint32",
          "minimum": 0,
          "description": "Which attempt of the run, counted from 1."
        },
        "workflow": {
          "type": "string",
          "description": "The workflow the run belongs to."
        },
        "job": {
          "type": "string",
          "description": "The job within the run that recorded the execution."
        },
        "url": {
          "type": "string",
          "description": "Where a reader finds the run, as the provider addressed it at the time."
        },
        "head_sha": {
          "type": [
            "string",
            "null"
          ],
          "description": "The pull request's head commit when the run was a pull-request run (its `commit` is\nGitHub's merge commit, the tree that ran). Carried beside the commit, never\nsubstituted for it."
        },
        "event": {
          "type": [
            "string",
            "null"
          ],
          "description": "The event that started the run: `push`, `pull_request`, `schedule`,\n`workflow_dispatch`."
        }
      },
      "required": [
        "provider",
        "id",
        "attempt",
        "workflow",
        "job",
        "url"
      ],
      "description": "The continuous-integration run an execution was recorded in: enough to navigate back to\nthe run that produced it.\n\n`Origin` says what kind of run it was and deliberately names no provider. This names the\none run, in the words the provider addressed it with when it happened — its identifier,\nits attempt, the workflow and job, and the address a reader follows — because a link built\nlater from a template would point wherever the template's idea of the provider had drifted\nto. Only an adapter constructs it; nothing in the model reads the provider's name.\n\n```\nuse majordomus_cli::evidence::RunRef;\n\nlet actions = |k: &str| {\n    let v = match k {\n        \"GITHUB_ACTIONS\" => \"true\",\n        \"GITHUB_SERVER_URL\" => \"https://github.com\",\n        \"GITHUB_REPOSITORY\" => \"owner/repo\",\n        \"GITHUB_RUN_ID\" => \"42\",\n        \"GITHUB_RUN_ATTEMPT\" => \"2\",\n        \"GITHUB_WORKFLOW\" => \"validate\",\n        \"GITHUB_JOB\" => \"evidence\",\n        _ => return None,\n    };\n    Some(v.to_string())\n};\nlet run = RunRef::from_env(actions).unwrap();\nassert_eq!(run.id, \"42\");\nassert_eq!(run.attempt, 2);\nassert_eq!(run.url, \"https://github.com/owner/repo/actions/runs/42/attempts/2\");\n// a run that names neither its pull request's head nor its event still names itself\nassert_eq!((run.head_sha, run.event), (None, None));\n\nlet head = \"0123456789abcdef0123456789abcdef01234567\";\nlet pull = |k: &str| match k {\n    \"MJ_RUN_HEAD_SHA\" => Some(head.to_string()),\n    \"GITHUB_EVENT_NAME\" => Some(\"pull_request\".to_string()),\n    _ => actions(k),\n};\nlet run = RunRef::from_env(pull).unwrap();\nassert_eq!(run.head_sha.as_deref(), Some(head));\nassert_eq!(run.event.as_deref(), Some(\"pull_request\"));\n\n// outside a run there is nothing to name, and a half-described run is not a run\nassert!(RunRef::from_env(|_| None).is_none());\nassert!(RunRef::from_env(|k| (k == \"GITHUB_ACTIONS\").then(|| \"true\".to_string())).is_none());\n```"
    },
    "EvidenceClaimProof": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string",
          "description": "The claim's id, as `docs/CLAIMS.yaml` spells it."
        },
        "claim": {
          "type": "string",
          "description": "The claim sentence."
        },
        "status": {
          "type": "string",
          "description": "The status the claim declares: `guaranteed`, `advisory`, `planned`, `rejected`."
        },
        "source": {
          "type": [
            "string",
            "null"
          ],
          "description": "The document that defines the behaviour."
        },
        "implementation": {
          "type": [
            "string",
            "null"
          ],
          "description": "The file that implements it."
        },
        "test_path": {
          "type": [
            "string",
            "null"
          ],
          "description": "The path the claim names as its test, verbatim."
        },
        "test": {
          "type": [
            "string",
            "null"
          ],
          "description": "The test's stable identity, when a runner owns it."
        },
        "state": {
          "$ref": "#/$defs/ProofState",
          "description": "What the repository can say about the proof."
        },
        "meaning": {
          "type": "string",
          "description": "That state, in one sentence."
        },
        "execution": {
          "anyOf": [
            {
              "$ref": "#/$defs/EvidenceExecution"
            },
            {
              "type": "null"
            }
          ],
          "description": "The execution the state was derived from, when there is one."
        },
        "changed": {
          "type": "array",
          "items": {
            "type": "string"
          },
          "description": "The files this claim names that differ between the recorded commit and the working\ntree. Empty unless the state is `ProofState::Stale`.\n\nAlways emitted, empty included: the schema this capability publishes says the key is\nthere, and a client that has to tell \"no files changed\" from \"the server did not say\"\nis a client reading two different answers as one."
        },
        "detail": {
          "type": [
            "string",
            "null"
          ],
          "description": "Why the state is what it is, in one sentence, when the state alone does not say: a\ntest that declined to run, a run on a commit the presented revision does not\ncontain, a tree that was not its commit, a failing run the checkout holds uncommitted."
        },
        "reproduce": {
          "type": [
            "string",
            "null"
          ],
          "description": "The command that produces the proof again, when a runner owns the test."
        }
      },
      "required": [
        "id",
        "claim",
        "status",
        "state",
        "meaning",
        "changed"
      ],
      "description": "One claim, joined to whatever the repository actually recorded about it.\n\nThe claim's own fields are carried verbatim beside the derived ones, so that a reader\nwho disbelieves the state can check the join rather than take it.\n\n```\nuse majordomus_cli::evidence::{ClaimProof, ProofState};\n\nlet proof = ClaimProof {\n    id: \"scope-integrity\".into(),\n    claim: \"a commit outside the task's scope is refused\".into(),\n    status: \"guaranteed\".into(),\n    source: Some(\"docs/SCOPE.md\".into()),\n    implementation: Some(\"apps/majordomus-cli/src/scope.rs\".into()),\n    test_path: Some(\"test/cases/07_scope.sh\".into()),\n    test: Some(\"suite:07_scope\".into()),\n    state: ProofState::NotRun,\n    meaning: ProofState::NotRun.meaning().to_string(),\n    execution: None,\n    changed: vec![],\n    detail: None,\n    reproduce: Some(\"bash test/run.sh 07_scope\".into()),\n};\n\nlet json = serde_json::to_value(&proof).unwrap();\n// `changed` is emitted empty rather than omitted: a client must not have to tell\n// \"no file changed\" from \"the server did not say\"\nassert_eq!(json[\"changed\"], serde_json::json!([]));\n// an execution that does not exist is absent, not a null standing in for one\nassert!(json.get(\"execution\").is_none());\n// and so is a detail the state did not need\nassert!(json.get(\"detail\").is_none());\nassert_eq!(json[\"meaning\"], serde_json::json!(ProofState::NotRun.meaning()));\n```\n# Example\n\nA `ClaimProof` is produced by `report`; its `state` is the whole answer and its\n`meaning` is that answer in a sentence, worded once for every surface.\n\n```\nuse majordomus_cli::evidence::{report, ClaimProof, Ledger};\nuse majordomus_cli::synthetic::SyntheticRepository;\nlet repo = SyntheticRepository::small().unwrap();\nlet ledger = Ledger::load(repo.root()).unwrap();\nlet r = report(&repo.index().unwrap(), &ledger);\nlet proofs: &[ClaimProof] = &r.claims;\nassert!(proofs.is_empty(), \"a synthetic tree declares no claims\");\n```"
    },
    "ProofState": {
      "oneOf": [
        {
          "type": "string",
          "const": "proven",
          "description": "A passing run, and nothing has changed since it — the diff against the execution's\nown commit is empty but for the ledger. Proof of the tree in front of you."
        },
        {
          "type": "string",
          "const": "inputs_unchanged",
          "description": "A passing run, and nothing this claim names has changed since. Not proof at HEAD:\nsomething the claim does not name may have broken it."
        },
        {
          "type": "string",
          "const": "stale",
          "description": "A passing run, but something this claim names has changed since. The proof is older\nthan its subject."
        },
        {
          "type": "string",
          "const": "failing",
          "description": "The latest run of this claim's test did not pass."
        },
        {
          "type": "string",
          "const": "not_run",
          "description": "The claim names a test a runner owns, and no run of it has ever been recorded."
        },
        {
          "type": "string",
          "const": "unrunnable",
          "description": "The claim names a path no runner in this repository drives. Nothing can record it,\nso nothing can prove it this way."
        },
        {
          "type": "string",
          "const": "no_test",
          "description": "The claim names no test at all. Correct for a `planned` or `rejected` claim, and a\ndefect for any other."
        }
      ],
      "description": "What the repository can say about one claim's proof, right now.\n\nOrdered from strongest to weakest, so a summary that sorts by this reads as a ranking.\n\n```\nuse majordomus_cli::evidence::ProofState;\n\nlet mut states = vec![ProofState::NotRun, ProofState::Stale, ProofState::Proven];\nstates.sort();\nassert_eq!(\n    states,\n    vec![ProofState::Proven, ProofState::Stale, ProofState::NotRun]\n);\n\n// the two strongest are deliberately not one state: `inputs unchanged` is the absence\n// of a known invalidation, and collapsing it into `proven` is the unaccountable badge\nassert_ne!(ProofState::Proven, ProofState::InputsUnchanged);\nassert!(ProofState::Proven < ProofState::InputsUnchanged);\nassert!(ProofState::InputsUnchanged.passing());\n```\n# Example\n\n```\nuse majordomus_cli::evidence::ProofState;\nassert!(ProofState::Proven.passing());\nassert!(!ProofState::NotRun.passing());\nassert!(!ProofState::InputsUnchanged.meaning().is_empty());\n```"
    }
  }
}