Skip to content

Observability

What actually happened while nobody was watching, in a form that is not a transcript.

Observability

Workers run unattended. The question the next morning is not "what did they say" but "what changed, what was decided, what failed and what is now blocked". A transcript answers none of those without a person reading it, which is why a transcript is not observability.

9 moments

Every moment that names this one. Derived from the moments; this page keeps no list.

  1. asked what the workers did last night and had only transcripts to grepA conversation log records what was said, not what happened, so the operational question has to be answered by a person reading prose. cost: highcommon Why, and what to do →
  2. went looking for why we chose this and found a chat logA decision that was reached in a session is stored where only that session can read it, so it is neither reviewable nor discoverable. cost: highconstant Why, and what to do →
  3. was asked how far the milestone had got and had to go and ask four peopleProgress is an aggregate that exists only in people, because the parts it aggregates were never recorded in a form anything can add up. cost: mediumcommon Why, and what to do →
  4. could not say why one provider was used for that work and another for thisProvider and model selection is made implicitly, per session, and recorded nowhere, so it cannot be reviewed or repeated. cost: mediumcommon Why, and what to do →
  5. looked at the bill and could not say which of it produced anythingConsumption is measured per account and outcomes are recorded per person, so the two can never be joined. cost: mediumcommon Why, and what to do →
  6. lost a failure a worker had already reproduced, because the session endedA failure observed inside a session is described in that session and nowhere else, so the next one starts from the report rather than the evidence. cost: mediumcommon Why, and what to do →
  7. watched a worker spend a day on an approach that had already been ruled outNegative results are the majority of experimental output and the part nobody records, so the search space is re-explored. cost: mediumcommon Why, and what to do →
  8. could not reproduce a result because nothing recorded the state it came fromThe result is recorded and the conditions that produced it are not, so it is an anecdote rather than a measurement. cost: highoccasional Why, and what to do →
  9. was asked who made a change and under what policy, and had only a commitAttribution stops at the commit, so what authorised a change, what verified it and what governed it are unrecoverable. cost: highcommon Why, and what to do →