Software Evidence in a Minute
Test Evidence
What automated tests can support as evidence—and the common overclaims they invite.
In one sentence
Test evidence is the recorded outcome of executing defined automated checks against a specific revision.
Why it matters
“Tests passed” is the most common engineering claim. Without knowing which tests and what they assert, the phrase is nearly empty.
How it works
- Name the suite, command, revision, and result.
- Distinguish unit, integration, and end-to-end scopes.
- Inspect whether assertions match the product claim.
- Treat skipped, flaky, and newly weakened tests as first-class signals.
Example
Evidence: “commit 9f3c1a2 — `pytest tests/billing -q` — 42 passed, 0 failed.” Non-evidence: “CI is green” with no job, suite, or commit.
What this proves
Test evidence proves that the named checks produced the recorded results on the named revision.
What this does not prove
Passing tests do not prove absence of defects outside their scope, correct production config, or that assertions are strong.
Last reviewed 2026-09-06. Title for citation: Software Evidence in a Minute: Test Evidence.