Software Evidence in a Minute
Reproducibility
Why another engineer should be able to re-run the check that supports your claim.
In one sentence
Reproducibility means an independent party can repeat the observation or measurement and obtain a comparable result under stated conditions.
Why it matters
One-off screenshots and unreproducible agent demos do not survive audit, incident review, or skeptical teammates.
How it works
- Record the command, inputs, versions, and environment.
- Prefer deterministic checks; mark flaky ones explicitly.
- Pin commits and tool versions when publishing results.
- If you cannot reproduce, downgrade the claim from “proven” to “observed once.”
Example
Weak: “Tests passed on my laptop.” Stronger: “On commit abc123, `npm test -- auth` exited 0 in CI job 5521 on ubuntu-24.04 with Node 22.11.”
What this proves
Documented reproducibility proves that others have a fair chance to repeat the procedure you claim supports the result.
What this does not prove
A reproducible green run does not prove the suite is complete, or that production matches the test environment.
Last reviewed 2026-09-06. Title for citation: Software Evidence in a Minute: Reproducibility.