Reading an experiment readout: five checks before you trust it
A readout with a green checkmark and a big lift number is exactly the moment to slow down, not speed up. Here's what to check before believing it.
An experiment readout is the report summarizing what happened at the end of an A/B test. Like any report, it can be technically accurate and still misleading, if you don't check how it was actually put together.
🎯 Explain Like I'm Hired A readout is a claim, not a fact. Read it the way you'd read a claim someone's trying to convince you of, not something automatically true just because it has numbers in it. Example: a readout showing "+8% conversion, statistically significant, ship it" looks convincing, but if the split between the two test groups was supposed to be 50/50 and it actually came out 55/45, that alone can undermine the whole result, before you even look at the headline number.
Sign up to keep reading
Sign up free to unlock the worked examples, edge cases, and interview traps below.