Novelty & primacy effects: why an early test result might not last
A change can look like a huge win in week one and fade by week three, or start weak and grow stronger. Both are predictable, named patterns, not random noise.
Two named patterns explain why a test's early results can mislead you about its true, long-term effect: one inflates early numbers, the other deflates them.
🎯 Explain Like I'm Hired A novelty effect is people clicking on something purely because it's new and unfamiliar, like everyone poking a shiny new button in week one, then ignoring it once it stops being novel. A primacy effect is the opposite: people used to the old way resist the new one at first, even if it's genuinely better, so the metric looks artificially worse early on and recovers as people adjust. Example: a redesigned homepage might show a 15% engagement lift in week 1 (curiosity clicks) that settles to a real 2% lift by week 4. Reporting the week-1 number as the final result would badly overstate the win.
Sign up to keep reading
Sign up free to unlock the worked examples, edge cases, and interview traps below.