Insensitivity to Sample Size
We underestimate the impact of sample size when evaluating statistical results.
Insensitivity to sample size is a cognitive bias that occurs when people judge the probability of obtaining a sample statistic without considering the sample size. When asked whether a large or small hospital has more days where more than 60% of babies born are male, most people say it makes no difference. In reality, smaller samples are far more likely to show extreme proportions due to chance.
What it looks like in the wild.
- Everyday life Early Clinical Trial Results People dramatically overinterpret early clinical trial results with 20 patients as if they had the same reliability as studies with 2,000.
Recognising a definition is not the same skill as spotting it.
Two scenarios from the app, with distractors drawn from the same family of biases — which is what makes them hard.
A small startup features one successful founder on their website, leading people to assume the company's approach is proven despite having only a single data point.
For her next vacation, Chloe books a hotel that had five glowing reviews, believing it must be outstanding.
Biases rarely arrive alone.
Understand more. Assume less.
188 biases, 18,869 scenarios, and a record of how you actually decide.
Get Debias on Google Play