Illusion of Validity
We are overconfident in predictions made from data, even when the data has low predictive value.
The illusion of validity is a cognitive bias in which a person overestimates their ability to interpret and predict accurately the outcome of analyzing a set of data, particularly when the data analyzed shows a very consistent pattern — that is, when the data 'tells' a coherent story. This was identified by Daniel Kahneman and Amos Tversky; consistent but low-validity patterns create high confidence despite low accuracy.
What it looks like in the wild.
- Finance Stock Picking Confidence A financial analyst with a consistent-looking portfolio record feels highly confident in their stock predictions, unaware that their hit rate is no better than chance.
Recognising a definition is not the same skill as spotting it.
Two scenarios from the app, with distractors drawn from the same family of biases — which is what makes them hard.
Mike read one article about a health supplement and now believes it will improve his memory, even though the article mentioned only a small, preliminary study.
A teacher grades an essay. The student used very fancy vocabulary and complex sentence structures. The teacher gives it a top mark, assuming the depth of thought must match the complexity of the language.
Biases rarely arrive alone.
Understand more. Assume less.
188 biases, 18,869 scenarios, and a record of how you actually decide.
Get Debias on Google Play