Why Expert Predictions Keep Failing And What Actually Works
Every election season, the experts come out. Psephologists, political analysts, economists, retired bureaucrats. They appear on television and tell you with great confidence what is going to happen. And then, with remarkable regularity, they are wrong.
This is not a new observation. Philip Tetlock, a psychologist at the University of Pennsylvania, spent two decades studying thousands of forecasts made by experts across politics, economics, and international relations. His conclusion was uncomfortable. The average expert was barely more accurate than random chance. Some were worse.
The problem is not that experts are unintelligent. Most of them are very smart. The problem is structural.
Why expertise does not equal accuracy
When you become an expert in a field, you develop a framework. A lens through which you interpret everything. This framework is useful. It helps you process information quickly and spot patterns that a novice would miss.
But it also creates blind spots. You start filtering new information through your existing framework rather than updating the framework when the information challenges it. You become confident in your lens to the point where contradicting evidence gets explained away rather than absorbed.
There is also the incentive problem. Experts have reputations to protect. A bold, confident prediction gets you invited back on television. A hedged, probabilistic answer does not make good television. So experts make confident calls. Confidence sounds like knowledge. It is often not.
What actually works
Tetlock’s research did not just identify the problem. It identified the solution. In his later work, he assembled a group of ordinary people with no particular domain expertise and trained them in one thing: probabilistic thinking. How to assign numbers to uncertainty. How to update beliefs when new information arrives. How to think about base rates before making a call.
This group, which he called superforecasters, outperformed professional intelligence analysts with access to classified information. Not by a small margin. By a significant one.
The skill that made the difference was not knowledge of the subject matter. It was the discipline of thinking in probabilities, tracking accuracy, and updating honestly.
The crowd dimension
The picture gets even more interesting when you aggregate forecasts across a large group of independent thinkers. Individual forecasters, however good, have personal biases and blind spots. When you combine many independent probability estimates on the same question, the biases cancel out and the genuine signal gets amplified.
This is why structured forecasting communities have consistently outperformed both individual experts and opinion polls on questions ranging from elections to economic indicators to geopolitical events.
The mechanism is not magic. It is statistics. Diverse, independent estimates aggregate to something more accurate than any single estimate, however expert.
What this means for you
You do not need to be an expert to think clearly about uncertain events. You need to develop the habit of assigning probabilities rather than directions, updating when evidence changes, and tracking your accuracy over time.
That last part is the one most people skip. Without feedback, you never learn whether your intuitions are calibrated or systematically biased. Most people go their whole lives without ever finding out.
If you want to start building this skill, the Strategem360 blog has this guide on what makes someone genuinely good at forecasting. It is worth the thirty minutes.
Experts will keep appearing on television. The gap between their confidence and their accuracy will remain. The people who understand why that gap exists are the ones who will make better decisions in a world full of confident but unreliable voices.
