Hire experts for their critiques, not their predictions.

I’ve heard it said that the average expert is roughly as accurate as a dart-throwing chimpanzee. If that’s true, then why hire experts?

Why are those who have invested years developing comprehensive knowledge and skill no better than someone with limited or potentially none at predicting?

In seeking answers, I’ve found it helpful to go back to the reasons why we, as innovators, act in the ways that we do. I believe the common bond that unites us is our desire to create significant, positive change. Not just any change, because change by itself is not enough. We want the changes we make to be essential improvements. We want the advancements we create to be sufficiently significant to be noteworthy. And that’s the rub. In the vast majority of cases, we’re not sure about what changes, out of the countless number we can make, will produce the results we seek.

The question often takes the form, “What should we devote our time, effort, energy, and money to?” Answers to this question are a form of prediction. And prediction, at its core, is a guess. Meteorologists call their guesses a forecast. Gamers call it a gamble. Doctors call it a prognosis, attorneys a premise, and advisors a recommendation. Academics call it a conjecture and scientists a hypothesis. Theologians call it a prophecy. Engineers call it an assumption. Strategists call it a strategy. They’re all guesses. We form opinions and draw conclusions based on incomplete information as a starting point to making progress in our endeavors. In the absence of knowing, guessing is the best we can do.

Not all predictions are created equal. Some are like clockwork — repetitive, predictable, and accurate — such as when the sun will rise or set or when a train will arrive or depart. These predictions are reliable because we have a clear understanding of what causes what and can explain why.

Many other predictions are nothing like clockwork. Accurately predicting the outcome of political, economic, and cultural events is difficult. Being able to consistently choose between options reasonably and correctly, especially when the stakes are high and the results are not easy to foresee, is systemic in life an in innovation. It is in these situations when we do not yet have reasonable explanations of what causes what and why that we often turn to solutions for help.

In ancient Rome, people would turn to religious officials known as augurs to figure out whether their gods approved of their proposed actions. The augurs would observe naturals signs (such as the behavior of birds) to indicate their gods’ divine approval or disapproval. Today, our augurs might be priests, ministers, or religious leaders in the context of our spiritual affairs, consultants such as attorneys, physicians, or financial advisors in our life matters, and experts or specialists in our business concerns. We hire them as solutions to anticipate “What will happen?” or “What will happen if…?” we take a proposed action.

Some solutions we hire are much better at getting the prediction job done than others. Doctors subject their guesses to diagnostic tests, attorneys to debate, academics to peer review, scientists to experiments, and engineers to tests. These are all forms of constructive criticism, an attempt to detect and correct the errors in their guesses. Other solutions, in my experience, fare much worse. It is rare for prophets and pundits to subject their predictions to criticism or to test and track their accuracy. 

Much of the progress that we’ve enjoyed since the discovery and application of the sciences eluded our ancestors for millennia, despite them having the same faculties to reason and understand, because of their reliance on inferior prediction methods. Methods matter. Just as there are better routes to take on a road trip to reach a destination, there are better solutions to getting the prediction job done.

Experts can make better predictions in the areas in which they have acquired knowledge and know what works and what doesn’t. If you’re going to open a restaurant and you have never done that before, then I suggest you hire an expert to help. You can guess yourself and inevitably make mistakes — or — heed the advice of others who have already been down that path and can tell you which of their guesses failed. We can learn from their experience and increase our odds of avoiding the avoidable mistakes.

In the realm that we as innovators play — on the edge of the unknown — experts will be no better than us at guessing. I believe that the reason why is that they, like chimpanzees (and like us), are guessing. I suggest you stop hiring experts for their guesses unless they can explain why their way reliably produces better guesses across time and domains.

I’ve heard it said that predictions can be like old news — soon forgotten. Experts may be more knowledgeable and skilled. They may be more confident, have more charisma, and tell more compelling stories with conviction. They are likely to be no better than us at guessing in the absence of them having better information, better methods, and a commitment to tracking accuracy and continuously improving.

To be fair, it is difficult to make accurate forecasts far into the future or in situations where we don’t yet have reasonable explanations for what causes what and why. At the same time, a good judge doesn’t decide the guilt of a defendant after only hearing from the prosecution. A good doctor doesn’t say, “Take two pills and call me in the morning” without having evaluated their patient. 

Instead of hiring experts for their predictions, let us instead turn to them to help us critique and to improve our guesses. Let us turn to them for guidance and recommendations on which of our many guesses are the best to experiment and test. Let us turn to them to help us to detect and correct errors in our ideas, methods, and solutions, rather than for their guesses.

Acknowledgements and Sources

I was inspired by this phrase in Philip Tetlock’s and Dan Gardner’s masterwork, Superforecasting: The Art and Science of Prediction.

“…the average expert was roughly as accurate as a dart-throwing chimpanzee.” 

Philip Tetlock’s and Dan Gardner

Their project, The Good Judgment Open, is worth a visit (and fun, too.)

Leave a Reply

%d bloggers like this: