Humans love to make predictions — whether about the movements of the stars, the gyrations of the stock market or the upcoming season's hot color — and the beginning of a new year brings out this tendency even more than usual.
Individuals are much harder to predict than they seem, not because people are infinitely complex, but because how we are apt to behave depends on subtle details of the situation. In recent decades, psychologists have conducted innumerable experiments showing that subtle changes in how a situation is framed, or even such seemingly irrelevant factors as background music or a writing font, can all have an impact on individual decision-making. Social phenomena are never just the product of individual people making decisions, but emerge out of many people making decisions in conjunction with each other.
Recently, my colleagues have started tracking the performance of one popular prediction market, at forecasting the outcome of weekly NFL games . So far, what they're finding is that the market predictions are better than the simple rule of always betting on the home team, but only slightly so — which, oddly, is very similar to what Tetlock found regarding his experts. Some outcomes, in other words, and possibly the outcomes we care about the most, simply aren't "predictable" in the way we would like.
Many people find this conclusion just a little depressing, because they associate a lack of predictability with an absence of meaning. But just because outcomes aren't predictable doesn't mean that there's nothing to be done. Consider venture capital firms. They expend a great deal of effort trying to identify the next "black swan" among the many pitches they receive. But they also recognize that no matter how hard they try, most of their swans will be white, and that a large percentage of their investments will fail to make money. These sound like lousy statistics, but many of those firms are doing quite nicely, thanks very much, and most of them are entirely comfortable with their apparent inability to predict winners. Unpredictability is built into their business model.
Naturally, you can't apply this model to everything: The United States cannot invest in six different foreign policies and wait to see which one pans out. Nevertheless, there are many circumstances that allow for far more experimentation than is currently the norm. And simply by acknowledging the limits of predictability, we can at least guard against the risk of overconfidence.