Need to boost forecast accuracy? Three factors have huge impact on the predictions: training, teams and tracking. It’s time to tee up!
Intelligence Advanced Research Projects Activity (IARPA) sponsored five different university teams that competed with each other to come up with the best possible ways to measure aggregate forecasts about events all over the world. They include military conflicts, elections, pandemics, refugee flows and even things like the price of commodities.
Wharton marketing professor Barbara Mellers recruited thousands of forecasters from blogs, professional societies, research centers and more. Her team had them make forecasts over a period of a year. The forecasters were given questions every two weeks, made their predictions, and updated their forecasts as often as they wanted.
The three t’s
Mellers team ran experiments to find out what improved forecasts. They found that three factors did extremely well. One was training people. They devised a one-hour probability-training module, and that seemed to improve predictions. Second was to put people in teams, as opposed to having them work individually and that improved predictions. The interaction, the information sharing, the debates about rationales boosted accuracy.
As third, tracking was a huge booster of forecasting accuracy. At the end of each year, the researchers took the top 2% of the thousands of forecasters, put them together in elite groups, and gave them the title of Super Forecasters.
“And these people increased their accuracy in more ways than we could possibly have imagined. They interacted more. They looked for more information. The net result was amazing. In fact, they helped us win the tournament three years in a row.”