Intells Test Crowdsourcing Forecasting

The intelligence community is looking to the general public, rolling out a crowdsourcing initiative to let people weigh in on the likelihood of potential future events and their probability of occurring. It will weigh accurate predictors more.

The beta Aggregative Contingent Estimation system (ACES) website, called Forecasting ACE, is funded by the Intelligence Advanced Research Projects Activity and has launched recently on July 15. It will aggregate respondents’ answers and use the results to see if the group can make accurate predictions.

Weight to accuracy
Respondents will be polled about potential social, political, scientific, military and economic events. Unlike other crowdsourcing initiatives that equally weight all answers, ACES will more heavily weigh answers from the most accurate predictors. “Our system will not rely on simple averaging, but will combine forecasts from many people in a way that substantially outperforms simple averaging of judgments,” according to information on the site.

“We’re trying to make good use of everybody’s individual opinions and trying to determine what aspects of them might be important and would lead to a good forecast,” says Dr. Dirk Warnaar, the principal investigator for the ACES project at Applied Research Associates (ARA), developer of the tool.

Vote for predictors
ARA also plans to add a tool to give users feedback on their forecasting accuracy to help them improve their success rate and “start thinking about this problem a little deeper than they otherwise would,” he added. One possibility is allowing users to elaborate on how they arrived at their predictions and let other users vote on those responses.

Read more…