crowdsourcing probable understanding delivering contingent wisdom exploring accurate insights exploring definitive estimations computing predictive forecasts mapping the future forecasting probable understanding computing probable futures calculating calibrated understanding composing probable contingencies assembling quantitative insights assembling intelligent contingencies assembling calibrated estimations aggregating intelligent insights

News

Questions from the IARPA forecasting competition

By Anthony on March 29, 2018, 3:09 p.m. GMT

You'll notice a smattering of new questions, which have been imported from a recently-begun IARPA forecasting competition. Here are some salient facts.

Back in 2010, the US Intelligence Advanced Research Projects Agency (IARPA) launched a program on "Aggregative Contingent Estimation". This program produced a lot of very interesting research on crowd-based prediction via solicitation and aggregation, much of which informed Metaculus's conception (though there has been no funding or collaboration between Metaculus and IARPA).

IARPA is now running a "Hybrid forecasting competition" testing methods of combining human and machine prediction. As part of this program they are running a public-facing contest geopolitical forecasting challenge in which teams are invited to download questions and submit predictions on them by aggregating IARPA-provided human predictions, as well as doing whatever else.

While Metaculus has decided not to enter this competition full-fledged (since it would distract from out own focus), we have set up the machinery to download questions and make predictions, and will put some live on Metaculus that we think might be of interest to our community. We'll then feed back the Metaculus prediction to the competition. This, we hope, will be an interesting exercise, comparing the Metaculus aggregation method (and predictions) to others.

If you have concerns about providing predictions that will (in aggregate) be fed to IARPA servers, you can of course simply not make predictions on these questions (which are labeled and can be found in this category).

Thoughts welcome!