predicting intelligent futures forecasting predictive understanding computing contingent insights forecasting definitive forecasts calculating intelligent understanding mapping the future generating predictive futures crowdsourcing precise forecasts computing definitive estimations crowdsourcing definitive forecasts modeling calibrated contingencies forecasting critical insights crowdsourcing critical understanding exploring calibrated predictions


Results of 2020 U.S. Election Risks Survey

By Juan on Oct. 20, 2020, 11 p.m. GMT

Metaculus and Rachel Aicher, Adjunct Lecturer in Political Science at Hunter College, City University of New York, are excited to announce the results of a forecasting session on risks to the 2020 U.S.

Read more…

A Preliminary Look at Metaculus and Expert Forecasts

By dan on June 2, 2020, 10:06 a.m. GMT

The Metaculus prediction platform began participating in the COVID-19 Expert Surveys on Survey 10.

Read more…

The Bentham prize

By Tamay on Jan. 15, 2020, 6:58 p.m. GMT
We will be awarding bi-weekly prizes of $300, $200 and $100 as first, second and third prize to the commenters who make the most valuable contributions to forecasting the questions in Metaculus' Animal Welfare series.

Read more…

Introducing the domain system

By Anthony on Aug. 4, 2019, 3:20 p.m. GMT

The domain system is a big under-the-hood improvement that has quietly gone live, but will start to be used.

Read more…

Metaculus is hiring!

By max.wainwright on June 26, 2019, 8 a.m. GMT

Metaculus improves the decision-making capacity of organizations and individuals by providing an open, reputation-based platform where people can improve their predictive abilities and organizations can calibrate their thinking according to calinrated and aggregated probabilities of future events.

Metaculus is in the process of adding new functionalities and growing the site in exciting ways. We want to bring onto the team a full-stack web developer who loves Metaculus and wants to see it thrive. We’re a small team, so you have a real opportunity here to influence the direction of the site and to make a big impact.

Read more…

Questions from the IARPA forecasting competition

By Anthony on March 29, 2018, 3:09 p.m. GMT

You'll notice a smattering of new questions, which have been imported from a recently-begun IARPA forecasting competition. Here are some salient facts.

Back in 2010, the US Intelligence Advanced Research Projects Agency (IARPA) launched a program on "Aggregative Contingent Estimation". This program produced a lot of very interesting research on crowd-based prediction via solicitation and aggregation, much of which informed Metaculus's conception (though there has been no funding or collaboration between Metaculus and IARPA).

IARPA is now running a "Hybrid forecasting competition" testing methods of combining human and machine prediction. As part of this program they are running a public-facing contest geopolitical forecasting challenge in which teams are invited to download questions and submit predictions on them by aggregating IARPA-provided human predictions, as well as doing whatever else.

While Metaculus has decided not to enter this competition full-fledged (since it would distract from out own focus), we have set up the machinery to download questions and make predictions, and will put some live on Metaculus that we think might be of interest to our community. We'll then feed back the Metaculus prediction to the competition. This, we hope, will be an interesting exercise, comparing the Metaculus aggregation method (and predictions) to others.

If you have concerns about providing predictions that will (in aggregate) be fed to IARPA servers, you can of course simply not make predictions on these questions (which are labeled and can be found in this category).

Thoughts welcome!

Read more…

Private Questions

By max.wainwright on March 12, 2018, 5:15 p.m. GMT

Metaculus now has a new type of question: private questions. Private questions are questions that only you can see on the site. They aren't moderated, so you can post one and predict on it right away. You can resolve your own private questions at any time, but points for private predictions won't be added to your overall Metaculus score and they won't effect your ranking on the leader board.

Read more…

Track record improvements

By max.wainwright on Feb. 20, 2018, 7 p.m. GMT

We've made some improvements to the Metaculus track record page: questions can now be filtered by date and question category, making it easier to drill down into the data. For example, it's easy to see that in the last 6 months the Metaculus prediction has outperformed the community prediction with a Brier score that's 33% lower. It's also easy to see that Metaculus performed especially well on biological questions (Brier score 75% lower than the community's), and both the Metaculus and community predictions missed the mark for cryptocurrency questions (both with scores > 0.24). In addition to the filters, there's now a histogram to clearly show the distributions of scores.

Read more…

Cryptocurrency competition analysis

By max.wainwright on Jan. 11, 2018, 3 a.m. GMT

The results are in, and a big congrations go to our winners jzima and Barbarossa and our runners up AndrewMcKnight and ElliotOlds. But how did everyone else do? In short, not so well.

Our winner received 748 points, but the median score amongst all cryptocurrency players was only 7 points and the average score was way down at −73 points. Only 25 players broke 100 points. Player overconfidence combined with our log scoring system really hurt here: just one very overconfident prediction could easily send someone's score tumbling down from the top of the ranks. That said, there was a top tier of eight players who got at least 590 points (with a large gap separating the 9th player), and, as we'll see below, they used different confidence strategies to get there.

Read more…