Our community of predictors is poised to surpass 3,000, and we are continuing to experience steady growth, with close to 25,000 predictions now on the books.

Metaculus was not developed with political questions as its central goal, but there was no avoiding the 2016 election and its public interest. We ended up with a number of election-related questions that just resolved:

• Unsurprising: Third-party candidates did not get a significant share of the votes: 171 Metaculus predictors forecasted “yes” with only 6% probability. The election was indeed decided by Nov. 11, and among the 119 Metaculus predictors on this question, 90% was the consensus forecast. Marijuana was voted legal in California, with 85% probability per 224 Metaculus predictors.
• Surprising: First, the U.S. elected a white, Christian, non-Hispanic man as president once again. Like nearly everyone — media, polls, pundits, experts — Metaculus got it wrong. That the consensus never went above 25% is interesting on a number of levels. Second, although the E-Mini near-month S&P 500 futures dove by ~5% when the early returns began indicating high probability of a Trump victory, by the opening bell on Tuesday things had stabilized so there was no 2.5% movement in the S&P500, and under the parameters of the question there was not even a downward movement correlated with a Trump victory. Indeed, on Thursdsay, the Dow reached its all-time high. (Though the 2.5% had a low forecast, that may have been in large part due to the low probability attributed to a Trump victory, and an overall rise was a surprise to many).

The election exhibits the importance of accountable and well-calibrated predictions. There is good indication that forecasts were over-confident, with many users attributing 5% or smaller odds to a Trump victory. Although there were some painful point losses to Metaculus predictors on several of the election-related questions, we regard this as a feature, not a bug: working toward accurate predictions individually and as a community, these lessons are very valuable!

• In the future, it will be interesting to analyze the relative performance of the election forecasts IF a solid method can be worked out.
• The more diverse, and the more active our user base is, the better the odds that the Metaculus consensus will be correct.
• It’s interesting to speculate whether combining data-driven machine forecasts with human judgement by skilled forecasters could do better.

Keep Predicting!

The Metaculus Team