computing definitive forecasts modeling calibrated forecasts computing probable forecasts predicting contingent futures composing contingent contingencies mapping the future exploring probable estimations assembling probable forecasts mapping accurate futures modeling probable wisdom exploring quantitative insights formulating definitive contingencies predicting calibrated contingencies generating predictive forecasts

News

Take the Research Impact Challenge

By Anthony on Dec. 27, 2016, noon GMT

One of the motivations behind creating Metaculus is to see whether researchers and others can get skilled at predicting the impact that newly-announced research will have, as I discussed here. When a new paper appears as a preprint or published research, and news stories start to appear about it, it can be very difficult for the public (or science writers, or even other researchers) to develop a useful sense of its impact and importance. Academic institutions and researchers, as well as journalists, are all motivated to maximize the perceived importance and interest of each story. (How many game-changing cancer, Parkinson’s and Alzheimer’s research breakthroughs have you seen in the news over the last years?)

One role Metaculus can play is to solicit and aggregate predictions as to how big a deal new science and tech stories will actually turn out to be; wouldn’t it be useful to know? It’s not easy to accurately quantify impact, and there are various metrics that could be used. Probably simplest (and widely adopted even if flawed) is by citations. So we’ve launched a number of questions asking how many citations some recent interesting and newsy papers will acquire over coming months. Here are a few:

In these questions, the goal is to predict whether or not the number of citations will exceed a certain number by a certain date. (In the very near future, we’ll launch the capability to directly predict the number of citations a paper will get by a given time.) Hopefully the community can create some useful tools, skills and methods to make these predictions accurate; it will be interesting to see!

Happy predicting!