One of the motivations behind creating Metaculus is to see whether researchers and others can get skilled at predicting the impact that newly-announced research will have, as I discussed here. When a new paper appears as a preprint or published research, and news stories start to appear about it, it can be very difficult for the public (or science writers, or even other researchers) to develop a useful sense of its impact and importance. Academic institutions and researchers, as well as journalists, are all motivated to maximize the perceived importance and interest of each story. (How many game-changing cancer, Parkinson’s and Alzheimer’s research breakthroughs have you seen in the news over the last years?)

One role Metaculus can play is to solicit and aggregate predictions as to how big a deal new science and tech stories will actually turn out to be; wouldn’t it be useful to know? It’s not easy to accurately quantify impact, and there are various metrics that could be used. Probably simplest (and widely adopted even if flawed) is by citations. So we’ve launched a number of questions asking how many citations some recent interesting and newsy papers will acquire over coming months. Here are a few:

• How will recent claims of 234 SETI signals in SDSS data be treated by the community?
• A resurgence in interest in modified gravity vs. dark matter?
• Vacuum birefringence probably found near neutron stars: more research to come?
• Seeing single photons: more research to come?

In these questions, the goal is to predict whether or not the number of citations will exceed a certain number by a certain date. (In the very near future, we’ll launch the capability to directly predict the number of citations a paper will get by a given time.) Hopefully the community can create some useful tools, skills and methods to make these predictions accurate; it will be interesting to see!

Happy predicting!