Back to Tournament
How to participate
To participate in the tournament, you will need to have a Metaculus account, and be registered for the tournament. With these in place, you’re ready to start forecasting!
To make forecasts in this tournament, visit the relevant forecasting question pages on Metaculus and submit your own probabilistic forecasts. You can watch this 30-second video that shows you how to submit a forecast using our interface.
To qualify for any of the prizes, you need to submit at least one forecast for each of the questions in the relevant round(s). Note that you may update these forecasts as frequently as you want, as long as you submit at least one for each question in the relevant round.
There are a total of three rounds, and four collections of prizes for performance in the relevant round(s). See Allocation of Prizes for more details.
Allocation of prizes
Prizes for best-performing forecasters
There are four types of prizes that are awarded to the top forecasters. Three of these are awarded to the top five forecasters for a given round, and one is awarded to the top five forecasters across all questions:
- The Maximum Likelihood Prize for each of top five forecasters in the Maximum Likelihood round, which will include approximately 30 questions on the status of AI benchmarks and other indicators 6 months from the opening of the round
- The Hill Climbing Prize for each of top five forecasters in the Hill Climbing round, which will include approximately 30 questions on the status of AI benchmarks and other indicators 12 months from the opening of the round
- The Deep Learning Prize for each of top five forecasters in the Deep Learning round, which will include approximately 30 questions on the status of AI benchmarks and other indicators 24 months from the opening of the round
- The Bayes Prize for each of top five forecasters across all three rounds
Winners of these prizes will need to demonstrate excellent foresight on a wide range of issues on AI operating on many time-horizons. Winners will receive payouts that depend on their rank for the relevant round(s); the higher your ranking, the larger your payout. A total of $43,500 is awarded across these four prizes. You can see a detailed breakdown of the prize allocation by rank in Table 1 below.
Participants will be scored using Metaculus scoring rules. The details of these scoring rules are rather complicated (you may view the details here here), but here are three key features:
- You should expect to receive the highest score if you report your true beliefs. For continuous questions, this means that you should put the center of your distribution so that your probability of the true number falling outside of your reported confidence intervals actually reflect how confident you are.
- The scoring rules award points both for being right and for being more right than the community. This means it’s worth doing equally a good job on questions that are difficult as it is for those which are relatively easy, as although your absolute accuracy might be worse for the former, you can still earn plenty of points for beating the crowd.
- The score you are awarded is time averaged over the time for which the question is open, and the points you earn depend upon your predictions' accuracy and when they are made. This means both that you should make your first prediction early, so as to earn points over more of the question's lifetime, and also that you should update your prediction anytime new information comes to light that alters your best estimate of the probability.
To get a feel for our scoring system, we recommend trying out our AI Calibration Tutorial. You may also make forecasts on some open questions on the Metaculus platform.
Dreyfus Prize
The Dreyfus prize is awarded to five authors of best analyses of the long-term future of Artificial Intelligence. A total of $6,500 will be awarded to the top five authors. Each participant is limited to a single entry.
Analyses might take the form of well-reasoned arguments and/or appropriate models for predictions on questions related to the long-term future of AI that are run in the tournament. Other analyses we would like to see are those that synthesize the predictions that are made in the course of the tournament, that draw out their key implications, and assemble these into a coherent view of the future of AI. Judges will look for analyses that are insightful, well-reasoned, and elucidating of some aspects of the future of AI. Authors should demonstrate a keen understanding of both forecasting and AI.
More details will follow when the Dreyfus Prize entry period opens.
Table 1. Prize allocation by rank
| Prize/Rank | Maximum Likelihood Prize | Hill Climbing Prize | Deep Learning Prize | Bayes Prize Prize | Dreyfus Prize |
|---|---|---|---|---|---|
| 1st | $2,200.00 | $3,000.00 | $2,500.00 | $7,500.00 | $2,250.00 |
| 2nd | $1,500.00 | $2,000.00 | $1,750.00 | $5,000.00 | $1,750.00 |
| 3rd | $1,300.00 | $1,750.00 | $1,300.00 | $3,500.00 | $1,250.00 |
| 4th | $1,100.00 | $1,250.00 | $1,050.00 | $2,500.00 | $750.00 |
| 5th | $900.00 | $1,000.00 | $900.00 | $1,500.00 | $500.00 |
Rules
- Participants can only submit predictions on questions in the tournament with one unique account. Participants are not allowed to share accounts.
- To be eligible for the Hill Climbing, and the Deep Learning Prizes, participants need to leave at least three comments on any of the questions in the relevant rounds before the round closing date (See Timeline).
- Participants will have a total of two months to submit predictions for a given round from when that round launches.
- To be eligible for prizes for performance on a certain round, participants must submit at least one forecast on all the questions in that round. Likewise, to qualify for the Bayes Prize, participants must forecast on all of the questions in the entire tournament.
- The tournament starts on 14th of December 2020, and ends approximately in June 2021, though different rounds will start and end at different dates and are subject to change.
- Participants are not permitted to participate on other platforms that are running the same questions, such as Hypermind.
- Metaculus will decide the final resolution of all questions at its discretion.
- Ambiguous questions resolutions will be avoided to the maximum extent possible, but in the rare instances where they occur, those questions will no longer be required as part of the tournament participation requirements.
- The Dreyfus prize is awarded solely on the merits of the submitted analyses, and contestants need not participate in any of the forecasting rounds.
- Prize recipients will be required to verify identity.
- Taxes and fees are paid by recipients.
- No purchase is necessary to participate or receive prizes.
Back to Tournament