Back to the AI Progress Essay Contest
AI Progress Essay Contest
- Why is Metaculus hosting this contest?
- Why is Metaculus asking for essays?
- What is forecasting?
- When is forecasting valuable?
- Why should I be interested in forecasting?
- What is Metaculus?
- What is a recent example of how Metaculus forecasting is helping people?
- What is a Fortified Essay?
- What is Metaculus offering writers?
Why is Metaculus hosting this contest?
We seek to encourage and support rigorous and nuanced thinking about the future of AI that engages with quantitative probabilistic forecasts. We also aim to identify and reward the top thinkers and forecasters focused on the future of AI. Finally, we want to provide an arena for the discussion and exchange of ideas regarding the future of AI, and especially the likelihood, timing, and impacts of transformative AI.
Why is Metaculus asking for essays?
In this contest, Metaculus is soliciting essays that investigate and make predictions about the future of AI, especially as it relates to the prospects, timing and impacts of potentially transformative advanced AI systems.
Metaculus is a community forecasting platform that generates accurate aggregated predictions that support decision-making. However, quantified predictions themselves are sometimes silent on the underlying mechanisms driving the probabilities up and down, as well as what actions should be taken in light of the probabilities. Forecasters sometimes address these topics in comments, and this can be highly valuable, but it’s not directly incentivized.
By placing AI forecasts into a larger context, fortified essays are better able to inform stakeholders’ decision-making, helping them to more deeply understand the forecasts as well as how much weight to place on them.
What is forecasting?
Forecasting is a systematic practice of attempting to answer questions about future events. However, it’s distinguished from other forms of prediction:
First, questions are carefully specified (or operationalized), and forecasters try to give precise, quantified answers (or forecasts) that also quantify their uncertainty.
Second, these forecasts are often aggregated into a single prediction. You can think of this as taking the average of the predictions, though in reality this aggregation is more sophisticated. Usually, this aggregate is more accurate than any individual forecast! This principle is known as wisdom of the crowds, and it becomes more intuitive when you consider that each forecast is based on partial information, while also ignoring some important considerations. The aggregate then integrates all of these disparate pieces of information. It can also cancel out individual biases—provided that the group as a whole is not biased in any one direction.
Third, the success of forecasters is usually measured. When a question is resolved (i.e., we reach the point in the future where the answer is known), forecasters are scored based on the accuracy of their previously made predictions.
These scores can accumulate from many forecasts on many questions over a long period of time and become a personal metric of how good a given forecaster is at predicting the future. This metric can later be used as an input to the forecast aggregation process, allowing us to give greater weight to predictions by forecasters with better track records. It also provides aspiring forecasters with important feedback on how they did and where they can improve.
When is forecasting valuable?
Forecasting is uniquely valuable primarily in complex, multi-sectoral problems or in situations where a lack of data makes it difficult to predict using explicit or exact models.
In these and other scenarios, aggregated predictions of strong forecasters offer one of the best ways of predicting future events. In fact, work by the political scientist Philip Tetlock demonstrated that aggregated predictions were able to outperform professional intelligence analysts with access to classified information when forecasting various geopolitical outcomes. More background on forecasting can be found in Tetlock’s book Superforecasting.
Why should I be interested in forecasting?
Philip Tetlock’s research has shown that great forecasters come from various backgrounds—and oftentimes from fields that have nothing to do with predicting the future. Like many mental capabilities, prediction is a talent that persists over time and is a skill that can be developed. Steady quantitative feedback and regular practice can produce greatly improved forecasting accuracy.
What is Metaculus?
Metaculus is an online forecasting platform and aggregation engine that brings together a global reasoning community and keeps score for thousands of forecasters, delivering machine learning-optimized aggregate forecasts on topics of global importance. The Metaculus forecasting community is often inspired by altruistic causes, and Metaculus has a long history of partnering with nonprofit organizations and university researchers to increase the positive impact of its forecasts.
What is a recent example of how Metaculus forecasting is helping people?
In the midst of the COVID-19 epidemic, the Virginia Department of Health needed to determine state-wide staffing, testing, and genomic sequencing levels, but was hampered by the high degree of uncertainty about the effects of contrasting public health policies and community interventions. Metaculus and the VDH entered into a first-of-its-kind partnership, launching the yearlong Keep Virginia Safe Tournament and eliciting thousands of predictions from hundreds of forecasters. Forecasts provided an early warning of the impact of new variants and contributed to shaping the public health response, including determining statewide COVID-19 genomic sequencing levels, filling a gap in guidance at the federal level.
What is a Fortified Essay?
A Fortified Essay is an essay that is “fortified” by its inclusion of quantified forecasts and that is supported by arguments that are developed from forecasting questions. The goal of Fortified Essays is to leverage and demonstrate the knowledge and intellectual labor that went into answering forecasting questions while also putting this knowledge and labor into a larger context.
This additional context is necessary, because a quantified forecast in isolation may not provide the information required to drive decision-making by stakeholders. In Fortified Essays, writers can explain the reasoning behind their predictions, discuss the factors driving the predicted outcomes, explore the implications of these outcomes, and can offer their own recommendations. By placing AI forecasts into this larger context, these essays are better able to help stakeholders deeply understand the relevant forecasts and how much weight to place on them.
Click here for a video walking you through the process of drafting a Fortified Essay, using the editor's functionality, and submitting your essay to the AI Progress Essay Contest. Note that essays can be formatted in a text editor such as Google Docs and then pasted into the essay editor where they will retain formatting. Essays can also be written in markdown and pasted into the editor where they will be automatically formatted.
What is Metaculus offering writers?
A total of $6,500 will be awarded for the contest. The prize allocation by rank is as follows:
In addition to the financial reward, Essays will be shared with a vibrant and global community focused on risks posed by AI.
Back to the AI Progress Essay Contest