M

Back to the Szilard Contest

Szilard Contest FAQ

What is forecasting?

Forecasting is a systematic practice of attempting to answer questions about future events. However, it’s distinguished from other forms of prediction:

First, questions are carefully specified (or operationalized), and forecasters try to give precise, quantified answers (or forecasts) that also quantify their uncertainty.

Second, these forecasts are often aggregated into a single prediction. You can think of this as taking the average of the predictions, though in reality this aggregation is more sophisticated. Usually, this aggregate is more accurate than any individual forecast! This principle is known as wisdom of the crowds, and it becomes more intuitive when you consider that each forecast is based on partial information, while also ignoring some important considerations. The aggregate then integrates all of these disparate pieces of information. It can also cancel out individual biases—provided that the group as a whole is not biased in any one direction.

Third, the success of forecasters is usually measured. When a question is resolved (i.e., we reach the point in the future where the answer is known), forecasters are scored based on the accuracy of their previously made predictions.

These scores can accumulate from many forecasts on many questions over a long period of time and become a personal metric of how good a given forecaster is at predicting the future. This metric can later be used as an input to the forecast aggregation process, allowing us to give greater weight to predictions by forecasters with better track records. It also provides aspiring forecasters with important feedback on how they did and where they can improve.

When is forecasting valuable?

Forecasting is uniquely valuable primarily in complex, multi-sectoral problems or in situations where a lack of data makes it difficult to predict using explicit or exact models.

In these and other scenarios, aggregated predictions of strong forecasters offer one of the best ways of predicting future events. In fact, work by the political scientist Philip Tetlock demonstrated that aggregated predictions were able to outperform professional intelligence analysts with access to classified information when forecasting various geopolitical outcomes. More background on forecasting can be found in Tetlock’s book Superforecasting.

Why should I be interested in forecasting?

Philip Tetlock’s research has shown that great forecasters come from various backgrounds—and oftentimes from fields that have nothing to do with predicting the future. Like many mental capabilities, prediction is a talent that persists over time and is a skill that can be developed. Steady quantitative feedback and regular practice can produce greatly improved forecasting accuracy.

What is Metaculus?

Metaculus is an online forecasting platform and aggregation engine that brings together a global reasoning community and keeps score for thousands of forecasters, delivering machine learning-optimized aggregate forecasts on topics of global importance. The Metaculus forecasting community is often inspired by altruistic causes, and Metaculus has a long history of partnering with nonprofit organizations and university researchers to increase the positive impact of its forecasts.

What is a recent example of how Metaculus forecasting is helping people?

In the midst of the COVID-19 epidemic, the Virginia Department of Health needed to determine state-wide staffing, testing, and genomic sequencing levels, but was hampered by the high degree of uncertainty about the effects of contrasting public health policies and community interventions. Metaculus and the VDH entered into a first-of-its-kind partnership, launching the yearlong Keep Virginia Safe Tournament and eliciting thousands of predictions from hundreds of forecasters. Forecasts provided an early warning of the impact of new variants and contributed to shaping the public health response, including determining statewide COVID-19 genomic sequencing levels, filling a gap in guidance at the federal level.

What will Metaculus do with data collected for this tournament?

The information Metaculus collects based on the individual predictions on nuclear risk is publicly available to anyone interested and is not used for any commercial purposes. With these forecasts, Metaculus seeks only to support the research of Rethink Priorities and other nuclear risk initiatives.

The best forecasting essays in this contest will be selected by Peter Scoblic, Matt Korda, and Michael Aird. They will then be provided to Rethink Priorities and published on the non-profit Effective Altruism Forum.

What is Rethink Priorities?

Rethink Priorities is a research organization that conducts critical research to inform policymakers and major foundations about how to best help people and animals in the present and the long-term future. The organization follows the principles of “effective altruism” and so is dedicated to using their resources in the way that will have the greatest impact. This tournament was started because nuclear risk is a promising and important but relatively neglected area.

What is Effective Altruism (EA)?

Effective altruism is a philosophical and social movement that uses evidence and reason to determine how to benefit others as much as possible before then taking action on that basis. The guiding principles of effective altruism are: commitment to others, a scientific mindset, openness, integrity, and a collaborative spirit. Conclusions drawn from the nuclear risk tournament and Szilard essay contest will be published publicly on the Effective Altruism Forum, which is read by a global community of effective altruists.

What is a Fortified Essay?

A Fortified Essay is an essay that is “fortified” by its inclusion of quantified forecasts and that is supported by arguments that are developed from forecasting questions. The goal of Fortified Essays is to leverage and demonstrate the knowledge and intellectual labor that went into answering forecasting questions while also putting this knowledge and labor into a larger context.

This additional context is necessary, because a quantified forecast in isolation may not provide the information required to drive decision-making by stakeholders. In Fortified Essays, writers can explain the reasoning behind their predictions, discuss the factors driving the predicted outcomes, explore the implications of these outcomes, and can offer their own recommendations. By placing nuclear risk forecasts into this larger context, these essays are better able to help stakeholders deeply understand the relevant forecasts and how much weight to place on them.

Why is Metaculus awarding the Szilard Prize?

We wanted to identify and reward insightful thinkers and forecasters on the topic of nuclear risk while providing a probabilistically and quantitatively minded arena for discussion and exchange. We hope this initiative will support a style of writing that’s quite distinct from most writing on nuclear risk, as it often leaves its predictions about the future too underspecified to act on or to falsify.

What is Metaculus offering writers?

In addition to the financial reward, essays will be shared with a vibrant and global effective altruism community of thousands of individuals and dozens of organizations—many of which focus on the existential risks posed by nuclear weapons and conduct research on the best policy solutions for mitigating these risks. Strong essays can impact the policy solutions currently being developed, which are then provided to stakeholders to support their decision-making.

Fortified Essay writers will also have their work reviewed by a distinguished panel of judges:

Peter Scoblic is a Fellow with the Social Innovation and Change Initiative at the Harvard Kennedy School Center for Public Leadership, where he researches the ways in which organizations navigate uncertainty. He is also the co-founder of Event Horizon Strategies, a senior fellow with the International Security Program at New America, and an instructor in Harvard’s Professional Development Program.

Matt Korda is a Senior Research Associate and Project Manager for the Nuclear Information Project at the Federation of American Scientists, where he co-authors the Nuclear Notebook. Matt is also an Associate Researcher with the Nuclear Disarmament, Arms Control and Non-proliferation Programme at the Stockholm International Peace Research Institute (SIPRI).

Michael Aird is a Research Scholar at the Future of Humanity Institute, as well as a Senior Staff Manager at Rethink Priorities. His research interests include the long-term future implications of various matters related to politics, policy, and security, such as nuclear weapons and authoritarianism.


Back to the Szilard Contest