Scroll to the bottom of this post for a sneak peek at what Metaculus "fan graphs" might look like and how they'll enhance visualization of time series data.
Faster, more consistent forecasting across all kinds of forecast questions
Question groups now support all question types, including binary and date questions. Predict efficiently across interconnected outcomes and ensure that related predictions are internally consistent.
This post describes several updates and enhancements to question groups. To read more about the basics of question groups, visit the FAQ here.
There can be only one
Who will be the next prime minister of France? Which country will win the 2022 FIFA World Cup? Some contests can only have one winner. When a question has multiple, mutually exclusive potential outcomes, question groups can bring binary subquestions together on one page to simplify comparisons and facilitate forecast updates.
Forecasts can be input for each outcome, and additional options may be added over the life of the question group. For example, on this question, as political winds shift, additional candidates for prime minister may be added to the page.
Question groups also allow for many questions to be asked simultaneously, but it's not required that you predict on every subquestion or outcome within a group. This World Cup question comprises what are in fact 32 different binary questions, and each is scored independently.
While question groups with multiple binary outcomes are not identical to multiple-choice questions, this feature update does streamline the process of forecasting on mutually exclusive outcomes.
For some questions, only a set number of binary outcomes can resolve positively. Only five cryptocurrencies can be in the top five by market cap. Question groups make it easier to keep your forecasts coherent by allowing you to see your predictions on related binary outcomes all in one place.
Consistency is key
Forecasting multiple sequential dates or a numerical outcome at different time points is challenging enough without worrying about conflicting forecasts. Question groups give you confidence your related forecasts make sense relative to each other by letting you see them all, in sequence, in a single location. For example, you might confirm you’ve predicted progressively wider distributions the further into the future you look in order to reflect your greater uncertainty.
Radio buttons next to the table change which question's plot is displayed and which subquestion is being forecast.
Multiple forecasts can be submitted with a single click. Simply input your forecast for each subquestion or outcome and click the Update button when finished.
Getting started with question groups
We’ve created several question groups comprising binary subquestions to showcase what this new feature is capable of:
- Who will be the prime minister of France on 2022-07-01?
- Who will win the 2022 FIFA World Cup?
- Keynesian Groupy Contest: Interquartile Range
We’re also proposing creating the following question groups. Comment below with whether you’d like to see these as groups, and let us know other question groups you want to forecast on.
- Who will be elected president of the United States in 2024?
- Who will be the Republican nominee for US President in 2024?
- Which country will be the first to select >10% of their newborns for intelligence?
- Who will be the Democratic nominee for US President in 2024?
- What will be the total number of confirmed or suspected cases of Monkeypox in the following regions in 2023?
- Who will be the next leader of the UK Conservative party?
- If life is discovered in the solar system by 2050, where will it be first discovered?
- If American Infrastructure is attacked by a Russian Cyberattack in 2022, which systems will be targeted?
- If there is at least one nuclear detonation by 2050, in which countries will there be at least one fatality?
- Will any of these countries approve a cultivated meat product for consumption by April 2023?
- Will any of these countries ban a major crop for export before April 2023?
- How much money will Open Philanthropy grant towards Global Health and Development in 2025?
Numerical question groups
Question groups with numerical subquestions have already been implemented in the FluSight Challenge and the Ukraine Conflict Tournament. Access them below:
- The FluSight Challenge
- How many military equipment losses will Oryx report for Ukraine in 2022?
- How many military equipment losses will Oryx report for Russia in 2022?
What happens to comments?
When individual questions are combined and converted into a question group, your comments on those questions are all moved to the new question group’s comment section and are flagged to indicate the change.
How are question groups scored?
Each outcome within a question group is its own subquestion and is scored as an independent question.
As an elaboration of question groups, we’re exploring introducing fan graphs for date-based questions. These would join a line chart for past resolved questions with "fans" that spread out over time to represent uncertain future outcomes. The mockups below give a sense of how these might look in practice. Share your thoughts in the comments!