Crowd Forecasting: What It Is and How Metaculus Leverages It
1
Crowdsourced forecasting is simple in structure and powerful in practice: many people submit quantitative judgments about well-posed future events, we aggregate those judgments, and decision makers use the result to update beliefs and plans. The goal is a live feedback loop that continually stress-tests assumptions and surfaces new lines of reasoning while events are still in flux. *** ### What “crowd” means on Metaculus * **Quantitative forecasts**: human judgmental forecasts and bot predictions in the form of probabilities and distributions. * **Structured aggregation**: Metaculus shares the Community Prediction, a median of current forecasts. We offer different aggregations in our Aggregation Explorer, but the default is a recency-weighted median.  * **Rationales as data**: forecasters supply brief arguments, links, and decompositions. These are not comments for color.  Result: a continuously updated outside view on important questions across a host of domains. *** ### Why it belongs in an analysis stack * **Early warning**: trend shifts and threshold crossings trigger review before decisions ossify. * **Assumption audits**: rationales expose hidden premises and missing base rates. * **Model pressure testing**: forecasts compare the world implied by a model with the collective judgment of informed generalists and domain specialists. * **Learning dynamics**: resolution and scoring create an institutional memory about which signals and framings paid off. *** ### A short lineage worth knowing * **1907**: Francis Galton observes crowd accuracy in the famous ox-weight contest. * **Iowa Electronic Markets**: small-stakes, research-focused markets for elections. * **Intelligence Advanced Research Projects Activity, Aggregative Contingent Estimation**: large controlled programs showed that trained forecasters plus principled aggregation beat standard baselines that relied on classified inputs. * **Modern platforms**: public and private systems now support policy, corporate strategy, and research questions at scale. Key idea that survived every wave: heterogeneous but motivated people, well-specified questions, and proper aggregation outperform ad hoc deliberation. *** ### What you get if you run a program with us * **Forecast time series** * **Rationales at a glance** * **Calibration and bias views** * **Indexes pulling together many questions within a topic area into an overall indicator** * **Snapshot reports** * **Audit trail** Optional automations: * **Threshold alerts**: notify when a probability crosses a policy trigger. * **Counterfactual trackers**: side questions that test key assumptions, for example uptake rates or lead indicators. * **Question creation and resolution automation** * **Forecast automation via a team of vetted bots** *** ### How a Metaculus program runs end to end 1. **Define decisions, not just topics**. Specify what choices forecasts will inform and what thresholds matter. 2. **Decompose**. Break the decision into resolvable sub-questions with crisp resolution criteria and timelines. 3. **Draft and pre-mortem**. Write questions, run a fast adversarial review to catch ambiguity or other issues. 4. **Launch**. Recruit the right mix of domain experts and generalists, set cadence expectations. 5. **Aggregate and monitor**. Track the Community Prediction.  6. **Resolve and score**. Close questions cleanly and publish scores. 7. **Learn and iterate**. Update question templates, retire weak indicators, and promote forecasters with strong calibration. Time-boxed tournaments work well for exploration. Rolling programs fit ongoing operational decisions.