Metaculus and Markets: What’s the Difference?

by RyanBeck•Jun 4, 2025•Edited on Jul 30, 2025•7 min read

Metaculus and prediction markets both aggregate forecasts about world events, but they do so through notably different methods. Here we outline key strengths and weaknesses of each, and explain why we consider Metaculus’ approach particularly valuable.

## How do they differ?

Imagine a market predicting “whether it will rain tomorrow.” On prediction markets, traders can buy "Yes" shares that pay \$1 if it rains and \$0 if it doesn't. For example, if “Yes” shares are trading at \$0.75, the market is “predicting” a 75% chance of rain. If you believe there's really an 85% chance, you could buy shares at \$0.75. Depending on a number of factors that are beyond the scope of this article (market liquidity, book depth, etc.), your purchase would typically drive the market price up, closer to your prediction. When tomorrow comes, you earn \$1 per share if it rains (for a profit of \$0.25 per share), or lose your \$0.75 per share if it doesn't rain.

On Metaculus, instead of buying shares, you’d directly enter 85%. Your forecast would be [aggregated](https://www.metaculus.com/faq/#community-prediction) with other predictions, typically moving the 75% aggregated forecast higher. Once the outcome is known, you’re [scored](https://www.metaculus.com/help/scores-faq/#scores-section) on how accurate you were compared to other forecasters and compared to a “baseline” prediction that’s as uncertain as possible. (For example, on binary questions, this would be 50% — a coin flip.)&nbsp;

## Which performs better?

Both approaches improve our ability to forecast future events, and platforms using either method have achieved impressive accuracy and calibration. However, direct head-to-head comparisons between prediction markets and forecast aggregators are rare, making definitive conclusions difficult. Instead, each method has unique strengths and weaknesses depending on the application, and we suggest considering them in this context.

![](https://metaculus-web-media.s3.amazonaws.com/user_uploaded/metac-vs-markets_D2ekmH1.jpg)

### Limitations of prediction markets

Let’s first examine some issues specific to prediction markets that don't arise with Metaculus’ prediction aggregation approach:

* **Market mechanics break down at extreme probabilities.** Market frictions make markets less useful for very high or low probabilities. For example, if you believe an event has a 1% probability when the market shows 2%, the expected return on your capital is minimal — you'd risk \$0.98 to potentially gain just \$0.01 per share. This becomes even less attractive when accounting for market fees, making it impractical to express accurate beliefs at these probability extremes. This explains Michelle Obama’s 6% chance of becoming the 2024 Democratic nominee on [Polymarket](https://polymarket.com/event/democratic-nominee-2024?tid=1724174308005) in June of 2024.
* **Long-term markets tie up money.** In long-term markets, money is tied up in bets that don’t grow like investing does, such that&nbsp; one can usually get a better return by investing. As a result such markets tend to suffer from low liquidity. For example, see [this plot](https://wip.gatspress.com/wp-content/uploads/2024/05/thu9F-cumulative-traded-volume-on-the-2020-us-election-4-1024x897.png) from a [Works in Progress article](https://worksinprogress.co/issue/why-prediction-markets-arent-popular/) showing the trading volume on Betfair for the 2020 US presidential election. There was very little volume far in advance of the election, with most of the trading volume occurring only a month out from the election.¹ Some markets have introduced interest accrual on open positions, which can help ameliorate this effect.
* **Markets aren’t always about making the best forecast.** Prediction market incentives aren't always aligned with making the most accurate predictions. Consider that one potential use for prediction markets is hedging against risky outcomes. Additionally, prediction market participants are often incentivized to predict where they think the market price will move in the short term, rather than the ultimate outcome. For example, traders might 'buy the rumor' even if they believe the rumor is false, simply expecting others to drive up the price temporarily. This creates dynamics where market prices can be influenced by anticipated trading patterns rather than genuine probability assessments of the final outcome.
* **“Whales” have the potential to move markets.** Some observers have [raised concerns](https://asteriskmag.com/issues/05/prediction-markets-have-an-elections-problem-jeremiah-johnson) about the ability of massive bets made by participants with large bankrolls (often referred to as “whales”) to shift market probabilities. There is some uncertainty as to whether such large bets represent special insight or signal, or whether these kinds of bets overweight the views of a few individuals.
* **Markets don’t tell you what individuals think will happen.** Participants in prediction markets are expressing whether they think the probability is higher or lower than the market price, not making a forecast. If someone thinks the market is too low at 35% and bets accordingly, you don't know whether they think the true probability is 40% or 80%. This doesn't really impact the usefulness of the aggregate, but it does make the data less rich and informative, and harder to see the full distribution of beliefs like you can with the histograms for binary questions on Metaculus.
* **Markets can’t allow blind forecasts.** The ability to see individual forecasts not influenced by the responses of others is a valuable tool, and one markets don’t typically have the capability to allow. Trading on a market requires the ability to see the market price, whereas on Metaculus questions can have a “hidden period” that lasts as long as desired, allowing each forecaster to make their own prediction unbiased by others.
* **Performance in markets doesn’t solely reflect forecasting skill.** Excellent individual market performance might just signal proficiency at operating in markets, or ability to take advantage of bad bets made by others — or simply wealth (more money means more influence on the market). For example, see [this post](https://www.cspicenter.com/p/salem-tournament-5-days-in#:~:text=The%20first%20problem%20we%20saw%20was%20that%20there%20were%20some%20individuals%20who%20made%20a%20killing%20by%20taking%20advantage%20of%20those%20who%20did%20not%20know%20how%20the%20markets%20work%20$see%20discussion%20here$.) about a tournament organized on Manifold where traders took a large early lead just due to intelligent use of limit orders. Since Metaculus elicits individual probabilities from every forecaster, we can better assess and recruit excellent forecasters.
* **Metaculus performs comparably to markets without the need to manage a portfolio.** There are only a handful of apples-to-apples comparisons between platforms, but [these](https://www.metaculus.com/notebooks/15359/predictive-performance-on-metaculus-vs-manifold-markets/) [find](https://firstsigma.substack.com/p/midterm-elections-forecast-comparison-analysis) an [advantage](https://www.astralcodexten.com/p/who-predicted-2023) for Metaculus over prediction markets. Note that sample sizes tend to be small. However, there is also an [indirect comparison](https://calibration.city/) (indirect because it does not consider the same questions across platforms) which found that prediction markets are more calibrated. Instead of having to carefully manage your resources across markets you’re participating in, forecasters on Metaculus can simply enter probabilities and receive scores on those probabilities. This is simpler, more informative, and doesn’t sacrifice accuracy.

### Limitations of prediction aggregators

Conversely, markets offer some advantages over prediction aggregators:

* **Updates take longer.** To move the aggregate prediction, platforms like Metaculus need enough forecasters to update their forecasts, while a market price can move very fast if only a few large bets are placed. This means forecasts on Metaculus won’t move instantly when news breaks. On the other hand, this also means Metaculus is less prone to noise and large swings in probability.
* **Less “skin in the game”.** While head-to-head data is limited, it’s possible that the monetary incentives from betting on markets produce higher-quality forecasts. In our view, from comparisons that are available, this effect is not very noticeable, if it exists at all. Still, it’s reasonable to think that people staking their own money are more incentivized to get things right than a forecaster on Metaculus who only stakes their time and reputation to try to earn a share of a cash prize.
* **Equal weight to viewpoints might miss signal or special knowledge.** Prediction aggregators typically weigh each forecast equally, or may assign weight based on a forecaster’s track record. However, bettors willing to stake more money may be doing so because they have special knowledge or insight. This makes it harder for a particular forecaster’s “edge” to show up in the aggregate.
* **Fewer opportunities for arbitrage.** Arbitrage, where a bettor can exploit the differences in closely related markets to guarantee a payout, is a useful feature of markets that helps ensure prices are consistent between related forecasts. Since no money is staked on prediction aggregators, the incentive to arbitrage between questions, driving their probabilities into more consistent territory, is reduced.

## Metaculus Offers a Proven Method for Collaborative Prediction

As we’ve described here, Metaculus and markets produce a consensus probability by different means, and we think there are strengths and limitations to each of these approaches. In our view, Metaculus’ approach provides better transparency and avoids a number of issues that emerge in forecasting conducted on real-money or play-money markets. With Metaculus, you enter your true beliefs in the form of probabilities, get scored on your forecasts, and contribute to an aggregate that tends to be more accurate than individual participants. This approach not only fosters a collaborative forecasting environment but also consistently delivers insights that are reliable and trustworthy, setting Metaculus apart as a uniquely effective tool for decision-making.

***

¹ This was still the case for the 2024 election [on Polymarket](https://polymarket.com/event/presidential-election-winner-2024), which saw \$3.7 billion in volume, though the share of volume earlier than a month out from the election was higher than on the 2020 Betfair market. See [chart](https://metaculus-web-media.s3.amazonaws.com/user_uploaded/image_5.png), hoverable [dashboard](https://dune.com/ryooan/polymarket-2024-election-trade-volume), and [code](https://dune.com/queries/5212670/8574490).