M

This Discussion is now a Draft.

Once it's ready, please submit it for review by our team of Community Moderators. Thank you!

Pending

This Discussion now needs to be approved by community moderators.

Discontinuing the Final Forecast Bonus

Metaculus Meta

Previously, a 50% point bonus was awarded for a forecaster’s final prediction on a question. This final prediction bonus was meant to incentivize forecasts on questions near closing, and it only applied when the question closed before it resolved. (As opposed to resolving early, before the question had the time to close naturally.)

After feedback from the community and internal discussion, we determined that this bonus had more drawbacks than benefits, namely:

  1. It lowered the relative importance of forecasts made at the beginning of the question's lifetime.
  2. It created a discrepancy: Questions that resolved before they closed (and didn't offer the bonus) received 50% fewer points than questions that closed before they resolved (and featured the bonus).
    • This was compounded by the fact that questions that resolve early already receive fewer points—in order to preserve the properness of our scoring rule. (See an explanation for why that is here.)
  3. It was one more rule for users to remember.

Discontinuing the bonus makes our scoring more consistent and better incentivizes making accurate predictions over the entire lifetime of a question. But it has two drawbacks:

  1. The now-retired contests rely on points, and some of them have unresolved questions.

  2. Some forecasters have been keeping track of the points-per-question metric, and this would make comparisons difficult before and after the scoring update.

To address this, we are:

  1. Only applying the new scoring (without bonus) to questions with a closing date after 2022-04-30. All contest questions have been closed for a long time, so they are all unimpacted.

  2. Multiplying all points by 1.5. This helps keeping the points-per-question metric comparable, though it will increase the points earned on questions which resolve early.

What this means for your points

  1. Already resolved questions are not affected. The change only applies to questions with a close date after 2022-04-30.

  2. Questions that resolve after they close will no longer award 50% more points than other questions.

  3. The displayed expected points for open questions will reflect the new rule, with no bonus points.

  4. This update has no effect on the tournament scoring system.

We welcome your questions and feedback.

Categories:
Metaculus