We’ve recently rolled out a significant overhaul of the way scoring works, which we believe will lead to more accurate predictions, and more overall fun.
The update to the scoring system is accompanied by some behind-the-scenes changes that will allow an upcoming expansion of the types of questions we can pose. A full description of the scoring system can be found at the FAQ. Here’s what you need to know:
- You are still best off entering your best estimate of the outcome probability given all available information.
- Scoring is still based on a combination of being right, and also being more right than others, though you’ll see that in detail the numbers have changed a bit. What’s new is that your awarded points are a time-average of your point on-the-line at any given time, starting at question open and ending at question close. This means that you will maximize your points if:
- You predict as early as possible, since your points on-the-line is zero until your first prediction, and
- You update your prediction whenever you have reason to think your best estimate of the probability has changed.
- As before, you dial in your prediction using the slider, and you’ll see that points on-the-line change as you slide it. Once you make a prediction you will also see a “score history” tab that will show you how your points vary with time, in reflection of changes in your prediction, the community’s prediction, and the number of predictors.
The big advantage of this new scoring system is that it rewards players for being right for as much time as possible, and for making predictions early, rather than just sneaking in with a prediction at the end. It also fixes a concern with how early resolutions are treated. The disadvantage is that now a last-minute prediction has very little impact. We’re considering tweaking the system so that a fraction of your score comes from the final prediction, and the rest from your averaged score.