The results are in, and a big congrations go to our winners jzima and Barbarossa and our runners up AndrewMcKnight and ElliotOlds. But how did everyone else do? In short, not so well.
Our winner received 748 points, but the median score amongst all cryptocurrency players was only 7 points and the average score was way down at −73 points. Only 25 players broke 100 points. Player overconfidence combined with our log scoring system really hurt here: just one very overconfident prediction could easily send someone's score tumbling down from the top of the ranks. That said, there was a top tier of eight players who got at least 590 points (with a large gap separating the 9th player), and, as we'll see below, they used different confidence strategies to get there.
After some internal deliberating and data crunching, we're finally ready to announce the results of the 2017 Cryptocurrency Prediction Competition. First we'll say a few words on the reason for delay, then the results.
As those following the competition know, we got stalled on four questions that were based on the highest or total amount amount of initial coin offerings (ICOs) in October and November. It turns out that the data for ICOs is terribly incomplete and unreliable, with numbers dramatically varying from one source to another. On top of this, it seems fairly clear that there is an order-of-magnitude uncertainty in the ICO funds actually raised by Paragon. In the end, we decided to resolve these questions per their letter, keeping with Metaculus's general policy. However, in recognition that these questions could be considered flawed because based on a misleading resource, we've decided to also issue prizes to the two predictors who would have won had we resolved these four questions as ambiguous.
So, without further ado, the winners are:
Fortune mag just came out with a big list of predictions.
*Update Jan 1, 2018: This position has now been filled.
There have been a lot of updates to the site over the past few weeks. Some are big and obvious, and some are less so. We hope that they collectively make Metaculus more fun and easier to use.
First, and most obvious to those reading this, we've brought the Metaculus blog onto the Metaculus site. Feel free to leave comments on news postings just as you would for Metaculus questions! You can always find the news in a link at the bottom of any page.
The biggest update, however, is the introduction of tachyons. Tachyons are Metaculus's in-game currency which can be used to go backwards in time to remove or change an old prediction, or to zoom forwards and sneak a peak at the Metaculus prediction before it closes. You can currently earn weekly tachyons just by logging in, and bonus tachyons by completing achievements or tutorials. We have more plans for tachyonic uses and how one might earn them more quickly or trade them with other players, but we'd love to hear your feedback and ideas too.
It's been interesting, educational, and frustrating in equal measures to see the cryptocurrency competition play out.
From the game of Go to self-driving cars, there is widespread agreement that progress in artificial intelligence has accelerated in recent years. Yet there is remarkably little consensus as to what this progress portends in the coming years or decades. This is especially true of the big question of if and when we will have general-purpose human (or super-human) level AI. I’ve seen estimates by credible AI researchers range from ten years to “never”, and everywhere in-between!
While difficult, this question is extremely important. Humans have (for better and worse) agency over the fate of planet Earth due to our intellectual capability and technology; as we create systems with comparable capabilities on various axes, it will profoundly change the course of society and perhaps life itself.
Toward making good decisions and allocating resources wisely, it is vital that we have some ability to predict how various types of AI systems and capabilities will unfold; that is the aim of the new “AI Milestones” question series on Metaculus.
Happy new year!
A whole boatload of questions resolved in the last couple of weeks of December (many of the form “Will X happen in 2016?”. Somewhat bizarrely, just about all of them (see the list here) resolved negative! In the great majority of cases, these negative resolutions were forecast correctly. I guess we all have to come up with some more likely things to predict about…
The leaderboard has now moved around a lot, and a couple have now attained “Predictor” status, homing in on “superpredictor.” We may even have a seer or a prophet soon!
I looked for a good numerical index with which to ask the question “will 2017 be better than 2016?”, but could not see a great candidate — if anyone else does please leave a comment, or just suggest a question onsite.
Although a lot can be learned from asking about whether or not various things will occur in the future, there are many cases where it would be much nicer to predict a number, such as a date by which something happens, or an amount of money, sales, citations, hits, etc., etc.
As of now, such questions can be posed on Metaculus. Rather than input a probability of yes, users may now define a probability distribution, i.e. a probability assigned to each possible value within a defined range. You get more points if you assign a higher probability to the actual realized number, but you have a fixed amount of probability to assign; thus you can make a narrow distribution if you are fairly certain of a particular value, and a wider distribution to reflect more uncertainty. See the FAQ for a lot of detail on how this works, and how these questions are scored at resolution.