From the game of Go to self-driving cars, there is widespread agreement that progress in artificial intelligence has accelerated in recent years. Yet there is remarkably little consensus as to what this progress portends in the coming years or decades. This is especially true of the big question of if and when we will have general-purpose human (or super-human) level AI. I’ve seen estimates by credible AI researchers range from ten years to “never”, and everywhere in-between!

While difficult, this question is extremely important. Humans have (for better and worse) agency over the fate of planet Earth due to our intellectual capability and technology; as we create systems with comparable capabilities on various axes, it will profoundly change the course of society and perhaps life itself.

Toward making good decisions and allocating resources wisely, it is vital that we have some ability to predict how various types of AI systems and capabilities will unfold; that is the aim of the new “AI Milestones” question series on Metaculus.

With 16 questions at launch (and more to be added in time), the question series probes at both the fairly short-term and fairly long-term. The hope will be both to gather predictions from many people with strong insight, but also to be able to track how those projections change over time, and to weight them by predictors’ track records. Metaculus is quite rare in utilizing the quantifiable prediction credibility that can only really be built by actually getting well-defined questions right!

In terms of particular questions, the series touches upon labor automation with questions on robots that could replace human fine-motor and general assembly skills, autonomous trucking, and autonomous taxis.

Virtual assistant, chatbots, and the like are addressed in questions about a chatbot prize, speech transcription, and low-level programming.

There are a number of questions about humans vs. AIs in game-playing, which like Chess and Go, indicate progress toward both some specific and some general capabilitis. We target Poker, Angry Birds, Starcraft 2, and world cup soccer.

Then there are tests of AI as such. The classic Turing Test is addressed via the Loebner Prize. A more recent scheme of testing understanding, the “Winograd Scheme” also has an annual competition, and a question about it. A third question asks when a machine can out-compete a trio of smart capable humans at a very general human-administered exam, thus displaying human intellectual parity in terms of general-purpose problem solving capability. Finally we ask when AI systems will be able to write general-purpose very highly sophisticated programs (programs that are themselves as capable as contemporary programmers of AI systems). Such systems would almost certainly be capable of recursive self-improvement or at the very least creating a very rapid proliferation of human-parity systems, probably portending something like an AI “singularity.” So, how far off is that?

We’re interested in your predictions, and also any suggestions for good, well-defined questions. We’d also eagerly invite you to share this with — and invite the participation of — people you think would have insight into AI timelines and capabilities: the more participants the better!