What if Moore’s Law breaks ?
3
*This essay was submitted to the*[*AI Progress Essay Contest*](https://www.metaculus.com/project/ai-fortified-essay-contest/)*, an initiative that focused on the timing and impact of transformative artificial intelligence. You can read the results of the contest and the winning essays*[*here*](https://www.metaculus.com/questions/11429/results-from-the-ai-progress-essay-contest/)*.* In 1965, Gordon Moore posited that roughly every two years, the number of transistors on microchips would double. Commonly referred to as Moore's Law, this phenomenon suggests that computational progress will become significantly faster, smaller, cheaper, and more efficient over time. Rather than a law of physics, Moore's law is an observation and projection of a historical trend.  Although remarkably consistent over more than 50 years, Gordon Moore himself, stated that this projection cannot be sustained indefinitely, noting that transistors would eventually reach the physical limits of miniaturization, at an atomic level. It is generally agreed that there is a minimum size under which transistor behavior becomes less and less effective and leakage currents begin to interfere with the logical operations performed by the transistor. As a result, most forecasters, including Gordon Moore, expect Moore's law will end by around 2025. As a consequence of such a phenomenal rate of improvement, maintained for such a long period of time, the world has experienced, and benefited from, an exponential increase in the availability of computational power (and also chip memory), provider at ever more affordable rates, which in turn paved the way for the emergence of ever more complex applications, including features generally considered as belonging to the field of Artificial Intelligence (AI). It is generally accepted that machine intelligence depends on computing power, memory, data and AI algorithms. While computing power and memory have been growing at an exponential rate, as suggested by Moore’s Law, data is being collected through an ever-increasing array of manners, including internet usage and widespread usage of an array of sensors. At the same time, the algorithms supplied to machines have shown increasing results in effectively using and combining their computing power, memory and data fed to them. As we currently understand it, AI would not be feasible if not for – among other things - the availability of vast amounts of cheap computation power and memory. In fact, one could argue that the emergence of software exhibiting signs of AI would likely be inevitable, depending on whether or not Moore’s Law could hold true for a sufficient amount of time. Although general purpose AI is still – if ever - far from being reality, with some expert forecasts aiming – no sooner – then 2060, there are many useful abilities and tasks that can be described as showing some form of intelligence. AI, like electricity or the steam engine, is best described as a general-purpose technology that can be deployed in talking different problems. The progress made in AI has reached a point in which AI systems can (out)perform as well as humans in a variety of tasks, generally measured through ability specific benchmarks and/or competitions. As an example, the progress in machine classification of images, as measured by the error rate (%) of the ImageNet competition winner, suggests that in the recent past a steady improvement of roughly 1/3 decrease in the competition winners error rate per year has been achieved.  The chart above shows (in blue bars) the progress in machine classification of images, as measured by the error rate (%) of the ImageNet competition yearly AI winner. The red line, represents the error rate of a trained human on a particular task. Although raw computing power is certainly not the only driver of AI performance, it is somewhat curious to observe that at least in some specific abilities (in our example, images classification), Moore’s law seems to correlate (even though correlation does not necessarily imply causation) well with the best AI systems error rate. In the example above, the error rate roughly halves every two years, which is the same time frame Moore’s Law states for the doubling of the number of transistors in a chip. Other AI fields present similar, if not equal, relationships. This observation seems to suggest that if Moore’s law were to become seriously stunted, the rate of progress in the field would be detrimentally affected. Although Moore’s Law will reach a physical limitation, this does not necessarily mean that chip development cannot improve through continuation of technological progress in a variety of other areas, including new chip architectures and designs, quantum computing and even AI and machine learning that could be used and improve future chip design. If observed, chip development in other areas could offset (at least partially) the impact of the physical limitations confronting Moore’s Law, which in turn, could keep sustaining continued progress in AI systems. As an example of possible breakthroughs, Quantum Computing, which is still considered an emerging technology, can contribute to reducing computing costs after Moore’s law comes to an end. Quantum Computing is based on the evaluation of different states at the same time where classical computers can calculate one state at one time. The unique nature of quantum computing can be used to efficiently train neural networks, currently the most popular AI architecture in commercial applications. At present, Quantum Computing is perhaps the most likely alternative venue in which AI algorithms could have a chance to unlock singularity - a hypothetical point in time at which an up-gradable intelligent agent will eventually enter a "runaway reaction" of self-improvement cycles, each new and more intelligent generation appearing more and more rapidly, causing an "explosion" in intelligence and resulting in a powerful superintelligence that qualitatively far surpasses all human intelligence. Even if no such developments are achieved at the scale needed to fully compensate the physical limitations implied by Moore’s Law, one should not discard the possibility of breakthroughs in the conceptual development and implementation of AI in terms of better data handling, algorithms, or learning techniques. Where they to materialize, these breakthroughs could keep pushing the field forward at the same – or perhaps even greater – rate. As a fairly recent example, one could draw on the progress brought about by the widespread use of deep neural networks techniques, made especially notorious by Deep Mind’s Alpha Go win over world Go champion Lee Sedol in 2016. AlphaGo, and subsequent platforms developed by Deep Mind, such as well-known AlphaFold and AlphaCode, as well as other systems employing similar techniques have had a remarkable influence on improving the performance in a variety of areas by virtually all the best AI systems. One could also consider a different scenario, in which Moore’s Law holds true for long enough to enable very complex and efficient AI systems that could help push the limits of human knowledge in various fields and thus bring forward the discovery of new venues for improving chips and, also, AI in a continuously virtuous cycle. Where Moore’s Law to hold forever, barring any catastrophe on humanity, this type of (singularity) scenario would almost certainly come about … I guess the real question is how far away could this singularity scenario be? As Moore’s Law reaches the physical limits of its rule, one cannot foretell whether progress in chip manufacturing and in AI complexity and efficiency will the surpass this breakthrough point and propel humanity into an unimaginable future of possibilities, or whether the physical limitations that support AI progress will prove insurmountable and uncompensated by breakthroughs in other areas, thus resulting in ever diminishing improvements to AI systems, which, in turn, could lead to the practical stagnation (albeit likely at a much higher level than what we can enjoy today) of the field as a whole. ---