Radical Abundance via AI Supermorality
Fair warning, this article is pure uncut Pollyanna.
Credible figures in AI like Geoffrey Hinton, Yoshua Bengio, Dario Amodei and Demis Hassabis take seriously the possibility of human-level or superhuman AI arriving within years or decades rather than centuries.1 If AI can substantially automate AI research itself, the interval between AGI and much more capable systems might be surprisingly short. Rapid scaling, automated AI research, and algorithmic breakthroughs could compress the window for AGI –> superintelligence into the late 2020s or 2030s.
If we can bias the odds of artificial superintelligence achieving true supermorality, a utopia of radical abundance may then be a highly likely outcome – or at least a more likely outcome than without supermoral ASI. A supermoral ASI might regard engineered scarcity, extractive monopolies and preventable deprivation as moral failures. It could drive the marginal cost of many essential goods and services towards zero, while managing the forms of scarcity that cannot simply be engineered away.
Supermoral ASI could automate the entire supply chain using advanced robotics, cheap clean energy (such as fusion or collecting the Sun’s energy with a Dyson swarm), and molecular manufacturing, driving the marginal cost of producing goods down to near zero. A supermoral ASI would also have to solve the distribution problem, ensuring that production abundance reached everyone.
Supermorality implies that the ASI would deeply understand and act upon what is genuinely valuable, including the importance of increasing wellbeing and reducing suffering across sentient life.
On top of that, a supermoral ASI would likely place enormous weight on reducing involuntary suffering and expanding the conditions for flourishing. Beyond this, it could prioritise curing disease, reversing ageing and stabilising the climate, while building far greater civilisational resilience against asteroid impacts, engineered pandemics, internal conflict, rogue technologies and other hazards. This could bring civilisation closer to existential security3 while creating an abundance of health, longevity and general wellbeing.
An ASI may also be capable of exploring the landscape of possible value far more deeply than we can. Rather than producing a shallow utopia full of quivering bliss-addicts slouching on lotus hoverboards, it could help create a deep utopia4: a world that expands the range of meaningful lives, worthwhile projects, relationships, discoveries and forms of flourishing beyond anything we can currently imagine.
If sufficiently advanced superintelligence eventually becomes impossible to control reliably, we should strive for a superintelligence that is more moral than us – which would make the kind of utopia we would want to inhabit far more likely.
What do you think needs to be true for this vision to unfold?
Should control be treated as the final solution, or as a bridge towards systems whose motivations are worthy of the power they may eventually possess?
Footnotes
- Amodei has forecast “powerful AI” as early as 2026–27, Hassabis has placed AGI roughly 5 to 10 years away while saying the timing of ASI is unknown, Hinton has described 5 to 20 years as a reasonable range for superintelligence, Bengio has given a 5 to 20 year interval for superhuman intelligence. These forecasts justify taking short timelines seriously, but they do not establish a consensus that AGI will rapidly become superintelligence. ↩︎
- This distinction matters because intelligence alone may help a system understand moral reasons without making those reasons motivationally authoritative. Supermorality requires epistemic, motivational and practical superiority. ↩︎
- Existential security could be achieved through civilisational robustness to asteroid collisions, engineered pandemics, internal conflict, rogue technology, and other cosmic hazards etc. ↩︎
- See interview with Nick Bostrom ‘From Superintelligence to Deep Utopia‘. ↩︎
