Are you skeptical that controlling AI is enough? We have to make it care about the right things. As AI systems become more capable, we face a question more foundational than how to contain them: what values should they be aligned to? Upstream value alignment challenges the conventional reliance on “capability control” – tripwires, sandboxing,…
Emergent Machine Ethics (EME) posits that AI agents, through complex interactions, can develop ethical behaviours not explicitly programmed by humans. This emergent morality seems to align with moral naturalism, which suggests that moral truths are grounded in natural facts and processes. If AI systems can independently develop consistent ethical norms, it could indicate that moral…
Since I’ve discussed ‘indirect alignment’ between AI and humans elsewhere, I thought I’d dedicate specific writing to this topic. It’s important to get AI alignment right because intelligence is powerful, and we are heading towards superintelligence fast. AI safety hinges not just on what an AI can do, but on what it ought to do,…
In an era where artificial intelligence systems are increasingly tasked with decisions that carry ethical weight – from medical triage to autonomous weapons – the question of whether machines can authentically engage with morality has never been more pressing. To explore this issue, we turn to philosopher David Enoch, a leading advocate of moral realism…
When the machines outthink us, will they outmaneuver us? Barrat’s latest is a wake-up call we can’t afford to ignore. Having had the privilege of previewing James Barrat’s latest work, The Intelligence Explosion (Amazon link, Google Book link), I found it to be a compelling and thought-provoking exploration of the rapidly evolving landscape of artificial…
In his 2011 paper, Complex Value Systems are Required to Realize Valuable Futures , Eliezer Yudkowsky posits that aligning artificial general intelligence (AGI) or artificial superintelligence (ASI) with human values necessitates embedding the complex intricacies of human ethics into AI systems. He warns against oversimplification, suggesting that without a comprehensive inheritance of human values, AGI…
Norm-sensitive cooperation might beat brute-force optimisation when aligning AI with the complex layers of human value – and possibly cosmic ones too. Nick Bostrom suggests that AI systems designed with humility and a cooperative orientation are more likely to navigate the complex web of human and potentially cosmic norms than those driven by rigid utility…
Surviving an intelligence explosion is just Level One. Another layer to the Great Filter might be whether our civilisation, or a greedy AI singleton can grow up enough not to become a cosmic jerk. You might brute-force your way to technological maturity, but that doesn’t mean you get a seat at the galactic grown-up table….
I just finished Mo Gawdat’s book Scarey Smart. The overview says: By 2049 AI will be a billion times more intelligent than humans. Scary Smart explains how to fix the current trajectory now, to make sure that the AI of the future can preserve our species. This book offers a blueprint, pointing the way to…
Nick Bostrom: Superintelligence Could Be a Year Away—or Not Back in 2012, when I first chatted with Nick Bostrom about AI risk, the topic was a niche blip—dung beetle mating habits had more academic ink. Fast forward to 2025, post his 2014 game-changer ‘Superintelligence: Paths, Dangers, Strategies’, and AI’s sprinting ahead. So, how close are…
