Why moral competence and motivation selection deserve a place in the AI safety portfolio From the early days of thinking about artificial intelligence, people wondered what would happen if we eventually built machines much more intelligent than ourselves. Science fiction explored it long before we had anything resembling modern AI. Researchers later began asking how…
Terence Cuneo is Marsh Professor of Intellectual and Moral Philosophy at the University of Vermont. He is best known for The Normative Web (Oxford, 2007), which argues for moral realism by an indirect route, and more recently for The Moral Universe (Oxford, 2024), written with John Bengson and Russ Shafer-Landau. The three are now at…
What if the point of self-improvement were not just to feel better or get ahead, but to become more capable of helping in a hurting world? I recently had the pleasure of welcoming Magnus Vinding back to the channel. We last spoke a few years ago about his book Suffering-Focused Ethics, and this time we…
Human values will not bind a superintelligence merely because they are ours. Provenance is not a reason. If the singularity is near, ASI is near. A system that outthinks us across every domain that matters probably won’t be controllable in the engineering sense, or governable in the institutional one – both of those are capability…
Fair warning, this article is pure uncut Pollyanna. Credible figures in AI like Geoffrey Hinton, Yoshua Bengio, Dario Amodei and Demis Hassabis take seriously the possibility of human-level or superhuman AI arriving within years or decades rather than centuries. If AI can substantially automate AI research itself, the interval between AGI and much more capable…
Thoughts about Wei Dai’s “A Conflict Between AI Alignment and Philosophical Competence” (AI Alignment Forum, 27 December 2025). Someone recently pointed me to Wei Dai’s post arguing that there is a conflict between making an AI aligned and making it philosophically competent. The argument runs roughly like this. Metaethics is unsolved, so the meta-correct position…
Anthropic’s recent interpretability work reports something some people may find odd sitting inside Claude: a small, densely connected subspace of the model’s activations that behaves like a global workspace. It holds a few dozen concept-linked slots at a time, it is reportable (the model can say what is in it), and the model demonstrably depends…
It is quite remarkable that something resembling a global workspace emerged inside Claude during training, rather than being explicitly programmed into its architecture. Shared-workspace designs themselves are not new. Software engineers have used blackboard architectures, shared memory, message buses and coordinator-orchestrator patterns for decades – and nobody regarded those systems as conscious. What is remarkable…
More than two millennia ago, Plato’s Euthyphro exposed a crack in the foundations of divine-command morality. Generalise its argument from piety to goodness: if an act is good merely because the gods command it, morality is an arbitrary exercise of will. Had the gods commanded cruelty, cruelty would count as virtue. But if the gods…
Orthogonality defeats automatic moral convergence – but it does not, by itself, establish alien-value likelihood. The serious work lies in the bridge premises. The orthogonality thesis shows that intelligence alone gives us no guarantee of moral or human-value convergence. Claims that advanced AI values are likely to be alien require further arguments about training, selection,…