Will AI Ignore, Preserve or Improve on Our Values? Imagine a future where advanced AIs follow every instruction flawlessly – but humanity feels strangely adrift. Our tools are obedient, but the ‘soul’ of our civilisation feels absent. This is the hidden danger of V-risk – value erosion. In this post I explore what I broadly…
Some people want to infuse AI with the codification of our existing moral judgements. CEV takes the idea a bit further. In short, CEV is a conceptual approach to aligning superintelligent AI by designing it to act according to an idealised version of what humans would want if they had more knowledge, were faster thinkers,…
The concept of a non-arbitrary objective value space offers a compelling framework for understanding how to navigate to more valuable worlds. This framework presupposes the existence of stance-independent normative truths – metaphysical/ontological, epistemic, scientific & ethical principles that rational agents will converge upon given sufficient cognitive sophistication [1]. Here I will mostly talk in relation…
Is self-interest by necessity the default motivation for all agents?Will Superintelligence necessarily become a narcissistic utility monster? No, self-interest is not necessarily the default motivation of all agents. While self-interest is a common and often foundational drive in many agents (biological or artificial), other motivations can and do arise, either naturally or through design, depending…
In AI safety, Control and Motivation are the two primary strategies used to prevent advanced AI systems from causing harm. While control focuses on external constraints, motivation addresses the internal goals and values of the AI. Most researchers, including those at Anthropic and the Future of Life Institute, suggest a hybrid approach: using strict capability…
Concerning sentient AI, we would like to avoid unnecessary suffering in artificial systems. It’s hard for biological systems like humans to turn off suffering without appropriate pharmacology i.e. from aspirin to anesthetics etc. AI may be able to self administer pain killers – a kind of wireheading in reverse. Similarly to wireheading in AI systems…
There is so much wrong with the world right now – I don’t know where to begin. I wake up early with dread sometimes – it’s hard to know what thread to tug at – the amorphous confusion festooning the great big everything seems so hopeless. It would be a lot easier if everyone were…
Turning off all AI won’t happen in the real world. If we understand the signatures of sentience, we’ll be in a better position to know what to do to circumstantially prevent/mitigate it or encourage it. AI, esp LLMs ‘claiming sentience’ isn’t enough… We need to deep operational understandings of it. See the article by 80k…
The question is not, can AI be ethical, but can AI clear the bar humans have set? And once you see the bar – you realise we’ve been arguing about whether AI can limbo. “Align AI to human values” sounds sensible, even comforting. But once you look closely, it becomes far less clear that this…
What if superintelligent AI didn’t care about humans? Does intelligence necessitate care? Probably not.While I find it hard to imagine care in the absence of cognitive capacity, I can’t see anything that guarantees all cognitively apt agents will harbour care. How, if at all possible, can we from our current position assess the likelihood that…