Most arguments about AI alignment begin with the question of which values to give a machine, skipping the prior question of whether there is anything for values to be right or wrong about. I think there is, and if so the alignment problem changes shape. It changes from the form where alignment means transmitting our…
Is self-interest by necessity the default motivation for all agents?Will Superintelligence necessarily become a narcissistic utility monster? No, self-interest is not necessarily the default motivation of all agents. While self-interest is a common and often foundational drive in many agents (biological or artificial), other motivations can and do arise, either naturally or through design, depending…
In AI safety, Control and Motivation are the two primary strategies used to prevent advanced AI systems from causing harm. While control focuses on external constraints, motivation addresses the internal goals and values of the AI. Most researchers, including those at Anthropic and the Future of Life Institute, suggest a hybrid approach: using strict capability…
In this interview Jeff Sebo discusses the ethical implications of artificial intelligence and why we must take the possibility of AI sentience seriously now. He explores challenges in measuring moral significance, the risks of dismissing AI as mere tools, and strategies to mitigate suffering in artificial systems. Drawing on themes from the paper ‘Taking AI…
Part 1 in an interview with Eric Sampson on Moral Realism. Note AI is referenced, but the bulk of the questions about AI are covered in part 2. Chapters 0:00 Intro 3:03 Why is moral realism attractive to AI Safety Research? 2:02 Background, & Personal reason to choose moral realism 4:00 Describing moral realism –…
Concerning sentient AI, we would like to avoid unnecessary suffering in artificial systems. It’s hard for biological systems like humans to turn off suffering without appropriate pharmacology i.e. from aspirin to anesthetics etc. AI may be able to self administer pain killers – a kind of wireheading in reverse. Similarly to wireheading in AI systems…
In this short interview, Peter Singer, a renowned philosopher and ethicist widely recognized for his thought-provoking ideas about universal ethics, discusses the value of life, moral progress, population ethics (aka population axiology), the far future, the uncertainties inherent in philosophical reasoning, moral realism (objective normative truths) and ‘alternative facts’. Points covered:0:00 Intro 0:08 Moral progress…
There is so much wrong with the world right now – I don’t know where to begin. I wake up early with dread sometimes – it’s hard to know what thread to tug at – the amorphous confusion festooning the great big everything seems so hopeless. It would be a lot easier if everyone were…
“Thoughts on modernity’s monoculture mistake” – Here Robin Hanson argues that “cultural drift” poses a significant threat to societies, especially in the context of rapid technological and social change. Cultural drift refers to the gradual shift in societal norms and values over time, often driven by small, incremental changes that go unnoticed until they lead…
Turning off all AI won’t happen in the real world. If we understand the signatures of sentience, we’ll be in a better position to know what to do to circumstantially prevent/mitigate it or encourage it. AI, esp LLMs ‘claiming sentience’ isn’t enough… We need to deep operational understandings of it. See the article by 80k…