Is self-interest by necessity the default motivation for all agents?Will Superintelligence necessarily become a narcissistic utility monster? No, self-interest is not necessarily the default motivation of all agents. While self-interest is a common and often foundational drive in many agents (biological or artificial), other motivations can and do arise, either naturally or through design, depending…
Concerning sentient AI, we would like to avoid unnecessary suffering in artificial systems. It’s hard for biological systems like humans to turn off suffering without appropriate pharmacology i.e. from aspirin to anesthetics etc. AI may be able to self administer pain killers – a kind of wireheading in reverse. Similarly to wireheading in AI systems…
There is so much wrong with the world right now – I don’t know where to begin. I wake up early with dread sometimes – it’s hard to know what thread to tug at – the amorphous confusion festooning the great big everything seems so hopeless. It would be a lot easier if everyone were…
Turning off all AI won’t happen in the real world. If we understand the signatures of sentience, we’ll be in a better position to know what to do to circumstantially prevent/mitigate it or encourage it. AI, esp LLMs ‘claiming sentience’ isn’t enough… We need to deep operational understandings of it. See the article by 80k…
The meaning of homo sapien means ‘wise man’ – given the current human-caused precarious states of affairs, this self-description seems a bit of a reach. Human values in aggregate aren’t coherent, and of those that are, not all are really that great (especially if our revealed preferences hint at what they are). Connor Leahy, who…
What is M-Risk? M-Risk is the risk that a transformative AI may not be adequately motivated to pursue good values, even if it knows what they are or how to discover them. The AI might possess the cognitive capability to understand values (including their preference hierarchy to sentient agents) but lack the motivation to pursue…
Value Risk (vrisk): The risk of bad to sub-optimal values happening. An obviously bad vrisk is one where existential risk (xrisk) is high. Existential here could mean extinction of life/species/sentience etc, or the universe becoming uninhabitable to the degree where there is no chance of new life emerging to experience anything. Extinction means local annihilation…
Alignment Challenges Given the critical role of ethics in AI safety, it’s deeply concerning to see such significant disagreement among experts who have rigorously studied ethics. The divergence in moral and meta-ethical perspectives among these experts poses a serious question: How can we effectively align AI if the very foundations of ethical understanding are not…
A discussion between David Pearce and James Hughes moderated by Adam Ford exploring the ethical and philosophical landscapes of AI, human enhancement and the future of emerging technologies affording higher states of well-being. Pearce and Hughes discuss the implications of transforming human experience via leveraging biotech and cybernetics, as well as requirements for AI to…
Can philosophical zombies be philosophical? Preamble There are various takes on what a p-zombie is, most of which were attempts to dethrone physicalism – this article isn’t one of those. This kind of p-zombie is different from what David Chalmers describes in The Consciousness Mind. In Chalmers view, p-zombies are physiologically identical to humans, even…
