The question is not, can AI be ethical, but can AI clear the bar humans have set? And once you see the bar – you realise we’ve been arguing about whether AI can limbo. “Align AI to human values” sounds sensible, even comforting. But once you look closely, it becomes far less clear that this…
What is M-Risk? M-Risk is the risk that a transformative AI may not be adequately motivated to pursue good values, even if it knows what they are or how to discover them. The AI might possess the cognitive capability to understand values (including their preference hierarchy to sentient agents) but lack the motivation to pursue…
This is an attempt to formally introduce the concept of value risk (v-risk or VRisk). This is not an attempt at repackaging well known established risk categories like x-risk and s-risk, it’s an attempt to carve out axis of concerns about value choice, hopefully to help increase clarity and weight to what might otherwise sound…
Make the AIs really smart, and ask them what to do. Alignment Challenges It’s hardly a week goes by and there has been some new breakthrough in AI, it’s vaulting in capability to new heights all the time – it’s increasingly hard to deny how critical a role ethics has to play in shaping AI…
In AI safety, direct specification refers to the attempt to program specific goals, rules, or behaviours directly into an AI system. AI trained to optimise values may over-optimise those we specify at the expense of real value we don’t specify. Political or market dynamics incentivise racing to powerful AI – the race dynamics come with…
A discussion between David Pearce and James Hughes moderated by Adam Ford exploring the ethical and philosophical landscapes of AI, human enhancement and the future of emerging technologies affording higher states of well-being. Pearce and Hughes discuss the implications of transforming human experience via leveraging biotech and cybernetics, as well as requirements for AI to…
Can philosophical zombies be philosophical? Preamble There are various takes on what a p-zombie is, most of which were attempts to dethrone physicalism – this article isn’t one of those. This kind of p-zombie is different from what David Chalmers describes in The Consciousness Mind. In Chalmers view, p-zombies are physiologically identical to humans, even…
J. Dmitri Gallow discusses the principles of instrumental convergence and divergence in AI. The orthogonality thesis, which states intelligence and desire are independent, and the instrumental convergence thesis, which suggests intelligent beings will have similar instrumental desires, are critical concepts. Gallow’s argument focuses on the instrumental divergence, which emerges from the complexity and unpredictability of AI’s actions based on its desires.
This interview was conducted in 2012 in San Francisco. In the future may will see it is not ethically responsible to play genetic roulette and instead take the decision to have happy, healthy, pro-social offspring. 0:00 Introduction0:36 Alleviating Suffering 7:00 Justified Suffering? 13:12 Buddhism 14:42 The World Transhumanist Association 22:35 Recalibration of Society or Biology?…
Hard realist alignment: align advanced AI to stance-independent moral facts rather than human preference aggregates. Preference-realism hybrid: align to human preferences, regularised by stance-independent moral facts.Mechanism: indirect normativity with explicit procedures for discovery, uncertainty handling, and safety interlocks. The principles behind the development of powerful AI carry exceptionally high stakes. This post considers AI alignment,…