In AI safety, Control and Motivation are the two primary strategies used to prevent advanced AI systems from causing harm. While control focuses on external constraints, motivation addresses the internal goals and values of the AI. Most researchers, including those at Anthropic and the Future of Life Institute, suggest a hybrid approach: using strict capability…
Concerning sentient AI, we would like to avoid unnecessary suffering in artificial systems. It’s hard for biological systems like humans to turn off suffering without appropriate pharmacology i.e. from aspirin to anesthetics etc. AI may be able to self administer pain killers – a kind of wireheading in reverse. Similarly to wireheading in AI systems…
There is so much wrong with the world right now – I don’t know where to begin. I wake up early with dread sometimes – it’s hard to know what thread to tug at – the amorphous confusion festooning the great big everything seems so hopeless. It would be a lot easier if everyone were…
Turning off all AI won’t happen in the real world. If we understand the signatures of sentience, we’ll be in a better position to know what to do to circumstantially prevent/mitigate it or encourage it. AI, esp LLMs ‘claiming sentience’ isn’t enough… We need to deep operational understandings of it. See the article by 80k…
“Align AI to human values” sounds sensible, even comforting. But once you look closely, it becomes far less clear that this is the right target. What are “higher values”? I don’t mean values I personally approve of, but with nicer branding – I mean better-justified values discovered through improved moral and epistemic reflection. What are…
What if superintelligent AI didn’t care about humans? Does intelligence necessitate care? Probably not.While I find it hard to imagine care in the absence of cognitive capacity, I can’t see anything that guarantees all cognitively apt agents will harbour care. How, if at all possible, can we from our current position assess the likelihood that…
What is M-Risk? M-Risk is the risk that a transformative AI may not be adequately motivated to pursue good values, even if it knows what they are or how to discover them. The AI might possess the cognitive capability to understand values (including their preference hierarchy to sentient agents) but lack the motivation to pursue…
This is an attempt to formally introduce the concept of value risk (v-risk or VRisk). This is not an attempt at repackaging well known established risk categories like x-risk and s-risk, it’s an attempt to carve out axis of concerns about value choice, hopefully to help increase clarity and weight to what might otherwise sound…
Make the AIs really smart, and ask them what to do. Alignment Challenges It’s hardly a week goes by and there has been some new breakthrough in AI, it’s vaulting in capability to new heights all the time – it’s increasingly hard to deny how critical a role ethics has to play in shaping AI…
In AI safety, direct specification refers to the attempt to program specific goals, rules, or behaviours directly into an AI system. AI trained to optimise values may over-optimise those we specify at the expense of real value we don’t specify. Political or market dynamics incentivise racing to powerful AI – the race dynamics come with…