Is controlling AI is enough? AI safety has a steering problem, not just a brakes problem. We have to make it care about the right things. As AI systems become more capable, the foundational question isn’t merely how do we contain them? It’s: what are they optimising for, and why should we trust that objective?…
In AI safety, Control and Motivation are the two primary strategies used to prevent advanced AI systems from causing harm. While control focuses on external constraints, motivation addresses the internal goals and values of the AI. Most researchers, including those at Anthropic and the Future of Life Institute, suggest a hybrid approach: using strict capability…
The question is not, can AI be ethical, but can AI clear the bar humans have set? And once you see the bar – you realise we’ve been arguing about whether AI can limbo. “Align AI to human values” sounds sensible, even comforting. But once you look closely, it becomes far less clear that this…
What is M-Risk? M-Risk is the risk that a transformative AI may not be adequately motivated to pursue good values, even if it knows what they are or how to discover them. The AI might possess the cognitive capability to understand values (including their preference hierarchy to sentient agents) but lack the motivation to pursue…