Live from Japan! This time we have Future Day spanning 2 days -Feb 28th / March 1st! Why are nearly all our holidays focused on celebrating the past, or the cyclical processes of nature? Why not celebrate the amazing future we are collectively creating? Don’t miss this special Future Day 2025 event—where the future isn’t…
Discussion between Anders Sandberg & Kristian Rönn on the AI Arms Race & the Darwinian Trap. Kristian Rönn is the CEO and co-founder of Normative, and author of the book ‘The Darwinian Trap’ (buy it here). He has a background in mathematics, philosophy, computer science and artificial intelligence. Anders Sandberg is a researcher at the…
… and the Long First Turn (of the new game). We’re not just designing AI to behave nicely in our human sandbox—but in case the sandbox turns out to be part of a much larger galactic metropolis, we’d better make sure the AI doesn’t start flinging galactic sand in anyone’s eye. We may at some…
No one in their right mind can deny the struggle to survive animates natural selection. Though it’s a bit misleading to say that ‘survival is paramount – and that every strategy serves that goal’: survival plays a crucial role in evolution, but it is not the ultimate goal. As emphasised by Charles Darwin’s theory of…
Value Space and “Good” Human Values Human values can be conceptualised as occupying regions in a vast objective multidimensional “value space.” These regions reflect preferences for cooperation, survival, flourishing, and minimising harm, among other positive traits. If TAI / SI can approximate the subset of human values deemed “good” (e.g., compassion, fairness, cooperation), while avoiding…
Note: this is by no means exhaustive. it’s focus is on western philosophy. I’ll expand and categorise more later… The earliest precursors to indirect normativity can be traced back to early philosophical discussions on how to ground moral decision-making in processes or frameworks rather than specific, static directives. While Nick Bostrom’s work on indirect normativity…
Implications for AI Safety and Indirect Normativity The challenge of understanding value space has moved from a philosophical curiosity to an urgent technical problem. As we approach the development of advanced artificial intelligence, the structure of value and our ability to specify it has become perhaps the most critical challenge facing humanity. This challenge assumes…
Kristian Rönn discusses his amazing book The Darwinian Trap – Macroeconomics & Game theoretic Imagineering: avoiding the minefields of the tragedy of the common in the hope that we can stave off the onslaught of Darwinian Demons, and ultimately find some nash equilibrium of fairness for all humanity, sentient AI an non-human animals. The book…
Will AI Ignore, Preserve or Improve on Our Values? Imagine a future where advanced AIs follow every instruction flawlessly – but humanity feels strangely adrift. Our tools are obedient, but the ‘soul’ of our civilisation feels absent. This is the hidden danger of V-risk – value erosion. In this post I explore what I explore…
Some people want to infuse AI with the codification of our existing moral judgements. CEV takes the idea a bit further. In short, CEV is a conceptual approach to aligning superintelligent AI by designing it to act according to an idealised version of what humans would want if they had more knowledge, were faster thinkers,…
