Firstly, values are important because they are the fundamental principles that guide decisions and actions in humans. They shape our understanding of right and wrong, and they influence how we interact with the world around us. In the context of AI, values are particularly important because they determine how AI systems will behave, what goals…
About We already can make deepfakes of ourselves and others. There is an issue of consent; ideally people should consent to have digital twins of themselves made, even if its your grandma. But if we start making digital twins of ourselves, which is already being done, they can go out and do things on our…
Live from Japan! This time we have Future Day spanning 2 days -Feb 28th / March 1st! Why are nearly all our holidays focused on celebrating the past, or the cyclical processes of nature? Why not celebrate the amazing future we are collectively creating? Don’t miss this special Future Day 2025 event—where the future isn’t…
Discussion between Anders Sandberg & Kristian Rönn on the AI Arms Race & the Darwinian Trap. Kristian Rönn is the CEO and co-founder of Normative, and author of the book ‘The Darwinian Trap’. He has a background in mathematics, philosophy, computer science and artificial intelligence. Anders Sandberg is a researcher at the Mimir Center for…
… and the Long First Turn (of the new game). We’re not just designing AI to behave nicely in our human sandbox—but in case the sandbox turns out to be part of a much larger galactic metropolis, we’d better make sure the AI doesn’t start flinging galactic sand in anyone’s eye. We may at some…
No one in their right mind can deny the struggle to survive animates natural selection. Though it’s a bit misleading to say that ‘survival is paramount – and that every strategy serves that goal’: survival plays a crucial role in evolution, but it is not the ultimate goal. As emphasised by Charles Darwin’s theory of…
Value Space and “Good” Human Values Human values can be conceptualised as occupying regions in a vast objective multidimensional “value space.” These regions reflect preferences for cooperation, survival, flourishing, and minimising harm, among other positive traits. If TAI / SI can approximate the subset of human values deemed “good” (e.g., compassion, fairness, cooperation), while avoiding…
Note: this is by no means exhaustive. it’s focus is on western philosophy. I’ll expand and categorise more later… The earliest precursors to indirect normativity can be traced back to early philosophical discussions on how to ground moral decision-making in processes or frameworks rather than specific, static directives. While Nick Bostrom’s work on indirect normativity…
Implications for AI Safety and Indirect Normativity The challenge of understanding value space has moved from a philosophical curiosity to an urgent technical problem. As we approach the development of advanced artificial intelligence, the structure of value and our ability to specify it has become perhaps the most critical challenge facing humanity. This challenge assumes…
Kristian Rönn discusses his amazing book The Darwinian Trap – Macroeconomics & Game theoretic Imagineering: avoiding the minefields of the tragedy of the common in the hope that we can stave off the onslaught of Darwinian Demons, and ultimately find some nash equilibrium of fairness for all humanity, sentient AI an non-human animals. The book…
