Motivation Selection

AI doesn’t need a moustache-twirling villain to go wrong – it just needs the wrong metaethics in an unforgiving game. In this post I argue that: (1) many human moral norms are shaped by competitive, “Molochian” pressures; (2) if we give AI a purely constructivist / anti-realist metaethic, it will inherit and amplify those pressures;…