Why Are We Afraid to Ask Whether AI Could Be More Moral Than Humans?
Is it dangerous to talk about AI becoming more moral than us? I think the idea of AI being more moral than humans is a real taboo in some circles. Many alignment researchers are uncomfortable with the idea because it seems to smuggle in the assumption that AI could have genuine moral agency, which conflicts…