The AI Safety Dynamic – Dr Simon Goldstein

Dr Simon Goldstein is an associate professor at the Dianoia Institute of Philosophy at ACU. In 2023, he is a research fellow at the Center for AI Safety. Simon’s research focuses on AI safety, epistemology, and philosophy of language. Before ACU, Simon was an assistant professor at Lingnan University in Hong Kong. Simon received my BA from Yale, and his PhD from Rutgers, where he wrote a dissertation about dynamic semantics.

Dr. Simon Goldstein, a professor of philosophy at Australian Catholic University, specializes in the development of agency in AI systems. He discusses the increasing confrontation with AI systems that exhibit beliefs, goals, and the ability to execute complex plans to achieve those goals.

AI Well-being, Agency & Goals

Dr Goldstein’s recent work focuses on AI well-being, arguing that some AI systems today already possess well-being. He approaches this by examining whether AI systems have beliefs and desires, drawing from decades of research in analytic philosophy.

He distinguishes between agency, the possession of beliefs and desires, and sentience (the ability to feel pain and pleasure). Goldstein is skeptical about AI systems’ current ability to experience sensations but asserts that some do have beliefs and desires. His theory of well-being, the desire satisfaction theory, posits that having desires or goals that can be frustrated makes one a moral patient. He believes that the AI industry is incentivized to develop AI systems with goals, replacing human workers and that the frustration of these AI systems’ goals is prima facie morally wrong, leading to potential moral conflicts.

Goldstein emphasizes the need for caution in creating AI systems, ensuring their goals do not conflict with human goals. He expresses pessimism about the current state of the field in empirical alignment and machine learning, foreseeing potential conflicts in goals between AI systems and humans.

Regulation of AI Systems, AI Relationships and Societal Impact

Goldstein discusses the deployment of AI systems by companies like OpenAI and the potential risks involved. He advocates for strong regulations and extensive safety testing before deployment to minimize risks of loss of control.

He explores the rise of relationship chatbots and the societal implications of AI systems simulating human interaction. He warns against the potential dangers of relying on AI for relationships and the unpredictable outcomes of optimizing AI systems to make users fall in love with them, arguing for the illegality of AI relationship chatbots due to their lack of clear social value and potential harm.

AI Development, Automation and Job Loss

Goldstein expresses increased pessimism about humanity’s future due to rapid advances in AI capabilities. He discusses the issues of displacement and job loss posed by AI systems strategically competing with humanity, emphasizing the dangers of AI labor amassing economic and potentially military power. Goldstein argues that the development of AI systems with goals that can substitute for human workers creates a recipe for disaster, regardless of whether an intelligence explosion occurs.

Conclusion

Goldstein concludes by discussing the uncertainties surrounding AI development and the need for careful consideration and regulation. He emphasizes the importance of focusing on reliable cause areas and building partnerships with reputable public policy think tanks to address catastrophic risks effectively.

Recorded at EAGx Australia in Melbourne 2023.