Brian Tomasik on Ethics, Suffering, the Future and What to Do About It
We discussed Brian’s background, why he is very motivated to reduce suffering, various (sometimes un-intuitive) forms of suffering, utilitarianism (negative, hedonistic, and novelty [inc whether to intrinsically value novelty]), fun theory, axiological quests that we and/or advanced AI’s might make to learn more about what to value in the future, physics of suffering, emergentism (strong and weak), utilitronium shockwaves etc, suffering in computation and exp reinforcement learning, mind-crime, the singularity/intelligence explosion, ethical AI, non-human-value biased ethics, moral realism, moral relativism etc.
Background & The Foundational Research Institute: Brian Tomasik is a former Microsoft employee and co-founder of the Foundational Research Institute (FRI) in Switzerland. Tomasik explains that FRI’s primary mission is to prioritise the most effective ways to reduce suffering in the far future. Unlike other organisations that predominantly focus on existential risks (like human extinction) or preserving fun, FRI explicitly focuses on artificial intelligence trajectories, suffering-focused ethical views, and the risks of astronomical suffering.
Origins of His Suffering-Focused Ethics: Tomasik traces his philosophical journey back to the year 2000, inspired initially by political activism and a speech by Ralph Nader. His worldview drastically shifted upon discovering Peter Singer’s utilitarianism and the immense scale of animal suffering. Ultimately, influenced by David Pearce’s Hedonistic Imperative, he adopted Negative Utilitarianism, concluding that preventing extreme suffering carries a heavier moral weight than generating extreme pleasure.
AI Trajectories and the Far Future: A significant portion of the interview delves into how future artificial intelligence might function and its moral implications. Tomasik stresses the importance of steering AI trajectories in a direction that minimises suffering – whether it ends up as a centralised agent, a distributed economic process, or a paperclip maximiser. He emphasises the astronomical stakes of the far future, where minor present-day trajectory changes could impact trillions of digital or biological minds.
Moral Philosophy and Modesty: Tomasik navigates complex thought experiments – such as The Ones Who Walk Away from Omelas – to discuss moral thresholds and the repugnant conclusion. While he leans heavily toward utilitarianism regarding intrinsic value, he acknowledges the instrumental usefulness of rule-based systems (deontology) in human societies. Finally, he advocates for intellectual modesty, noting that anthropic arguments (like the Doomsday argument or the simulation hypothesis) suggest the far future may not be as expansive as we assume, warning against letting far-future speculations entirely dominate ethical decision-making without challenge.
Timestamps & Descriptions
0:00 – Introduction to Brian Tomasik: Background in computer science, statistics, his time at Microsoft, and his focus on philosophy and reducing suffering.
1:09 The Foundational Research Institute (FRI): Tomasik explains the core mission of FRI, focusing on AI trajectories, wild animal suffering, and suffering-focused ethical views.
4:42 AI Risks and Suffering: Differentiating FRI’s approach from other existential-risk organisations by focusing on the risks of suffering caused by AI rather than just human extinction.
5:50 Tomasik’s Personal Motivation: How a speech by Ralph Nader in 2000 sparked his interest in altruism, leading to discovering Peter Singer’s writings and utilitarianism.
7:13 The Hedonistic Imperative: Tomasik discusses how David Pearce’s writings helped refine his view that preventing extreme suffering is more morally important than promoting intense pleasure.
9:19 Negative Utilitarianism: Exploring the practical conclusions of Negative Utilitarianism and the concept of weighing pain against pleasure.
14:02 Moral Thresholds & Thought Experiments: Using Ursula K. Le Guin’s short story The Ones Who Walk Away from Omelas to discuss whether immense suffering for one entity can be justified by the existence or happiness of the rest of the world.
21:57 Deontology vs. Utilitarianism: Tomasik expresses that while he is a utilitarian regarding intrinsic value, he sees the instrumental value of deontology (rule-based ethics) in practical decision-theory and human cooperation.
23:54 Trajectories over Pure Ethics: Arguing that figuring out the correct long-term trajectory of AI is currently more important than settling abstract ethical debates.
2:12:57 The Scale of the Far Future: Discussing how tiny shifts in today’s trajectory could influence the wellbeing of trillions of minds in the distant future.
2:13:19 Anthropic Arguments & Modesty: Why we should remain intellectually modest about the far future, citing Copernican mediocrity, the Doomsday argument, and the simulation hypothesis.
2:15:22 Future AI Scenarios: Exploring different structural outcomes for AI (e.g., human-controlled vs. a paperclipping AI) and analysing which scenarios result in more or less suffering.
