Are Machines Capable of Morality? Join Professor Colin Allen!

The Ethics of AI: Why We Need Moral Machines, Not Just Smart Ones

In an era defined by rapid advancements in artificial intelligence, we often find ourselves wrestling with profound ethical questions. One of the most provocative of these is whether machines can, or should, be given a sense of morality. In this recent video interview distinguished Professor Colin Allen delves into this very topic, offering a nuanced and grounded perspective on a subject often dismissed as pure science fiction. This interview was a fantastic opportunity to delve into the concepts from his book and explore how the field of AI has evolved since its publication.

From Good Old-Fashioned AI to LLMs

Look back at the origins of AI, from its founding at the Dartmouth workshop in 1956 to the state of the field in the early 2000s – there has been a significant leap in AI capabilities driven by advancements in transformer networks, deep learning, and large language models (LLMs). Professor Allen contrasts today’s massive neural networks, which can still take weeks to train, with his first PhD project: a simple network with fewer than 100 nodes that took three weeks to train.

Allen highlighted a key paradigm shift: early AI, or “good old-fashioned AI,” (GOFAI) was logic-heavy and struggled with complexity. However, with the scaling of neural networks, LLMs have proven that associative learning can solve difficult problems in ways previously thought to require a more formal, structured approach.

Why We Need Artificial Moral Agents

A central theme of our discussion was the core question from his book: “Why do we need moral machines?”. Professor Allen clarified that while “AI ethics” is a broad term covering the social context of machines, his work focuses on the machines’ capacity to be artificial moral agents (AMAs) themselves.

He argued that machines need this capacity because they increasingly operate independently of immediate human oversight, such as with chatbots or self-driving cars. Allen referenced the tragic Uber self-driving car accident in Arizona, noting that while the human operator was distracted, the system’s design fostered a dangerous level of confidence, highlighting the need for AI systems to have built-in safeguards and ethical decision-making capabilities.

The Problem of Global Incoherence

Professor Allen brought up what he calls a key failing of modern AI: local coherence without global coherence. He gave an example of an AI-generated image of a cat that, while locally coherent with well-rendered fur, had a tail that morphed into a paw. He also mentioned the case of LaMDA, the Google chatbot, whose dialogue sometimes revealed globally incoherent statements.

This is a stark contrast to human intelligence, which is capable of maintaining long-range context over years and decades, allowing us to rapidly adapt and find novel solutions in new situations. Professor Allen noted that unlike humans who can be convinced of a new fact and have it immediately and permanently alter their beliefs, LLMs treat every piece of data in their training set as having equal value. This is why systems often require “human feedback reinforcement learning” to suppress objectively “bad” content that the core model cannot distinguish from good content.

Visualising Moral Capability in AI/Robots

The chart to the right categorises machines into three distinct groups:

• Operational Morality: This is the most basic level. An AI with operational morality, like a chess program, follows rules to achieve a goal. It’s essentially a system of “oughts” or “don’ts,” but with no understanding of why. The system is designed to perform a task without violating specific constraints, but it can’t evaluate the morality of those constraints.
• Functional Morality: At this level, an AI can make decisions that affect a moral outcome. A good example would be a self-driving car programmed to prioritize passenger safety over minor property damage. It can evaluate different outcomes and choose the “better” one, but it still lacks a deeper understanding of the ethical principles guiding its decision. It’s following a pre-programmed moral calculus, not an internal ethical framework.
• Full Moral Agency: This is the highest level, and it’s where AI becomes a true “moral machine.” A fully moral agent would not only be able to make ethical decisions, but it would also understand the reasons behind those decisions. It would be able to learn, adapt, and even develop its own moral principles, similar to a human. This is the ultimate goal, but as the interview with Professor Allen suggests, we are a long way from achieving it.

Where Are We Today?

Most of today’s AI and robots, or “(ro)bots,” fall into the first two categories. Large Language Models (LLMs) like ChatGPT, while incredibly sophisticated, primarily operate with functional morality at best. They can be programmed to reject harmful or biased outputs, but they do so based on human feedback and pre-programmed rules. They don’t have a personal sense of right and wrong, nor can they justify their decisions from a first-principles ethical standpoint.

Professor Allen’s point about “local coherence without global coherence” discussed above perfectly illustrates this. An LLM might produce a locally sensible response, but it lacks the long-range, integrated understanding needed for true moral agency.

Current self-driving cars and other autonomous systems also fall into the category of functional morality. They are programmed to make decisions within a predefined ethical framework (e.g., minimising harm, following traffic laws). However, they cannot, as Professor Allen puts it, “evaluate and justify moral judgments with reference to ethical principles.”

In short, while we have made great strides in creating highly functional AI, we are still far from building a true “moral machine.” The path forward, as Professor Allen’s work suggests, requires not just more powerful algorithms, but a deeper understanding of what it means to be a moral agent in the first place.

The Road to Artificial General Intelligence (AGI)

Our conversation ended on a forward-looking note, discussing whether we are close to achieving AGI, or human-level intelligence across all domains. Professor Allen expressed skepticism, emphasising that while AI can excel at specific tasks like winning chess or a math olympiad¹, it still lacks the general-purpose adaptability and embodied nature of humans. He pointed out the old adage in AI and robotics circles: the easy things (like playing chess) are hard, and the hard things (like cleaning a house) are easy. Our discussion underscored that while AI has made incredible strides, the journey toward building truly moral and generally intelligent machines is still very much ongoing.

Prolegomena to any future artificial moral agent

Colin Allen also co-authored (Gary Varner & Jason Zinser) the seminal paper ‘Prolegomena to any future artificial moral agent‘ which came out in the year 2000. It argues that as AI becomes more autonomous, the need to develop artificial moral agents (AMAs) is becoming urgent. They highlight the risk that autonomous robots, while useful, could also cause harm. The authors identify two primary challenges: philosophical disputes about moral theory itself and computational limits in implementing these theories. They propose a “moral Turing Test” to evaluate machine morality and assess the computational difficulties of different approaches. Replicating flawed human-like moral performance in machines might not be acceptable, also perfect moral behaviour may be computationally unattainable. They stress that the most critical task for AI designers is to create machines intelligent enough to understand and consider the effects of their actions on sentient beings.

Bio

Colin Allen is a distinguished professor of philosophy at the University of California, Santa Barbara, co-author of the seminal book Moral Machines: Teaching Robots Right from Wrong. He is widely recognised for his work at the intersection of philosophy, cognitive science, and AI.

1. An advanced version of Gemini Deep Think won the gold medal at IMO. Google’s artificial intelligence (AI) research arm DeepMind has won a gold medal at the International Mathematical Olympiad (IMO), the world’s most prestigious competition for young mathematicians. See here, and here for more detail.
   Next stop, the International Moral Olympiad – so, are we all agreed on the scoring system yet? ↩︎