Value Space
The concept of a non-arbitrary objective value space offers a compelling framework for understanding the moral landscape. This framework presupposes the existence of stance-independent moral truths—ethical principles that rational agents will converge upon given sufficient cognitive sophistication [1].
Human values are a narrow slice of value space – In the vast space of all conceivable values, human values occupy a relatively narrow region. Historically, the primary shifts within this space have revolved around expanding the boundaries of moral consideration [2] – extending moral concern from kin to broader social groups, and eventually to all sentient beings. Despite this expansion, the underlying structure of human values remains heavily influenced by evolutionary and cultural factors, limiting their reach to specific areas of the value landscape.
Human values, while approximately reflecting some universal moral truths (e.g., the aversion to suffering), are highly contingent. They are shaped by specific environmental and cognitive constraints, suggesting that other intelligent agents might converge on different, yet overlapping, regions of value space under alternate circumstances [3]. An ideal observer could objectively assess the moral worth of these regions, independent of human bias, through their alignment with stance-independent moral principles. Unfortunately we don’t have access to an ideal observer, so have to resort to the best methods of rationality we have access to at the time and perhaps indirect normativity with the aid of oracle AI.
“All reality is a game. Physics at its most fundamental, the very fabric of our universe, results directly from the interaction of certain fairly simple rules, and chance; the same description may be applied to the best, most elegant and both intellectually and aesthetically satisfying games. By being unknowable, by resulting from events which, at the sub-atomic level, cannot be fully predicted, the future remains malleable, and retains the possibility of change, the hope of coming to prevail; victory, to use an unfashionable word. In this, the future is a game; time is one of the rules.
Iain M. Banks – The Player of Games
Differentiating Beliefs, Preferences, and Values
A clear distinction between beliefs, preferences, and values is essential for mapping the moral terrain.
- Beliefs refer to descriptive claims about the world (e.g., “Exercise promotes health”), which can be true or false.
- Preferences denote subjective inclinations or desires (e.g., “I prefer chocolate to vanilla”), which may or may not align with moral truths.
- Values, however, reflect deeper normative commitments, often tied to notions of what ought to be (e.g., “Compassion is morally good”).
While preferences are often fluid and context-dependent, values tend to be more stable and reflective of underlying moral principles – these principles I argue should reflect stance independent moral truths.
In the space of all possible values, values occupy positions informed by rational deliberation and evidence i.e. compassion as a value might emerge from its capacity to reduce suffering (suffering is a morally relevant feature of the real world that transcends individual preferences).
Hierarchical Structure of Value Space
Within value space, certain regions can be considered “higher” or “lower” based on their moral quality. Higher regions align more closely with objective moral truths, such as minimizing suffering and promoting well-being, while lower regions reflect morally deficient configurations, such as unchecked cruelty or indifference. The distinction is not arbitrary but derives from the objective features of states these regions represent—features such as the prevalence of flourishing or the mitigation of harm.
Importantly, some values naturally approximate higher regions. For example, valuing empathy or fairness tends to lead agents closer to morally optimal configurations, as these values promote cooperation and reduce conflict. Conversely, values like greed or domination approximate lower regions, undermining collective well-being.
Practical Accessibility and Optimization Power
Not all regions of value space are equally accessible. The ability to “reach” certain regions depends on environmental circumstances and the optimization power of the agent. Optimisation power, in this context, refers to the agent’s cognitive sophistication which informs it’s powers of moral reasoning, and capacity for intentional action – i.e. achieving a civilization with near-complete elimination of suffering would likely require both advanced technological solutions and a highly refined understanding of moral truths.
Similarly, environmental constraints shape the practical range of accessible values. A society grappling with scarcity may prioritize survival-oriented values, whereas a existentially secure post-scarcity society might explore values related to aesthetic appreciation, non-instrumental meaning generation or intellectual flourishing [4]. These factors suggest that the moral progression of intelligent agents is contingent on both their intellectual maturity and the circumstances in which they operate.
Therefore
The framework of non-arbitrary objective value space illuminates the structured and hierarchical nature of morality. The better angels of human nature, and “good” human values, though limited, offer a glimpse into higher moral regions while also reflecting the constraints of our evolutionary and cultural history. By fostering greater cognitive sophistication and creating conducive environmental conditions, humanity may approximate regions of value space that more fully align with objective moral truths, paving the way for transformative ethical progress.
Machina Ex Machina
The Player of Games is a brilliant novel in the Culture series by Iain M. Banks. The title of one of the final chapters is ‘Machina Ex Machina’ (lit. “machine from the machine”) is a very clever play on the classical literary device “deus ex machina” (god from the machine). In classical theater, a deus ex machina was literally a machine that would lower a thespian playing a god onto the stage to resolve seemingly unsolvable plot situations.
The game itself is a machine-like system, and from within its own mechanical structure comes the resolution. The game of Azad, which the Empire created as a machine to perpetuate their social order, becomes the very mechanism of their undoing.
In this chapter the final game of Azad between Jernau Gurgeh and Emperor Nicosar reaches its climax. The game’s outcome proves that the Empire’s competitive, hierarchical philosophy is inferior to the Culture’s cooperative approach. This revelation triggers significant social and political upheaval within the Empire of Azad, effectively achieving the Culture’s goal of demonstrating the system’s inherent flaws through its own premier institution. At its core, the chapter presents a clash at the fault lines between two fundamentally opposed value systems – ways of understanding reality and organizing society.
The Empire of Azad embraces a philosophy of social Darwinism – the idea that competition, hierarchy, and the survival of the fittest are natural laws that should govern society. This manifests in their cruel social order and even in the game of Azad itself, which they use to determine social position and power.
The Culture, represented by Gurgeh, embodies a radically different philosophy based on cooperation, post-scarcity economics, and the belief that societies can transcend primitive competitive impulses. What makes this philosophical battle so fascinating is that it plays out through the very mechanism (the game) that the Empire created to justify its worldview.
There’s a deep irony in how the Culture achieves its victory. Rather than imposing its values through force (which would contradict its own philosophy), it uses the Empire’s own system to demonstrate the superiority of cooperative approaches.
I saw an interesting parallel from this to darwinistic natural selection – in evolutionary terms, what appears on the surface to be purely driven by competition and survival actually gives rise to cooperation as an emergent property. We see this in examples like reciprocal altruism, where seemingly “selfish” genes produce behaviors that favor mutual aid. Social insects, for instance bees, demonstrate how competitive pressures can lead to highly cooperative societies. Even human empathy and moral reasoning may have emerged from the evolutionary advantages of social cooperation (as Jaan Tallinn discusses in his talk ‘The Intelligence Stairway‘).
Were they to argue metaphysics, here, now, with the imperfect tool of language, when they’d spent the last ten days devising the most perfect image of their competing philosophies they were capable of expressing, probably in any form?
Iain M. Banks – The Player of Games [5]
What, anyway, was he to say? That intelligence could surpass and excel the blind force of evolution, with its emphasis on mutation, struggle and death? That conscious cooperation was more efficient than feral competition? That Azad could be so much more than a mere battle, if it was used to articulate, to communicate, to define…?
I wish Iian M. Banks were alive and healthy today.
Notes:
[1] Stance independent truth: These truths are grounded in non-moral features such as states of pain and pleasure – they are independent of mere subjective preferences or culturally contingent norms. Instead, they form an intricate, structured space of possible values, akin to a multidimensional topology, wherein certain regions represent morally superior or inferior configurations. [2] see the well known framing of moral progress as the expansion of the moral circle over time [3] Iain M. Banks in his Culture series explores the notion that the values which agents converge upon is contingent on environmental factors. [4] Nick Bostrom explores post-instrumental values in his book Deep Utopia [5] More extensive transcript of conversation between Morat Gurgeh & Emperor Nicosar“You disgust me, Morat Gurgeh,” Nicosar said to the red glow in the west. “Your blind, insipid morality can’t even account for your own success here, and you treat this battle-game like some filthy dance. It is there to be fought and struggled against, and you’ve attempted to seduce it. You’ve perverted it; replaced our holy witnessing with your own foul pornography… you’ve soiled it… male.”
Gurgeh dabbed at the blood on his lips. He felt dizzy, head swimming. “That… that may be how you see it, Nicosar.” He swallowed some of the thick, salty blood. “I don’t think you’re being entirely fair to—”
“Fair?” the Emperor shouted, coming to stand over Gurgeh, blocking the view of the distant fire. “Why does anything have to be fair? Is life fair?” He reached down and took Gurgeh by the hair, shaking his head. “Is it? Is it?”
Gurgeh let the apex shake him. The Emperor let go of his hair after a moment, holding his hand as though he’d touched something dirty. Gurgeh cleared his throat. “No, life is not fair. Not intrinsically.”
The apex turned away in exasperation, clutching again at the curled stone top of the battlements. “It’s something we can try to make it, though,” Gurgeh continued. “A goal we can aim for. You can choose to do so, or not. We have. I’m sorry you find us so repulsive for that.”
“ ‘Repulsive’ is barely adequate for what I feel for your precious Culture, Gurgeh. I’m not sure I possess the words to explain to you what I feel for your… Culture. You know no glory, no pride, no worship. You have power; I’ve seen that; I know what you can do… but you’re still impotent. You always will be. The meek, the pathetic, the frightened and cowed… they can only last so long, no matter how terrible and awesome the machines they crawl around within. In the end you will fall; all your glittering machinery won’t save you. The strong survive. That’s what life teaches us, Gurgeh, that’s what the game shows us. Struggle to prevail; fight to prove worth. These are no hollow phrases; they are truth!”
Gurgeh watched the pale hands grasping the dark stone. What could he say to this apex? Were they to argue metaphysics, here, now, with the imperfect tool of language, when they’d spent the last ten days devising the most perfect image of their competing philosophies they were capable of expressing, probably in any form?
What, anyway, was he to say? That intelligence could surpass and excel the blind force of evolution, with its emphasis on mutation, struggle and death? That conscious cooperation was more efficient than feral competition? That Azad could be so much more than a mere battle, if it was used to articulate, to communicate, to define…? He’d done all that, said all that, and said it better than he ever could now.
“You have not won, Gurgeh,” Nicosar said quietly, voice harsh, almost croaking. “Your kind will never win.” He turned back, looking down at him. “You poor, pathetic male. You play, but you don’t understand any of this, do you?”
Gurgeh heard what sounded like genuine pity in the apex’s voice. “I think you’ve already decided that I don’t,” he told Nicosar.
The Emperor laughed, turning back to the distant reflection of the continent-wide fire still below the horizon. The sound died in a sort of cough. He waved one hand at Gurgeh. “Your sort never will understand. You’ll only be used.” He shook his head in the darkness. “Go back to your room, morat. I’ll see you in the morning.”