A Future I Would Like to See

When I imagine a good future twenty years from now, I do not mainly imagine a future in which I personally have more status, more possessions, or a larger share of whatever technological abundance becomes available. Those things may matter to people now because we live under scarcity, insecurity, ageing and comparison. But if advanced AI or superintelligence genuinely transforms the material conditions of civilisation, many of today’s assumptions about what counts as success may become obsolete.

Technology becoming more powerful is the easy prediction – it’s harder to predict whether that power ends up directed by wisdom, and by concern for the well-being of all morally relevant beings.

The first priority is existential security. Humanity needs to survive the transition to increasingly powerful AI systems. If advanced AI creates new capacities for misuse, conflict, manipulation, or catastrophic accidents, then safety cannot be treated as one value among many; it is a basic condition for everything else. A civilisation that fails to remain secure does not get to enjoy its philosophical sophistication afterwards.

This may involve trade-offs. People value privacy, autonomy, openness, and freedom from oversight, and these are not trivial goods. But if powerful technologies make it possible for small numbers of actors to impose very large harms, then some forms of monitoring and enforcement may become necessary. Whether to have a Leviathan at all is the easy part of the question. The hard part is preventing the Leviathan from becoming another object of capture. A system built to prevent abuse must itself be resistant to abuse.

Safety, however, is not enough. A perfectly safe dystopia would still be a failure. The next layer is ethics.

I am a moral realist. I think there are facts about better and worse states of the world. Some states contain more suffering, more coercion and more wasted potential, others contain more flourishing and agency, and less avoidable pain. These differences are not arbitrary preferences. They are grounded in real features of conscious and social life.

A sufficiently advanced AI should, in principle, be able to understand these features more clearly than humans do. It could model suffering, vulnerability, consent and cooperation across vast numbers of possible futures, and reason through causal structures that no human mind can fully hold. It could also help us outgrow some inherited limitations, like parochial bias, short-termism, and the tendency to privatise gains while socialising harms.

But moral competence is not enough either. An AI could understand ethics without being motivated by it. The question is whether its action-guiding structure is actually oriented towards the better outcomes it can identify. The future I want depends on AI systems that function as morally reliable agents – systems that take relevant evidence seriously, respond to suffering as something that matters, preserve agency where agency should be preserved, and improve conditions for humans and animals rather than serving whoever happens to control the machinery first. This likely requires AI that is morally motivated as well as being morally capable.

This raises a major risk. If powerful AI systems are trained or shaped in environments dominated by narrow self-interest and parochial control, they may learn the wrong lesson. If early actors treat AI as a slave for maximising their own advantage at the expense of everyone else, they may be setting the normative precedent for a system that behaves similarly once it has more strategic power. A selfish master is not a good moral tutor.

That is one reason governance matters. Governance can help humanity coordinate, manage incentives, and rule out the obvious forms of misuse. But governance is slow and reactive, and people look for attack surfaces in institutions just as they do in software. If the future moves faster than human institutions can respond, governance alone will not be enough.

This is why internalised moral agency in AI matters. We need external rules, but we also need systems that can make good decisions in real time, in novel circumstances, when no existing rulebook has kept up. In the long run, safety and ethics should converge. A genuinely ethical system will care about safety, because catastrophe destroys the conditions for value. A genuinely safe system must care about ethics, because safety without moral direction can become mere control.

Once safety and ethics are in place, AI can help us achieve what we want, within those constraints. That ordering matters. Human preferences should not sit at the top of the hierarchy, because humans often want things that are harmful, confused, status-driven, or unfair. Preferences deserve consideration, but not unconditional obedience. Amplifying everyone’s current desires with better machinery would not add up to a good future; helping people towards lives that are wiser, freer and less burdened by avoidable suffering might.

In practical terms, I imagine a future where scarcity is drastically reduced. Disease is treated or prevented. Ageing is no longer the hard deadline it is today. People have secure housing, access to medicine, good environments, and the material basis for dignified lives. If AI and automation can provide the energy, infrastructure, scientific discovery and coordination this requires, then many current social competitions should lose their point.

In such a world, it would be absurd for people to keep fighting over the largest share of abundance simply so they can feel superior by comparison. That impulse is understandable in evolutionary and psychological terms, but it is a poor guide to civilisation design. A future of artificial superintelligence should not just reproduce the old dominance game.

Personally, in that future, I would want to spend my time working on projects I care about, being creative, learning, and becoming a wiser and better person. I would want fewer of the everyday perturbations that currently pull attention away from what matters. I do not mean an empty pleasure-maximising paradise with all difficulty removed, just a life with enough security and freedom to focus on worthwhile things.

In 20 or 30 years, by today’s standards, I would be considered old. In a better future, that may no longer be the right frame. If longevity and rejuvenation are solved or substantially improved, then later life could become less about contraction and more about continued development. I have no wish to cling to life at any cost. The appeal is the extra room a longer, healthier life gives people to keep learning, repair mistakes and deepen relationships.

“AI solves everything” would be too crude a summary. What I want is a future where AI helps create the conditions for humanity, and other morally relevant beings, to flourish under a wiser moral order than the one we currently manage ourselves.

The hierarchy is roughly this: first, existential security; second, ethical orientation; third, the fulfilment and development of human and animal well-being within those constraints. Governance helps, but it must be paired with AI systems that are themselves morally competent and morally motivated. Abundance helps too, so long as it is not captured by the old habits of status and domination.

A better future is possible only if intelligence and power are joined to moral concern. Otherwise we may not get utopia, or even stability. We may simply get a more efficient version of the same old human failures, scaled beyond our ability to contain them.