Posts

Is there a Meaningful Future for Non-Optimal Moral Agents?

In an interview last year, I had a discussion with John Danaher on the Hedonistic Imperative & Superintelligence – a concern he has with HI is that it denies or de-emphasises some kind of moral agency – in moral theory there is a distinction between moral agents (being a responsible actor able to make moral decisions, influence direction of moral progress, shapes its future, and owes duties to others) and moral patients who may be deemed to have limited or no grounds for moral agency/autonomy/responsibility – they are simply a recipient of moral benefits – in contrast to humans, animals could be classified as moral patients – (see Stanford writing on Grounds for Moral Status).

As time goes on, the notion of strong artificial intelligence leading to Superintelligence (which may herald in something like an Intelligence Explosion) and ideas like the hedonistic imperative becomes less sensational sci-fi concepts and more like visions of realizable eventualities. Thinking about moral endpoints comes to me a paradoxical feeling of triumph and disempowerment.

John’s concern is that ensuring the well-being of humans (conscious entities) is consistent with denying their moral agency – minimizing their capacity to act – that there is a danger that the outcome of HI or an Intelligence Explosion may result in sentient life being made very happy forever, but unable to make choices – with a focus on a future entirely based on bliss whilst ignoring other aspects of what makes for a valuable or worthwhile existence.

Artificial Heart chipsSo even if we have a future where a) we are made very happy and b) we are subject to a wide variety of novelty (which I argue for in Novelty Utilitarianism) without some kind of self-determination we may not be able to enjoy part of what arguably makes for a worthwhile existence.

If the argument for moral agency is completely toppled by the argument against free will then I can see why there would be no reason for it – and that bliss/novelty may be enough – though I personally haven’t been convinced that this is the case.

Also the idea that moral agency and novelty should be ranked as auxiliary aspects to the main imperative of reducing suffering/increasing bliss seems problematic – I get the sense that they (agency/novelty) could easily be swapped out for most non-optimal moral agents in the quest for -suffering/+bliss troublesome.
The idea that upon evaluating grounds for moral status, our ethical/moral quotient may not match or even come close to a potential ethical force of a superintelligence is also troubling. If we are serious about the best ethical outcomes, when the time comes, should we be committed to resigning all moral agency to agents that are more adept at producing peek moral outcomes?
ancillary-one-esk-glitchIs it really possible for non-optimal agents to have a meaningful moral input in a universe where they’ve been completely outperformed by moral machines? Is a life of novelty & bliss the most optimal outcome we can hope for?

There probably should be some more discussion on trade-offs between moral agency, peek experience and novelty.

Discussion in this video here starts at 24:02

Below is the whole interview with John Danaher:

Longevity & the Future of Fun with Jamais Cascio

I asked Jamais Cascio about The Hedonistic Imperative & longevity as part of my interview with him, and got some interesting responses

Cascio warns about wireheading* – Jamais urges cautious about changing cognitive systems to increase pleasure because we may lack a sufficient understanding of the 2nd or 3rd order effects – which isn’t to say that we should never ever do it.  Though he is usually not someone to jump at a chance to apply the precautionary principle, he thinks this is one case that warrants it.

 

Longevity may increase peoples tendency to be thoughtful about the future – imagine a decade to think about the ramifications of a certain action (or inaction).

If we have more ability to think about the consequences of our actions, we have a greater ability not just to see the future, but to see ourselves in the future.

Since we evolved to think about our immediate relations, we are somewhat selfish – living a long time might cause us to be more concerned about the future and those in it.

I believe that one of the benefits of radical life-extension will be the radical expansion of our sense of time – our presence in time.

Jamais Cascio on Life Extension & the Hedonistic Imperative.00_01_02_11.Still008

*Note, it should be clear that Wireheading isn’t the same as re-calibration of the hedonic treadmil that David Pearce advocates. See this interview with David Pearce on the subject of wireheading for more details.

Source: Wonder Workshop

Source: Wonder Workshop

– Support SciFuture via Patreon: https://www.patreon.com/scifuture
– Please Subscribe to the SciFuture YouTube Channel: http://youtube.com/subscription_center?add_user=TheRationalFuture