Posts

On Consciousness, Qualia, Valence & Intelligence with Mike Johnson

Andrés L. Gómez Emilsson

Andrés Gómez Emilsson joined in to add very insightful questions for a 3 part interview series with Mike Johnson, covering the relationship of metaphysics to qualia/consciousness/hedonic valence, and defining their terms, whether panpsychism matters, increasing sensitivity to bliss, valence variance, Effective Altruism, cause prioritization, and the importance of consciousness/valence research .

Andrés Gómez Emilsson interviews Mike Johnson

Carving Reality at the Joints

Andrés L. Gómez Emilsson: Do metaphysics matter for understanding qualia, consciousness, valence and intelligence?

Mike Johnson: If we define metaphysics as the study of what exists, it absolutely does matter for understanding qualia, consciousness, and valence. I think metaphysics matters for intelligence, too, but in a different way.

The big question is whether terms like qualia, consciousness, and valence “carve reality at the joints” or whether they’re emergent linguistic constructs that don’t reflect the structure of the universe. And if these things are ‘real’ in some sense, the follow-up question is: how can we formalize these concepts?

Intelligence seems different: it seems like a ‘fuzzy’ concept, without a good “crisp”, or frame-invariant, definition.

Andrés: What about sources of sentient valence outside of human brains? What is the “minimum viable valence organism”? What would you expect it to look like?

Mike Johnson

Mike: If some form of panpsychism is true- and it’s hard to construct a coherent theory of consciousness without allowing panpsychism- then I suspect two interesting things are true.

  1. A lot of things are probably at least a little bit conscious. The “minimum viable valence experiencer” could be pretty minimal. Both Brian Tomasik and Stuart Hameroff suggest that there could be morally-relevant experience happening at the level of fundamental physics. This seems highly counter-intuitive but also logically plausible to me.
  2. Biological organisms probably don’t constitute the lion’s share of moral experience. If there’s any morally-relevant experience that happens on small levels (e.g., quantum fuzz) or large levels (e.g., black holes, or eternal inflation), it probably outweighs what happens on Earth by many, many, many orders of magnitude. Whether it’ll outweigh the future impact of humanity on our light-cone is an open question.

The big question is whether terms like qualia, consciousness, and valence “carve reality at the joints” or whether they’re emergent linguistic constructs that don’t reflect the structure of the universe. And if these things are ‘real’ in some sense, the follow-up question is: how can we formalize these concepts?

In contrast with Brian Tomasik on this issue, I suspect (and hope) that the lion’s share of the qualia of the universe is strongly net positive. Appendix F of Principia Qualia talks a little more about this.

Andrés: What would be the implications of finding a sure-fire way to induce great valence for brief moments? Could this be used to achieve “strategic alignment” across different branches of utilitarianism?

Mike: A device that could temporarily cause extreme positive or negative valence on demand would immediately change the world.

First, it would validate valence realism in a very visceral way. I’d say it would be the strongest philosophical argument ever made.

Second, it would obviously have huge economic & ethical uses.

Third, I agree that being able to induce strong positive & negative valence on demand could help align different schools of utilitarianism. Nothing would focus philosophical arguments about the discount rate between pleasure & suffering more than a (consensual!) quick blast of pure suffering followed by a quick blast of pure pleasure. Similarly, a lot of people live their lives in a rather numb state. Giving them a visceral sense that ‘life can be more than this’ could give them ‘skin in the game’.

Fourth, it could mess a lot of things up. Obviously, being able to cause extreme suffering could be abused, but being able to cause extreme pleasure on-demand could lead to bad outcomes too. You (Andres) have written about wireheading before, and I agree with the game-theoretic concerns involved. I would also say that being able to cause extreme pleasure in others could be used in adversarial ways. More generally, human culture is valuable and fragile; things that could substantially disrupt it should be approached carefully.

A friend of mine was describing how in the 70s, the emerging field of genetic engineering held the Asilomar Conference on Recombinant DNA to discuss how the field should self-regulate. The next year, these guidelines were adopted by the NIH wholesale as the basis for binding regulation, and other fields (such as AI safety!) have attempted to follow the same model. So the culture around technologies may reflect a strong “founder effect”, and we should be on the lookout for a good, forward-looking set of principles for how valence technology should work.

One principle that seems to make sense is to not publicly post ‘actionable’ equations, pseudocode, or code for how one could generate suffering with current computing resources (if this is indeed possible). Another principle is to focus resources on positive, eusocial applications only, insofar as that’s possible– I’m especially concerned about addiction, and bad actors ‘weaponizing’ this sort of research. Another would be to be on guard against entryism, or people who want to co-opt valence research for political ends.

All of this is pretty straightforward, but it would be good to work it out a bit more formally, look at the successes and failures of other research communities, and so on.


A question I find very interesting is whether valence research is socially disruptive or socially stabilizing by default. I think we should try very hard to make it a socially stabilizing force.

A question I find very interesting is whether valence research is socially disruptive or socially stabilizing by default. I think we should try very hard to make it a socially stabilizing force. One way to think about this is in terms of existential risk. It’s a little weird to say, but I think the fact that so many people are jaded, or feel hopeless, is a big existential risk, because they feel like they have very little to lose. So they don’t really care what happens to the world, because they don’t have good qualia to look forward to, no real ‘skin in the game’. If valence tech could give people a visceral, ‘felt sense’ of wonder and possibility, I think the world could become a much safer place, because more people would viscerally care about AI safety, avoiding nuclear war, and so on.

Finally, one thing that I think doesn’t make much sense is handing off the ethical issues to professional bioethicists and expecting them to be able to help much. Speaking as a philosopher, I don’t think bioethics itself has healthy community & dresearch norms (maybe bioethics needs some bioethicsethicists…). And in general, I think especially when issues are particularly complex or technical, I think the best type of research norms comes from within a community.

Andrés: What is the role of valence variance in intelligence? Can a sentient being use its consciousness in any computationally fruitful way without any valence variance? Can a “perfectly flat world(-simulation)” be used for anything computational?

 

Mike: I think we see this today, with some people suffering from affective blunting (muted emotions) but seemingly living functional lives. More generally, what a sentient agent functionally accomplishes, and how it feels as it works toward that goal, seem to be correlated but not identical. I.e., one can vary without the other.

But I don’t think that valence is completely orthogonal to behavior, either. My one-sentence explanation here is that evolution seems to have latched onto the

Why we seek out pleasure: the Symmetry Theory of Homeostatic Regulation

property which corresponds to valence- which I argue is symmetry– in deep ways, and has built our brain-minds around principles of homeostatic symmetry. This naturally leads to a high variability in our valence, as our homeostatic state is perturbed and restored. Logically, we could build minds around different principles- but it might be a lot less computationally efficient to do so. We’ll see. 🙂 One angle of research here could be looking at people who suffer from affective blunting, and trying to figure out if it holds them back: what it makes them bad at doing. It’s possible that this could lead to understanding human-style intelligence better.

Going a little further, we can speculate that given a certain goal or computation, there could be “valence-positive” processes that could accomplish it, and “valence-negative” processes. This implies that there’s a nascent field of “ethical computation” that would evaluate the valence of different algorithms running on different physical substrates, and choose the one that best satisfices between efficiency and valence. (This is of course a huge simplification which glosses over tons of issues…)

Andrés: What should we prioritize: super-intelligence, super-longevity or super-happiness? Does the order matter? Why?

Mike: I think it matters quite a bit! For instance, I think the world looks a lot different if we figure out consciousness *before* AGI, versus if we ignore it until AGI is built. The latter seems to involve various risks that the former doesn’t.

A risk that I think we both agree is serious and real is this notion of “what if accelerating technology leads to Malthusian conditions where agents don’t- and literally can’t, from a competitive standpoint- care about qualia & valence?” Robin Hanson has a great post called “This is the Dream Time” (of relaxed selection). But his book “Age of Em” posits a world where selection pressures go back up very dramatically. I think if we enter such an era without a good theory of qualia, we could trade away a lot of what makes life worth living.

 

Andrés: What are some conceptual or factual errors that you see happening in the transhumanist/rationalist/EA community related to modeling qualia, valence and intelligence?

Mike: First, I think it’s only fair to mention what these communities do right. I’m much more likely to have a great conversation about these topics with EAs, transhumanists, and rationalists than a random person off the street, or even a random grad student. People from this community are always smart, usually curious, often willing to explore fresh ideas and stretch their brain a bit, and sometimes able to update based on purely abstract arguments. And there’s this collective sense that ideas are important and have real implications for the future. So there’s a lot of great things happening in these communities and they’re really a priceless resource for sounding out theories, debating issues, and so on.

But I would highlight some ways in which I think these communities go astray.

Computationalism, functionalism, fun theory, ‘hedonic brain regions’, ‘pleasure neurochemicals’, the reinforcement learning theory of valence, and so on all give the illusion of explanatory depth but don’t actually explain things in a way which allows us to do anything useful.

First, people don’t realize how bad most existing models of qualia & valence are. Michael Graziano argues that most theories of consciousness are worse than wrong- that they play to our intuitions but don’t actually explain anything. Computationalism, functionalism, fun theory, ‘hedonic brain regions’, ‘pleasure neurochemicals’, the reinforcement learning theory of valence, and so on all give the illusion of explanatory depth but don’t actually explain things in a way which allows us to do anything useful.

Second, people don’t realize how important a good understanding of qualia & valence are. They’re upstream of basically everything interesting and desirable.

Here’s what I think has happened, at least in the rationalist community: historically, consciousness research has been a black hole. Smart people go in, but nothing comes out. So communities (such as physicists and LessWrong) naturally have an interest in putting up a fence around the topic with a sign that says

historically, consciousness research has been a black hole. Smart people go in, but nothing comes out. So communities .. naturally have an interest in putting up a fence around the topic with a sign that says ‘Don’t go here!’

‘Don’t go here!’ – But over time, people forgot why the mystery was blocked off, and started to think that the mystery doesn’t exist. This leads to people actively avoiding thinking about these topics without being able to articulate why.

Andrés: Is there value in studying extreme cases of valence? E.g. Buddhist monks who claim to achieve extreme sustainable bliss, or people on MDMA?

Mike: ‘What science can analyze, science can duplicate.’ And studying outliers such as your examples is a time-honored way of gathering data with high signal-to-noise. So yes, definitely. 🙂


Also see the 1st part, and the 2nd part of this interview series. Also this interview with Christof Koch will likely be of interest.

 

Mike Johnson is a philosopher living in the Bay Area, writing about mind, complexity theory, and formalization. He is Co-founder of the Qualia Research Institute. Much of Mike’s research and writings can be found at the Open Theory website.
‘Principia Qualia’ is Mike’s magnum opus – a blueprint for building a new Science of Qualia. Click here for the full version, or here for an executive summary.
If you like Mike’s work, consider helping fund it at Patreon.

Towards the Abolition of Suffering Through Science

An online panel focusing on reducing suffering & paradise engineering through the lens of science.

Panelists: Andrés Gómez Emilsson, David Pearce, Brian Tomasik and Mike Johnson

Note, consider skipping to to 10:19 to bypass some audio problems in the beginning!!


Topics

Andrés Gómez Emilsson: Qualia computing (how to use consciousness for information processing, and why that has ethical implications)

  • How do we know consciousness is causally efficacious? Because we are conscious and evolution can only recruit systems/properties when they do something (and they do it better than the available alternatives).
  • What is consciousness’ purpose on animals?  (Information processing).
  • What is consciousness’ comparative advantage?  (Phenomenal binding).
  • Why does this matter for suffering reduction? Suffering has functional properties that play a role in the inclusive fitness of organisms. If we figure out exactly what role they play (by reverse-engineering the computational properties of consciousness), we can substitute them by equally (or better) functioning non-conscious or positive hedonic-tone analogues.
  • What is the focus of Qualia Computing? (it focuses on basic fundamental questions and simple experimental paradigms to get at them (e.g. computational properties of visual qualia via psychedelic psychophysics)).

Brian Tomasik:

  • Space colonization “Colonization of space seems likely to increase suffering by creating (literally) astronomically more minds than exist on Earth, so we should push for policies that would make a colonization wave more humane, such as not propagating wild-animal suffering to other planets or in virtual worlds.”
  • AGI safety “It looks likely that artificial general intelligence (AGI) will be developed in the coming decades or centuries, and its initial conditions and control structures may make an enormous impact to the dynamics, values, and character of life in the cosmos.”,
  • Animals and insects “Because most wild animals die, often painfully, shortly after birth, it’s plausible that suffering dominates happiness in nature. This is especially plausible if we extend moral considerations to smaller creatures like the ~1019 insects on Earth, whose collective neural mass outweighs that of humanity by several orders of magnitude.”

Mike Johnson:

  • If we successfully “reverse-engineer” the patterns for pain and pleasure, what does ‘responsible disclosure’ look like? Potential benefits and potential for abuse both seem significant.
  • If we agree that valence is a pattern in a dataset, what’s a good approach to defining the dataset, and what’s a good heuristic for finding the pattern?
  • What order of magnitude is the theoretical potential of mood enhancement? E.g., 2x vs 10x vs 10^10x
  • What are your expectations of the distribution of suffering in the world? What proportion happens in nature vs within the boundaries of civilization? What are counter-intuitive sources of suffering? Do we know about ~90% of suffering on the earth, or ~.001%?
  • Valence Research, The Mystery of Pain & Pleasure.
  • Why is it such an exciting time round about now to be doing valence research?  Are we at a sweet spot in history with this regard?  What is hindering valence research? (examples of muddled thinking, cultural barriers etc?)
  • How do we use the available science to improve the QALY? GiveDirectly has used change in cortisol levels to measure effectiveness, and the EU (what’s EU stand for?) evidently does something similar involving cattle. It seems like a lot of the pieces for a more biologically-grounded QALY- and maybe a SQALY (Species and Quality-Adjusted Life-Year)- are available, someone just needs to put them together. I suspect this one of the lowest-hanging highest-leverage research fruits.

David Pearce: The ultimate scope of our moral responsibilities. Assume for a moment that our main or overriding goal should be to minimise and ideally abolish involuntary suffering. I typically assume that (a) only biological minds suffer and (b) we are probably alone within our cosmological horizon. If so, then our responsibility is “only” to phase out the biology of involuntary suffering here on Earth and make sure it doesn’t spread or propagate outside our solar system. But Brian, for instance, has quite a different metaphysics of mind, most famously that digital characters in video games can suffer (now only a little – but in future perhaps a lot). The ramifications here for abolitionist bioethics are far-reaching.

 

Other:
– Valence research, Qualia computing (how to use consciousness for information processing, and why that has ethical implications),  animal suffering, insect suffering, developing an ethical Nozick’s Experience Machine, long term paradise engineering, complexity and valence
– Effective Altruism/Cause prioritization and suffering reduction – People’s practical recommendations for the best projects that suffering reducers can work on (including where to donate, what research topics to prioritize, what messages to spread). – So cause prioritization applied directly to the abolition of suffering?
– what are the best projects people can work on to reduce suffering? and what to work on first? (including where to donate, what research topics to prioritize, what messages to spread)
– If we successfully “reverse-engineer” the patterns for pain and pleasure, what does ‘responsible disclosure’ look like? Potential benefits and potential for abuse both seem significant
– If we agree that valence is a pattern in a dataset, what’s a good approach to defining the dataset, and what’s a good heuristic for finding the pattern?
– What order of magnitude is the theoretical potential of mood enhancement? E.g., 2x vs 10x vs 10^10x

Panelists

David Pearce: http://hedweb.com/
Mike Johnson: http://opentheory.net/
Andrés Gómez Emilsson: http://qualiacomputing.com/
Brain Tomasik: http://reducing-suffering.org/

 

#hedweb ‪#EffectiveAltruism ‪#HedonisticImperative ‪#AbolitionistProject

The event was hosted on the 10th of August 2015, Venue: The Internet

Towards the Abolition of Suffering Through Science was hosted by Adam Ford for Science, Technology and the Future.

Towards the Abolition of Suffering Through Science

Towards the Abolition of Suffering Through Science

The Point of View of the Universe – Peter Singer

Peter Singer discusses the new book ‘The Point Of View Of The Universe – Sidgwick & Contemporary Ethics’ (By Katarzyna de Lazari-Radek and Peter Singer) He also discusses his reasons for changing his mind about preference utilitarianism.

 

Buy the book here: http://ukcatalogue.oup.com/product/97… Bart Schultz’s (University of Chicago) Review of the book: http://ndpr.nd.edu/news/49215-he-poin… “Restoring Sidgwick to his rightful place of philosophical honor and cogently defending his central positions are obviously no small tasks, but the authors are remarkably successful in pulling them off, in a defense that, in the case of Singer at least, means candidly acknowledging that previous defenses of Hare’s universal prescriptivism and of a desire or preference satisfaction theory of the good were not in the end advances on the hedonistic utilitarianism set out by Sidgwick. But if struggles with Singer’s earlier selves run throughout the book, they are intertwined with struggles to come to terms with the work of Derek Parfit, both Reasons and Persons (Oxford, 1984) and On What Matters (Oxford, 2011), works that have virtually defined the field of analytical rehabilitations of Sidgwick’s arguments. The real task of The Point of View of the Universe — the title being an expression that Sidgwick used to refer to the impartial moral point of view — is to defend the effort to be even more Sidgwickian than Parfit, and, intriguingly enough, even more Sidgwickian than Sidgwick himself.”

Suffering, and Progress in Ethics – Peter Singer

Peter Singer_profileSuffering is generally bad – Peter Singer (who is a Hedonistic Utilitarian), and most Effective Altruists would agree with this. Though in addressing the need for suffering today Peter acknowledges that, as we are presently constituted, suffering is useful as a warning sign (e.g. against further injury). But what about the future?
What if we could eliminate suffering?
Perhaps in the future we will have advanced technological interventions to warn us of danger that will be functionally similar to suffering, but without the nasty raw feels.
Peter Singer, like David Pearce, suggests that if we could eliminate suffering of non-human animals that are capable of suffering, perhaps in some way that is difficult to imagine now – that this would be a good thing.

Video Interview:

I would see no reason to regret the absence of sufferingPeter Singer
Peter can’t see any regret to lament the disappearance of suffering, though perhaps people may say it would be useful to help understand literature of the past. Perhaps there are some indirect uses for suffering – but on balance Peter thinks that the elimination of suffering would be an amazingly good thing to do.

Singer thinks it is interesting to speculate what might be possible for the future of human beings, if we do survive over the longer term. To what extent are we going to be able to enhance ourselves? In particular to what extent are we going to be more ethical human beings – which brings to question ‘Moral Enhancement’.

The Expanding Circle - Peter SingerHave we made Progress in Ethics? Peter argues for the case that our species has expanded the circle of our ethical concern we have in his book ‘The Expanding Circle‘, and more recently Steven Pinker took up this idea in ‘Better Angels Of Our Nature’ – and this has happened over the millennia, beyond initially the tribal group, then to a national level, beyond ethnic groups to all human beings, and now we are starting to expand moral concern to non-human sentient beings as well.

Steven Pinker thinks that increases in our ethical consideration is bound up with increases in our intelligence (as proposed by James Flynn – the Flynn Effect – though this research is controversial (it could be actual increases in intelligence or just the ability to do more abstract reasoning)) and increases in our ability to reason abstractly.

As mentioned earlier there are other ways in which we may increase our ability and tendency to be more moral (see Moral Enhancement), and in the future we may discover genes that may influence us to think more about others, to dwell less on negative emotions like anger or rage. It is hard to say whether people will use these kinds of moral enhancers voluntarily, or whether we need state policies to encourage people to use moral enhances in order to produce better communities – and there are a lot of concerns here that people may legitimately have about how the moral enhancement project takes place. Peter sees this as a fascinating prospect and that it would be great to be around to see how things develop over the next couple of centuries.

Note Steven Pinker said of Peter’s book:

Singer’s theory of the expanding circle remains an enormously insightful concept, which reconciles the existence of human nature with political and moral progress. It was also way ahead of its time. . . . It’s wonderful to see this insightful book made available to a new generation of readers and scholars.Steven Pinker

The Expanding Circle

Abstract: What is ethics? Where do moral standards come from? Are they based on emotions, reason, or some innate sense of right and wrong? For many scientists, the key lies entirely in biology–especially in Darwinian theories of evolution and self-preservation. But if evolution is a struggle for survival, why are we still capable of altruism?

Peter Singer - The Most Good You Should Do - EA Global Melbourne 2015In his classic study The Expanding Circle, Peter Singer argues that altruism began as a genetically based drive to protect one’s kin and community members but has developed into a consciously chosen ethic with an expanding circle of moral concern. Drawing on philosophy and evolutionary psychology, he demonstrates that human ethics cannot be explained by biology alone. Rather, it is our capacity for reasoning that makes moral progress possible. In a new afterword, Singer takes stock of his argument in light of recent research on the evolution of morality.

References:
The Expanding Circle book page at Princeton University: http://press.princeton.edu/titles/9434.html

The Flynn Effect: http://en.wikipedia.org/wiki/Flynn_effect

Peter Singer – Ethics, Evolution & Moral Progress – https://www.youtube.com/watch?v=91UQAptxDn8

For more on Moral Enhancement see Julian Savulescu’s and others writings on the subject.

Subscribe to this Channel: http://youtube.com/subscription_center?add_user=TheRationalFuture

Science, Technology & the Future: http://scifuture.org

Peter Singer & David Pearce on Utilitarianism, Bliss & Suffering

Moral philosophers Peter Singer & David Pearce discuss some of the long term issues with various forms of utilitarianism, the future of predation and utilitronium shockwaves.

Topics Covered

Peter Singer

– long term impacts of various forms of utilitarianism
– Consciousness
– Artificial Intelligence
– Reducing suffering in the long run and in the short term
– Practical ethics
– Pre-implantation genetic screening to reduce disease and low mood
– Lives today are worth the same as lives in the future – though uncertainty must be brought to bear in deciding how one weighs up the importance of life
– The Hedonistic Imperative and how people react to it
– Correlation to high hedonic set points with productivity
existential risks and global catastrophic risks
– Closing factory farms

David Pearce

– Veganism and reducitarianism
– Red meat vs white meat – many more chickens are killed per ton of meat than beef
– Valence research
– Should one eliminate the suffering? And should we eliminate emotions of happiness?
– How can we answer the question of how far suffering is present in different life forms (like insects)?

Talk of moral progress can make one sound naive. But even the darkest cynic should salute the extraordinary work of Peter Singer to promote the interests of all sentient beings.David Pearce
 

 

Many thanks for watching!
– Support me via Patreon: https://www.patreon.com/scifuture
– Please Subscribe to this Channel: http://youtube.com/subscription_cente…
– Science, Technology & the Future website: http://scifuture.org

The Knowledge Argument Applied to Ethics

A group of interested AI enthusiasts have been discussing Engineering Machine Consciousness in Melbourne for over a decade. In a recent interview with Jamais Cascio on Engineering Happy People & Global Catastrophic Risks, we discussed the benefits of amplifying empathy without the nasty side effects (possibly through cultural progress or technological intervention – a form of moral enhancement). I have been thinking further about how an agent might think and act differently if it had no ‘raw feels’ – any self-intimating conscious experience.

I posted to the Hedonistic Imperative Facebook group:

Is the limitations of empathy in humans distracting us from the in principle benefits of empathy?
The side effects of empathy in humans include increased distrust of the outgroup – and limitations in the amount of people we humans can feel strong empathy for – though in principle the experience of understanding another person’s condition from their perspective seems quite useful – at least while we are still motivated by our experience.
But what of the future? Are our post human descendants likely to be motivated by their ‘experiences of’ as well as their ‘knowledge about’ in making choices regarding others and about the trajectories of civilizational progress?

I wonder whether all the experiences of can be understood in terms of knowledge about – can the whole of ethics be explained without being experienced – though knowledge about without any experience of? Reminds me of the Mary’s Room/Knowledge Argument* thought experiment. I leaned towards the position that Mary could with a fully working knowledge of the visual system and relevant neuroscience wouldn’t ‘learn’ anything new when walking outside the grey-scale room and into the colourful world outside.
Imagine an adaptation of the Mary’s Room thought experiment – for the time being let’s call it Autistic Savant Angela’s Condition – in that:

class 1

Angela is a brilliant ethicist and neuroscientist (an expert in bioethics, neuroethics etc), whom (for whatever reason) is an Autistic savant with congenital insensitivity to pain and pleasure – she can’t at all feel pain, suffering or experience what it is like to be someone else who does experience pain or suffering – she has no intuition of ethics. Throughout her whole life she has been forced to investigate the field of ethics and the concepts of pleasure, bliss, pain and suffering through theory alone. She has a complete mechanical understanding of empathy, and brain states of subjects participating on various trolley thought experiments, hundreds of permutations of Milgrim experiments, is an expert in philosophies of ethics from Aristotle to Hume to Sidgwick etc. Suddenly there is a medical breakthrough in gene-therapy that would guarantee normal human function to feel without impairing cognitive ability at all. If Angela were to undergo this gene-therapy, would she learn anything more about ethics?

class 2

Same as above except Angela has no concept of other agents.

class 3

Same as class 2 except Angela is a superintelligent AI, and instead of gene-therapy, the AI recieves a software/hardware upgrade that allows the AI access to ‘fire in the equations’, to experience. Would the AI learn anything more about ethics? Would it act in a more ethical way? Would it produce more ethical outcomes?

 

Implications

Should an effective altruist support a completely dispassionate approach to cause prioritization?

If we were to build an ethical superintelligence – would having access to visceral experiences (i.e. pain/pleasure) change it’s ethical outcomes?
If a superintelligence were to perform Coherent Extrapolated Volition, or Coherent Aggregated Volition, would the kind of future which it produced differ if it could experience? Would likelihoods of various ethical outcomes change?

Is experience required to fully understand ethics? Is experience required to effectively implement ethics?
Robo Brain

 

footnotes

The Knowledge Argument Thought Experiment

jacksons-knowledge-argumentMary is a brilliant scientist who is, for whatever reason, forced to investigate the world from a black and white room via a black and white television monitor. She specializes in the neurophysiology of vision and acquires, let us suppose, all the physical information there is to obtain about what goes on when we see ripe tomatoes, or the sky, and use terms like ‘red’, ‘blue’, and so on. She discovers, for example, just which wavelength combinations from the sky stimulate the retina, and exactly how this produces via the central nervous system the contraction of the vocal cords and expulsion of air from the lungs that results in the uttering of the sentence ‘The sky is blue’. […] What will happen when Mary is released from her black and white room or is given a color television monitor? Will she learn anything or not?

The Life You Can Save – Interview with Peter Singer by Adam Ford

Transcript of interview with Peter Singer on ‘The Life You Can Save‘:

The Life You Can Save Amazon 41Cnq4M0rzL._SX322_BO1,204,203,200_I’ve been writing & thinking about the question of global poverty and what affluent people ought to do about it for more than four years now. The first article I published on that ‘Famine Affluence & Morality’ came out in 1972.
But I’d never written a book on that topic until a few years ago when I was encouraged by greater interest in the question of global poverty and greater interest in the question of the obligations of the affluent, and some very positive responses I had to an article that appeared in the New York Times Sunday Magazine. So it was then that I decided that it was time for a book on this topic – I wrote “The Life You Can Save” which came out in 2009 – essentially to encourage people to think that – if you really want to think of yourself as ethical – you want to say “I’m an ethical person, I’m living an ethical life” – it’s not enough to say “I don’t kill.. I don’t steal.. I don’t cheat.. I don’t assault people..” and so on and all those sorts of ‘thou shalt not’ kind of commands – but if you are an affluent person, or even just a middle class person in an affluent country (that’s undoubtedly affluent by global standards) then you’re not really living an ethical life unless you’re prepared to share some of the good fortune that you have with people who are less fortunate – and you know, just the fact that you are able to live in an affluent country means that you are fortunate.
One of the points that I try to emphasize is that the differences in the amount of wealth that people who are middle class or above in affluent societies as compared to people in extreme poverty have is so great that things that are really pretty frivolous in our lives (that we spend money on that are not significant) could make a difference in the lives of other people. So, we spend money on all kinds of things from going to a cafe and having coffee or buying a bottle of water when we could drink water out of a tap, to going on vacations, making sure we have the latest iPhones – a whole lot of different things that really are not at all life changing to us. But the amount that we spend on those things could be life changing to people in extreme poverty who are not able to afford minimal health care for themselves or their family, who don’t get enough to eat, to perhaps are not able to send their children to school because they need them to work in the fields – a whole of different things that we can help people with.
I think we ought to be helping – I think to do nothing is wrong. Now, you can then debate “well, how much ought we to be doing?” which is something I do discuss in the book – but I think that it’s clear that we ought to be setting aside some of what we have to donate to organizations that are effective and that have been proven to be effective in helping the global poor.
So that’s really what the book is about – and perhaps the other thing I ought to say is that many people feel that somehow world poverty is like a bottomless pit that we are pouring money down into – but I think that’s a mistake – I think that it’s clear that we are making a difference – that we’ve dramatically reduced the number of people who die from preventable poverty related illnesses. Even just since my book as been published (in the last 5 years for instance) the number people – the number of children who die each year (children under 5) has dropped from nearly 10 million a year to 6.6 million a year – so, you know, that real progress in just 5 years – and we can make progress with poverty if we are careful about the way in which we do it and fund organizations that are open to evidence about what works and what doesn’t.

Many people feel that somehow world poverty is like a bottomless pit that we are pouring money down into – but I think that’s a mistake – I think that it’s clear that we are making a difference.

Peter Singer – The Life You Can Save – Video inteview by Adam Ford


The Life You Can Save

Acting Now to End World Poverty is a 2009 book by Australian philosopher Peter Singer. The author argues that citizens of affluent nations are behaving immorally if they do not act to end the poverty they know to exist in developing nations.

The book is focused on giving to charity, and discusses philosophical considerations, describes practical and psychological obstacles to giving, and lists available resources for prospective donors (e.g. charity evaluators). Singer concludes the book by proposing a minimum ethical standard of giving.

Christian Barry and Gerhard Overland (both from the Centre for Applied Philosophy and Public Ethics) described the widespread acceptance for the notion that “the lives of all people everywhere are of equal fundamental worth when viewed impartially”. They then wonder, during the book review in the Journal of Bioethical Inquiry, why “the affluent do so little, and demand so little of their governments, while remaining confident that they are morally decent people who generally fulfil their duties to others?” The reviewers agree with Singer, and say they see a conflict between the behaviours of the affluent and the claims of the affluent to being morally decent people. The reviewers also discuss other practical ways to fight poverty.

Support The Life You Can SaveHistorically, every dollar donated to The Life You Can Save typically moves an additional three to four dollars to our effective charities. Our recommended charities provide services and support to men, women, and children in needy communities across the globe. Your support for our work means that you'll get the most out of your charitable giving!

Can I Really Save Lives?  The good news, in fact the great news, is that you can! While there are endless problems in the world that you as an individual cannot solve, you can actually save lives and reduce unnecessary suffering and premature death. Should you do it? Watch this video and decide for yourself. The information on our website can help you give most effectively to become a life saver.

Also see the Life You Can Save Infograph Video

Buy The Life You Can Save on Amazon

The Life You Can Save Facebook Page

Rationality & Moral Judgement – Simon Laham

Rationality & Moral Judgement – A view from Moral Psychology. Talk given at EA Global Melbourne 2015. Slides here.

Simon Laham - QandAWhat have we learned from an empirical approach to moral psychology – especially in relation to the role of rationality in most every day morality?
What are some lessons that the EA movement can take from moral psychology?

Various moral theorists over the years have had different emphasis on the roles that the head and heart play in moral judgement. Early conceptions of the role of the head in morality were that it drives moral judgement. A Kantian might say that the head/reasoning drives moral judgement – when presented with a dillema of some kind, the human engages with ‘system 2’ like processes in a controlled rational nature. An advocate of a Humean model may favor the idea that emotion or the heart (‘system 1’ thinking) plays the dominant role in moral judgement. Modern psychologists often take a hybrid model where both system 1 and system 2 styles of thinking are at play in contributing to the way we judge right from wrong.

Moral Judgement & Decision making is driven by a variety of factors:

  • Emotions (e.g., Valdesolo & DeSteno, 2006)
  • Values (e.g., Crone & Laham, 2015)
  • Relational and group membership concerns (e.g., Cikara et al., 2010)

Across a wide range of studies, a majority of people do not consistently apply abstract moral principles – Moral judgments are not decontextualized, depersonalized and asocial (i.e., not System 2)
Simon Laham - Rationality & Moral Judgement - Effective Altruism Global Melbourne 2015
Not only do people inconsistently apply rationality in moral judgments, many reject the idea that consequentialist rationality should have any place in the moral domain.

  • Appeals to consequentialist logic may backfire (Kreps and Monin, 2014)
  • People who give consequentialist justifications for their moral positions are viewed as less committed and less authentic

Is trying to change people’s minds the best way to expand the EA movement?
Moral judgment is subject to a variety of contextual effects. Knowledge of such effects can be used to ‘nudge’ people towards utilitarianism (see Thaler & Sunstein, 2008).

‘Practical’ take-home
Things beside rationality matter in morality and people believe that things beside rationality should matter.
So:
(a) present EA in a manner that does not trade utilitarian options off against deeply held values, identities, or emotions
(b) use decision framing techniques to ‘nudge’ people towards utilitarian choices

 


Consider watching Simon’s talk at the festival of dangerous ideas about his book ‘The Joy of Sin‘.

Also Simon wrote an article for Huffington post where he says : “I confess it, I am a sinner. I begin most days in a haze of sloth and lust (which, coincidentally, is also how I end most days); gluttony takes hold over breakfast and before I know it I’m well on my way to hell and it’s not yet 9 a.m. Pride, lust, gluttony, greed, envy, sloth and anger, the seven deadly sins, these are my daily companions.

And you? Are you a sinner?

The simple fact is that we all sin (or rather ‘sin’), and we do it all the time. But fear not: the seven deadly sins aren’t as bad for you as you might think.”


Simon Laham BandWSimon Laham is a senior lecturer in the psychology department at Melbourne University. He has worked over the last 8 years on the psychology of morality from the point of view of experimental social psychology.
Key research questions : How do we make moral judgments? How do others influence what we do?

Many thanks for watching!
– Support SciFuture via Patreon
– Please Subscribe the SciFuture YouTube Channel
Science, Technology & the Future

Philosophy & Effective Altruism – Peter Singer, David Pearce, Justin Oakley, Hilary Greaves

Panelists ([from left to right] Hilary Greaves, Peter Singer, Justin Oakley & David Pearce) discuss what they believe are important philosophical aspects of the Effective Altruism movement – from practical philosophy we can use today to possible endpoints implied by various frameworks in applied ethics. The panelists navigate through a wide range of fascinating and important topics that aspiring effective altruists and anyone whom is philosophically inclined will find both useful and enjoyable.

Panel moderated by Kerry Vaughan.

Panel Transcript

(in progress)

0:35 Question “What are the hot topics in philosophy that might change what effective altruists might focus on?”
Hilary Greaves – So, my answer to that – the one I’m most directly familiar with is the one I already mentioned in my talk earlier. I think that population ethics can make a massive difference to a significant proportion of the things we should worry about as EAs. In particular, the thing that gives rise to this is the situation where – at the moment we have lots of moral philosophers who really like their ivory tower abstract theorising – those people have done a lot of discussing this abstract question of ‘ok what is my theory of population ethics’ – then at the real world extreme we have lots of people engaging directly with the real world issues thinking, ok, how should we do our cost-benefit analysis, for example family planning. We have a big gap in the middle – we don’t really have a well developed community of people are both in touch with the background moral philosophy and who are interested in applying it to the real world. So because there is that gap I think there’s a lot of low hanging fruit at the moment for people who have a background in moral philosophy and who are also plugged into the EA community to build these bridges from theory to practise and see what it all means for the real world.

01:56 Peter Singer – I actually agree with that – that population ethics is an important area – and another place that connects to what Hilary was talking about earlier is for the existential risk questions. Because, we need to think about – suppose that the world were destroyed – is what’s so bad about that the fact that 7.5 people have lost their lives or is it the loss of the untold billions of people that Nick Bostrom has (10^56 or something, I don’t know – some vastly unimaginable number) of possible future lives that could have been good, and that would be lost. So that seems to me to be a real issue. If you want something that’s a little more nitty gritty towards what we are talking about today – another issue is – how do we get a grip on the nature and extent of animal suffering? (something that we will be talking a bit about in a moment) It’s really just hard to say – David just talked about factory farming and the vast amount of billions of animals suffering in factory farms – and I totally agree that this is a top priority issue – but in terms of assessing priorities, how do we compare the suffering of a chicken in a factory farm to, let’s say, a mother who has to watch her child dying of malaria? Is there some way we can get a better handle on that?

03:23 Justin Oakley – For me, I think, one of the key issues in ethics at the moment that bears on Effective Altruism at the moment is what’s known as a ‘situationists critique of virtue ethics’ – so trying to understand not only on how having a better character helps people to act well but also what environment they are in. Subtle environmental influences that might either support or subvert a person acting well – in particular having the virtue perhaps of liberality – so there is lots of interesting work being done on that – some people think that debate is beginning to die down – but it seems to be just starting up again with a couple of new books that are coming out looking at a new twist on that. So for me, I’m keen to do that – I guess my own work at Monash I teach a lot of health professionals so keen to look at what environmental influences there are on doctors that impede them having a theraputic relationship on patients – not only thinking about how to help them be more virtuous – I suppose which is not the only thing I aim to do with the doctors that I teach but I hope to have that influence to some extent.

Panel Greaves Singer Oakley Pearce - Orgasmatronium - 1

The Utilitarianism at the End of the Universe – Panelists Hilary Greaves, Peter Singer, Justin Oakley & David Pearce laugh about the possible ‘end games’ of classical utilitarianism.

04:25 David Pearce – Yes well I’m perhaps slightly out of touch now with analytic philosophy – but one issue that I haven’t really seen tackled by analytic philosophy is this disguised implication of classical utilitarianism of what we ought to be doing, which is essentially optimising matter and energy throughout the world – and perhaps the accessible universe – for maximum bliss. A questioner earlier was asking ‘Well, as a negative utilitarian, do you accept this apparent counter-intuitive consequence that one ought to wipe out the world to prevent the suffering of one person.’ But if one is a classical utilitarian then it seems to be a disguised consequence that it’s not good enough to aim merely for a civilization in which all sentient beings could flourish and enjoy gradients of intelligent bliss – instead one must go on remorselessly to when matter and energy is nothing but pure orgasmic bliss.
05:35 Peter Singer – I find it a remorseless and unusual term to describe it
[laughter…] 5:40 David Pearce – Well, I think this is actually rather an appealing idea to me but I know not everyone shares this intuition
[laughter…] 5:50 Question “So Peter I’d be interested to know if you have thoughts on whether you think that’s an implication of classical utilitarianism” – Peter Singer – If I accept that implication? Well – David and I talked about this a little bit earlier over lunch – I sort of, I guess, maybe I accept it but I have difficulty in grasping what it is to talk about converting matter and energy into bliss – unless we assume that there are going to be conscious minds that are going to be experiencing this bliss. And, of course then David would then very strongly agree that conscious minds not only have to experience bliss but also not experience any suffering certainly, presumably minimize anything that they experience other than bliss (because that’s not converting matter and energy into bliss) – so if what I’m being asked to imagine is a universe with a vast number of conscious minds that are experiencing bliss – yeah, maybe I do accept that implication.

06:43 Question “So this is a question mostly for Justin – effective altruists often talk about doing the ‘most good’ – should EAs be committed to doing ‘enough good’ instead of the ‘most good’?”

Justin Oakley – Yeah, that’s a good question to ask – one of the thinks I didn’t emphasize in my talk on virtue ethics is that standardly virtue ethics thinks that we should strive to be an excellent human being, which can fall a little way short of producing the maximum good.  So if you produce an excellent level of liberality or perhaps good or benefit to someone else then that’s enough for virtue.  I guess in some of the examples I was giving in my talk you might choose a partner who – although you’re not the ultimate power couple (you are the sub-optimal power couple) but you are none the less attracted to that other person very strongly – from the perspective of effective altruism it might sound like you are doing the wrong thing – but intuitively it doesn’t seem to be wrong.  That’s one example.

07:51 Hilary Graves – Surely, I mean – something I can say a bit about a similar issue looks like from a more consequentialist perspective – when people think of consequentialism they sometimes assume that consequentialists think that there is a moral imperative to maximize it – you absolutely have to do the most good and anything less than that is wrong.  But it’s worth emphasising that not all consequentialists emphasise that at all – not all consequentialists think that it’s even helpful to buy into this language of right and wrong.  So you don’t have to be a virtue ethicist to somewhat feel alienated from a moral imperative to do the absolute most good – you could just think something like the following: never mind right and wrong, never mind what I should vs not allowed to be doing.  I might just want to make the world better – I might just think I could order all the things I could possibly do in terms of better or worse.   And then you know, if I give away 1% of my income, that’s better thank giving away nothing – if I give 5% that’s better than giving away 1% – if I give away 50% that’s better than anything less – but I don’t have to impose some sharp cutoff and say that I’m doing something morally wrong if I give less than that – I think if we think in this way then we tend to alienate both ourselves and other people less – there’s something very alienating about holding up a very high standard and saying that anybody including ourselves who is falling short of this very high standard is doing something wrong with a capital ‘R’.

Panel including Peter Singer

09.14 Peter Singer – So, in a way I agree – you are talking about a spectrum view (of where we have a spectrum of black to white – or maybe we don’t want to use those terms for it) from one end to the other and you’re somewhere on the spectrum and you try and work your way further up the spectrum perhaps – which, I’m reasonably comfortable with that.  Another way of looking at it (and this goes back to something that Sidgewick also said) is that we ought to be clearer about distinguishing when we regard the act as the right act or the wrong act and when we regard it appropriate to praise or blame people for doing it.  And these are really separate things – especially if you are a consequentialist because praising or blaming someone is an act – and you ought to only do that if it will have good consequences.  So I suppose that we think that somebody in particular personal circumstances ought to  be giving 50% of his earnings away – but he is only giving 10% – but he is living in a society like ours in which by giving 10% he is at the top 99.99% of what people are giving.  Well to blame him saying ‘oh well your only giving 10% – you should be giving more’ looks like it’s going to be very counter productive – you really want to praise him in front of other people so that more people will give 10%.  So I think if we understand it that way that’s another way of looking at it – I’m not sure if it’s nesseccarily better than the spectrum view that you [Hilary] was suggesting – but it is another way of, if you like, softening this black white morality idea that is either is right or is wrong.

10:54 Question “A question for Hilary – You mention that one might find the ‘uncertainty’ that your talk generates kind of paralyzing – but you mention that wasn’t your conclusion – can you expland on why this is (paralyzation) not your conclusion?”
Hilary Greaves

 

  • Transcribed by Adam Ford

Biographies

Hilary Greaves - LectureHilary Greaves is an Associate Professor in Philosophy, at Somerville College in the University of Oxford. My current research focusses on various issues in ethics. Hilary’s interests include: foundational issues in consequentialism (‘global’ and ‘two-level’ forms of consequentialism), the debate between consequentialists and contractualists, aggregation (utilitarianism, prioritarianism and egalitarianism), moral psychology and selective debunking arguments, population ethics, the interface between ethics and economics, the analogies between ethics and epistemology, and formal epistemology. Hilary currently (2014-17) directs the project Population Ethics: Theory and Practice, based at the Future of Humanity Institute, and funded by The Leverhulme Trust.

Peter Singer - Non-Human Animal Ethics - EA Global Melbourne 2015Peter Singer is an Australian moral philosopher. He is currently the Ira W. DeCamp Professor of Bioethics at Princeton University, and a Laureate Professor at the Centre for Applied Philosophy and Public Ethics at the University of Melbourne. He specializes in applied ethics and approaches ethical issues from a secular, utilitarian perspective.  He is known in particular for his book, Animal Liberation (1975), a canonical text in animal rights/liberation theory. For most of his career, he supported preference utilitarianism, but in his later years became a classical or hedonistic utilitarian, when co-authoring The Point of View of the Universe with Katarzyna de Lazari-Radek.

Justin Oakley - Virtue & Effective Altruism - EA Global Melbourne 2015Justin Oakley is an Associate Professor at Monash University – the School of Philosophical, Historical & International Studies, and Centre for Human Bioethics. Justin has been part of the revival of the ethical doctrine known as virtue ethics, an Aristotelian doctrine which has received renewed interest in the past few decades.  Oakley is particularly well known for his work on professional ethics and also the so-called ‘problem’ of friendship. The problem of friendship looks at how a strict application of impartialist ethical doctrines, such as utilitarianism and Kantianism, conflicts with our notions of friendship or ‘true friendship’.

David PearceDavid Pearce is a British philosopher who promotes the idea that there exists a strong ethical imperative for humans to work towards the abolition of suffering in all sentient life. His book-length internet manifesto The Hedonistic Imperative outlines how technologies such as genetic engineering, nanotechnology, pharmacology, and neurosurgery could potentially converge to eliminate all forms of unpleasant experience among human and non-human animals, replacing suffering with gradients of well-being, a project he refers to as “paradise engineering”.

Be Greedy For The Most Good You Can Do – Kerry Vaughan – EA Global Melbourne 2015

Filmed at EA Global Melbourne 2015 Slides of talk are here
Kerry Vaughan discusses:
What is effective altruism? what is it’s history? what isn’t EA? and how to succeed at being an effective altruist.
Approaches to doing good include:
– Being Skeptical – using the case study of play pumps in africa – hoping to utilize the renewable energy of children playing – on the surface it looked like a good idea, but unfortunately it didn’t work – so be skeptical
– Changing your Mind – you can score social points in the EA movement by changing your mind – so yay! Moving beyond entrenched beliefs to better ways of thinking leads decision making – do change your mind, update your beliefs when there is evidence to support you doing so
– Do it! – when you find out better approaches to being altruistic, actually follow up and do it – without getting too involved in theorizing whether you have a moral obligation to solve the problem, just go solve it
– 3 strands to the history of EA – Peter Singer’s work, Holden Karlovsky and Elie Hassenfeld at Give Well, the rationalist movement (inc CFAR)
Kerry then discusses the growth of the EA movement.
Approaches to EA based on evidence (empiricism) and also strong philosophical arguments (esp in the absence of evidence – for instance with Existential Risks or far future scenarios)
How to succeed at EA Global: get help, and make radical life change.

Kerry Vaughan - Be Greedy for the Most Good - EA Global Melbourne 2015 - Effective Altruism 2

Many thanks for watching!
Support SciFuture via Patreon
Please Subscribe to the SciFuture Channel
Science, Technology & the Future website