Joscha Bach – GPT-3: Is AI Deepfaking Understanding?

Joscha Bach on GPT-3, achieving AGI, machine understanding and lots more!


Discussion points:
02:40 What’s missing in AI atm? Unified coherent model of reality
04:14 AI systems like GPT-3 behave as if they understand – what’s missing?
08:35 Symbol grounding – does GPT-3 have it?
09:35 GPT-3 for music generation, GPT-3 for image generation, GPT-3 for video generation
11:13 GPT-3 temperature parameter. Strange output?
13:09 GPT-3 a powerful tool for idea generation
14:05 GPT-3 as a tool for writing code. Will GPT-3 spawn a singularity?
16:32 Increasing GPT-3 input context may have a high impact
16:59 Identifying grammatical structure & language
19:46 What is the GPT-3 transformer network doing?
21:26 GPT-3 uses brute force, not zero-shot learning, humans do ZSL
22:15 Extending the GPT-3 token context space. Current Context = Working Memory. Humans with smaller current contexts integrate concepts over long time-spans
24:07 GPT-3 can’t write a good novel
25:09 GPT-3 needs to become sensitive to multi-modal sense data – video, audio, text etc
26:00 GPT-3 a universal chat-bot – conversations with God & Johann Wolfgang von Goethe
30:14 What does understanding mean? Does it have gradients (i.e. from primitive to high level)?
32:19 (correlation vs causation) What is causation? Does GPT-3 understand causation? Does GPT-3 do causation?
38:06 Deep-faking understanding
40:06 The metaphor of the Golem applied to civ
42:33 GPT-3 fine with a person in the loop. Big danger in a system which fakes understanding. Deep-faking intelligible explanations.
44:32 GPT-3 babbling at the level of non-experts
45:14 Our civilization lacks sentience – it can’t plan ahead
46:20 Would GTP-3 (a hopfield network) improve dramatically if it could consume 1 to 5 trillion parameters?
47:24 GPT3: scaling up a simple idea. Clever hacks to formulate the inputs
47:41 Google GShard with 600 billion input parameters – Amazon may be doing something similar – future experiments
49:12 Ideal grounding in machines
51:13 We live inside a story we generate about the world – no reason why GPT-3 can’t be extended to do this
52:56 Tracking the real world
54:51 MicroPsi
57:25 What is computationalism? What is it’s relationship to mathematics?
59:30 Stateless systems vs step by step Computation – Godel, Turing, the halting problem & the notion of truth
1:00:30 Truth independent from the process used to determine truth. Constraining truth that which can be computed on finite state machines
1:03:54 Infinities can’t describe a consistent reality without contradictions
1:06:04 Stevan Harnad’s understanding of computation
1:08:32 Causation / answering ‘why’ questions
1:11:12 Causation through brute forcing correlation
1:13:22 Deep learning vs shallow learning
1:14:56 Brute forcing current deep learning algorithms on a Matrioshka brain – would it wake up?
1:15:38 What is sentience? Could a plant be sentient? Are eco-systems sentient?
1:19:56 Software/OS as spirit – spiritualism vs superstition. Empirically informed spiritualism
1:23:53 Can we build AI that shares our purposes?
1:26:31 Is the cell the ultimate computronium? The purpose of control is to harness complexity
1:31:29 Intelligent design
1:33:09 Category learning & categorical perception: Models – parameters constrain each other
1:35:06 Surprise minimization & hidden states; abstraction & continuous features – predicting dynamics of parts that can be both controlled & not controlled, by changing the parts that can be controlled. Categories are a way of talking about hidden states.
1:37:29 ‘Category’ is a useful concept – gradients are often hard to compute – so compressing away gradients to focus on signals (categories) when needed
1:38:19 Scientific / decision tree thinking vs grounded common sense reasoning
1:40:00 Wisdom/common sense vs understanding. Common sense, tribal biases & group insanity. Self preservation, dunbar numbers
1:44:10 Is g factor & understanding two sides of the same coin? What is intelligence?
1:47:07 General intelligence as the result of control problems so general they require agents to become sentient
1:47:47 Solving the Turing test: asking the AI to explain intelligence. If response is an intelligible & testable implementation plan then it passes?
1:49:18 The term ‘general intelligence’ inherits it’s essence from behavioral psychology; a behaviorist black box approach to measuring capability
1:52:15 How we perceive color – natural synesthesia & induced synesthesia
1:56:37 The g factor vs understanding
1:59:24 Understanding as a mechanism to achieve goals
2:01:42 The end of science?
2:03:54 Exciting currently untestable theories/ideas (that may be testable by science once we develop the precise enough instruments). Can fundamental physics be solved by computational physics?
2:07:14 Quantum computing. Deeper substrates of the universe that runs more efficiently than the particle level of the universe?
2:10:05 The Fermi paradox
2:12:19 Existence, death and identity construction

Exciting progress in Artificial Intelligence – Joscha Bach

Joscha Bach discusses progress made in AI so far, what’s missing in AI, and the conceptual progress needed to achieve the grand goals of AI.
Discussion points:
0:07 What is intelligence? Intelligence as the ability to be effective over a wide range of environments
0:37 Intelligence vs smartness – interesting models vs intelligent behavior
1:08 Models vs behaviors – i.e. Deepmind – solving goals over a wide range of environments
1:44 Starting from a blank slate – how does an AI see an Atari Game compared to a human? Pac Man analogy
3:31 Getting the narrative right as well as the details
3:54 Media fear mongering about AI
4:43 Progress in AI – how revolutionary are the ideas behind the AI that led to commercial success? There is a need for more conceptual progress in AI
5:04 Mental representations require probabilistic algorithms – to make further progress we probably need different means of functional approximation
5:33 Many of the new theories in AI are currently not deployed – we can assume a tremendous shift in every day use of technology in the future because of this
6:07 It’s an exciting time to be an AI researcher

 

Principles of Synthetic Intelligence - Joscha BachJoscha Bach, Ph.D. is an AI researcher who worked and published about cognitive architectures, mental representation, emotion, social modeling, and multi-agent systems. He earned his Ph.D. in cognitive science from the University of Osnabrück, Germany, and has built computational models of motivated decision making, perception, categorization, and concept-formation. He is especially interested in the philosophy of AI and in the augmentation of the human mind.

Joscha has taught computer science, AI, and cognitive science at the Humboldt-University of Berlin and the Institute for Cognitive Science at Osnabrück. His book “Principles of Synthetic Intelligence” (Oxford University Press) is available on amazon.

 

Ethical Progress, AI & the Ultimate Utility Function – Joscha Bach

Joscha Bach on ethical progress, and AI – it’s fascinating to think ‘What’s the ultimate utility function?’ – should we seek the answer in our evolved motivations?

Discussion points:
0:07 Future directions in ethical progress
1:13 Pain and suffering – concern for things we cannot regulate or change
1:50 Reward signals – we should only get them for things we can regulate
2:42 As soon as minds become mutable ethics dramatically changes – an artificial mind may be like a Zen master on steroids
2:53 The ultimate utility function – how can we maximize the neg-entropy in this universe?
3:29 Our evolved motives don’t align well to this ultimate utility function
4:10 Systems which only maximize what they can consume – humans are like yeast

 

Principles of Synthetic Intelligence - Joscha BachJoscha Bach, Ph.D. is an AI researcher who worked and published about cognitive architectures, mental representation, emotion, social modeling, and multi-agent systems. He earned his Ph.D. in cognitive science from the University of Osnabrück, Germany, and has built computational models of motivated decision making, perception, categorization, and concept-formation. He is especially interested in the philosophy of AI and in the augmentation of the human mind.

Joscha has taught computer science, AI, and cognitive science at the Humboldt-University of Berlin and the Institute for Cognitive Science at Osnabrück. His book “Principles of Synthetic Intelligence” (Oxford University Press) is available on amazon.

 

 

The Grand Challenge of Developing Friendly Artificial Intelligence – Joscha Bach

Joscha Bach discusses problems with achieving AI alignment, the current discourse around AI, and inefficiencies of human cognition & communication.

Discussion points:
0:08 The AI alignment problem
0:42 Asimov’s Laws: Problems with giving AI (rules) to follow – it’s a form of slavery
1:12 The current discourse around AI
2:52 Ethics – where do they come from?
3:27 Human constraints don’t apply to AI
4:12 Human communication problems vs AI – communication costs between minds is much larger than within minds
4:57 AI can change it’s preferences

Principles of Synthetic Intelligence - Joscha BachJoscha Bach, Ph.D. is an AI researcher who worked and published about cognitive architectures, mental representation, emotion, social modeling, and multi-agent systems. He earned his Ph.D. in cognitive science from the University of Osnabrück, Germany, and has built computational models of motivated decision making, perception, categorization, and concept-formation. He is especially interested in the philosophy of AI and in the augmentation of the human mind.

Joscha has taught computer science, AI, and cognitive science at the Humboldt-University of Berlin and the Institute for Cognitive Science at Osnabrück. His book “Principles of Synthetic Intelligence” (Oxford University Press) is available on amazon.

Cognitive Biases & In-Group Convergences – Joscha Bach

Joscha Bach discusses biases in group think.

Discussion points:
– In-group convergence: thinking in true & false vs right & wrong
– The group mind may be more stupid than the smartest individuals in the group

Principles of Synthetic Intelligence - Joscha BachJoscha Bach, Ph.D. is an AI researcher who worked and published about cognitive architectures, mental representation, emotion, social modeling, and multi-agent systems. He earned his Ph.D. in cognitive science from the University of Osnabrück, Germany, and has built computational models of motivated decision making, perception, categorization, and concept-formation. He is especially interested in the philosophy of AI and in the augmentation of the human mind.

Joscha has taught computer science, AI, and cognitive science at the Humboldt-University of Berlin and the Institute for Cognitive Science at Osnabrück. His book “Principles of Synthetic Intelligence” (Oxford University Press) is available on amazon.

AI, Consciousness, Science, Art & Understanding – Joscha Bach

Here Joscha Bach discusses consciousness, it’s relationship to qualia and whether an AI or a utility maximizer would do with it.

What is consciousness? “I think under certain circumstances being conscious is an important part of a mind; it’s a model of a model of a model basically. What it means is our mind (our new cortex) produces this dream that we take to be the world based on the sensory data – so it’s basically a hallucination that predicts what next hits your retina – that’s the world. Out there, we don’t know what this is.. The universe is some kind of weird pattern generator with some quantum properties. And this pattern generator throws patterns at us, and we try to find regularity in them – and the hidden layers of this neural network amount to latent variables that are colors people sounds ideas and so on.. And this is the world that we subjectively inhabit – that’s the world that we find meaningful.”

… “I find theories [about consciousness] that make you feel good very suspicious. If there is something that is like my preferred outcome for emotional reasons, I should be realising that I have a confirmation bias towards this – and that truth is a very brutal vector”..

OUTLINE:
0:07 Consciousness and it’s importance
0:47 Phenomenal content
1:43 Consciousness and attention
2:30 When AI becomes conscious
2:57 Mary’s Room – the Knowledge Argument, art, science & understanding
4:07 What is understanding? What is truth?
4:49 What interests an artist? Art as a communicative exercise
5:48 Thomas Nagel: What is it like to be a bat?
6:19 Feel good theories
7:01 Raw feels or no? Why did nature endow us with raw feels?
8:29 What is qualia, and is it important?
9:49 Insight addiction & the aesthetics of information
10:52 Would a utility maximizer care about qualia?

BIO:
Principles of Synthetic Intelligence - Joscha BachJoscha Bach, Ph.D. is an AI researcher who worked and published about cognitive architectures, mental representation, emotion, social modeling, and multi-agent systems. He earned his Ph.D. in cognitive science from the University of Osnabrück, Germany, and has built computational models of motivated decision making, perception, categorization, and concept-formation. He is especially interested in the philosophy of AI and in the augmentation of the human mind.

Joscha has taught computer science, AI, and cognitive science at the Humboldt-University of Berlin and the Institute for Cognitive Science at Osnabrück. His book “Principles of Synthetic Intelligence” (Oxford University Press) is available on amazon.

Professor Peter Doherty – COVID19 Pandemic: Research & Action

Fascinating interview with Nobel Laureate Professor Peter Doherty on the COVID-19 pandemic, the nature of COVID-19, where it came from, it’s similarities to influenza other coronaviruses (i.e. SARS, MERS), how infectivity works, what we as citizens can do to stay safe and help minimise the burden on our health systems, achieving rapid responses to pandemics, a strategic infection strategy (variolation) in lieu of an actual vaccination, rejuvenating the thymus to help boost our immunity as we age, computer modelling of disease, and what we can hope to have learned from this ordeal after the pandemic is over.

Audio:

Youtube (video choppy, audio fine):

Peter’s book ‘Pandemics: What everyone needs to know’ can be found at Dymocks and Amazon.

 

Biography

Peter Charles Doherty, AC FRS FMedSci is an Australian veterinary surgeon and researcher in the field of medicine. He received the Albert Lasker Award for Basic Medical Research in 1995, the Nobel Prize in Physiology or Medicine jointly with Rolf M. Zinkernagel in 1996 and was named Australian of the Year in 1997. In the Australia Day Honours of 1997, he was named a Companion of the Order of Australia for his work with Zinkernagel. He is also a National Trust Australian Living Treasure. In 2009 as part of the Q150 celebrations, Doherty’s immune system research was announced as one of the Q150 Icons of Queensland for its role as an iconic “innovation and invention”.

https://en.wikipedia.org/wiki/Peter_C._Doherty

https://www.doherty.edu.au
https://www.nobelprize.org/prizes/medicine/1996/doherty/biographical/

#COVID_19 #Coronavirus #Pandemics #COVID19

Gero, Singapore AI startup bags $2.2m to create a drug that helps extend human life

Congrats to Gero for the $2.2m of funding to create a drug that helps extend human life!

I did two interviews with Gero in 2019 at Undoing Aging – here, with Peter Fedichev on Quantifying Aging in Large Scale Human Studies:

And here with Ksenia Tsvetkova on Data Driven Longevity:

Doris Yu at Tech In Asia said:

The company observed that as population growth slows down, the average lifespan increases. For example, there will only be 250 million people older than 65 by the end of the decade in China. Countries like Singapore, meanwhile, are not able to attract enough migrants to help offset the aging population.

Gero then wants to provide a medical solution to help extend healthspan as well as improve the overall well-being and productivity of its future customers.

It’s trying to do so by collecting medical and genetic data via a repository of biological samples and creating a database of blood samples collected throughout the last 15 years of patients’ lives. Its proprietary AI platform was able to determine a type of protein that could help with rejuvenation if blocked or removed.

What problem is it solving? “Aging is the most important single risk factor behind the incidence of chronic diseases and death. […] We are ready to slow down – if not reverse – aging with experimental therapies,” Peter Fedichev, co-founder and CEO of Gero, told Tech in Asia.

Explorebit.io wrote:

Gero, a Singapore-based company that develops new drugs for ageing and other complicated disorders using its proprietary developed artificial intelligence (AI) platform, secured $2.2m in Series A funding.

The round, which brought total capital raised since founding to over $7.5m, was led by Bulba Ventures with participation from previous investors and serial entrepreneurs in the fields of pharmaceuticals, IT, and AI. The co-founder of Bulba Ventures Yury Melnichek joined Gero’s Board of Directors. The company will use the funds to further develop its platform.

Led by founder Peter Fedichev, Gero provides an AI-based platform for analyzing clinical and genetic data to identify treatments for some of the most complicated diseases, such as chronic aging-related diseases, mental disorders, and others. The company’s experts used large datasets of medical and genetic information from hundreds of thousands of people acquired via biobanks and created a proprietary database of blood samples collected throughout the last 15 years of the patients’ lives.

Using this data, the platform determined the protein that circulates in people’s blood whose removal or blockage should lead to rejuvenation. Subsequent experiments at National University of Singapore involved aged animals and demonstrated mortality delay (life-extension) and functional improvements after a single experimental treatment. In the future, this new drug could enable patients to recover after a stroke and could help cancer patients in their fight against accelerated ageing resulting from chemotherapy.

The platform is currently also being utilized to develop drugs in other areas: for example, the group’s efforts to find potential therapies for COVID-19, including those that could reduce mortality from complications related to ageing, has already attracted a great deal of attention from large pharmaceutical companies and leading global media organizations.

Posthumanism – Pramod Nayar

Interview with Pramod K. Nayar on #posthumanism ‘as both a material condition and a developing philosophical-ethical project in the age of cloning, gene engineering, organ transplants and implants’. The book ‘Posthumanism’ by Pramod Nayar: https://amzn.to/2OQEA8z Rise of the posthumanities article: https://bit.ly/32Q67Pm
This time, I decided trying to itemize the interview so you can find sections via the time signature links:
0:00 Intro / What got Pramod interested in posthuman studies?
04:16 Defining the terms – what is posthumanism? Cultural framing of natural vs unnatural. Posthumanism is not just bodily or mental enhancement, but involves changing the relationship between humans, non-human lifeforms, technology and non-living matter. Displacement of anthropocentrism. 
08:01 Anthropocentric biases inherited from enlightenment humanist thinking and human exceptionalism. The formation of the transhumanist declaration with part of it focusing on the human with point 7 of the declaration focusing on the well-being of all sentience. The important question of empathy – not limiting it to the human species. The issue of empathy being a good lunching pad for further conversations between the transhumanists and the posthumanists. https://humanityplus.org/philosophy/t… 
11:10 Difficulties in getting everyone to agree on cultural values. Is a utopian ideal posthumanist/transhumanist society possible? 
13:25 Collective societies, hive minds, borganisms. Distributed cognition, the extended mind hypothesis, cognitive assemblages, traditions of knowledge sharing. 
16:58 Does the humanities need some form of reconfiguration to shift it towards something beyond the human? Rejecting some of the value systems that enlightenment humanism claimed to be universal. Julian Savulescu’s work on moral enhancement 
20:58 Colonialism – what is it? 
21:57 Aspects of enlightenment humanism that the critical posthumanists don’t agree with. But some believe the poshumanists to be enlightenment haters that reject rationality – is this accurate? 
24:33 Trying to achieve agreement on shared human values – is vulnerability rather than dignity a usable concept that different groups can agree with? 
26:37 The idea of the monster – people’s fear of what they don’t understand. Thinking past disgust responses to new wearable technologies and more radical bodily enhancements. 
29:45 The future of posthuman morphology and posthuman rights – how might emerging means of upgrading our bodies / minds interfere with rights or help us re-evaluate rights? 
33:42 Personhood beyond the human
35:11 Should we uplift non-human animals? Animals as moral patients becoming moral actors through uplifting? Also once Superintelligent AI is developed, should it uplift us? The question of agency and aspiration – what are appropriate aspirations for different life forms? Species enhancement and Ian Hacking’s idea of ‘Making up people’ – classification and how people come to inhabit the identities that exist at various points in history, or in different environments. https://www.lrb.co.uk/the-paper/v28/n… 
38:10 Measuring happiness – David Pearce’s idea of eliminating suffering and increasing happiness through advanced technology. What does it mean to have welfare or to flourish? Should we institutionalise wellbeing, a gross domestic happiness, world happiness index? 
40:27 Anders Sandberg asks: Transhumanism and posthumanism often do not get along – transhumanism commonly wears its enlightenment roots on its sleeve, and posthumanism often spends more time criticising the current situation than suggesting an out of it. Yet there is no fundamental reason both perspectives could not simultaneously get what they want: a post-human posthumanist concept of humanity and its post-natural environment seem entirely possible. What is Nayar’s perspective on this win-win vision? 
44:14 The postmodern play of endless difference and relativism – what is the good and bad of postmodernism on posthumanist thinking? 
47:16 What does postmodernism have to offer both posthumanism and transhumanism? 
49:17 Thomas Kuhn’s idea of paradigm changes in science happening funeral by funeral. 
58:58 – How has the idea of the singularity influenced transhumanist and posthumanist thinking? Shift’s in perspectives to help us ask the right questions in science, engineering and ethics in order to achieve a better future society. 
1:01:55 – What AI is good and bad at today. Correlational thinking vs causative thinking. Filling the gaps as to what’s required to achieve ‘machine understanding’. 
1:03:26 – Influential literature on the idea of the posthuman – especially that which can help us think about difference and ‘the other’ (or the non-human) 

Anders Sandberg – Freeman Dyson, Galactic Megastructures, Physical Eschatology & the Fermi Paradox

Many of you know the sad news that theoretical physicist & mathematician Freeman Dyson has passed away, so in celebration of his life and achievements, Anders Sandberg (Future of Humanity Institute) discusses Freeman Dyson’s influence on himself and others – How might advanced alien civilizations develop (and indeed perhaps our own)?

We discuss strategies for harvesting energy – star engulfing Dyson Spheres or Swarms, black hole swallowing tungsten dyson super-swarms and other galactic megastructures, we also discuss Kardashev scale civilizations (Kardashev was another great mind who we lost recently), reversible computing, birthing ideal universes to live in, Meinong’s jungle, ‘eschatological engineering’, the aestivation hypothesis, and how all this may inform strategies for thinking about the Fermi Paradox and what this might suggest about the likelihood of our civilization avoiding oblivion.  though Anders is more optimistic than some about our chances of survival..

 

 

Anders Sandberg (Future of Humanity Institute in Oxford ) is a seminal transhumanist thinker from way back who has contributed a vast amount of mind blowing material to futurology & philosophy in general. https://en.wikipedia.org/wiki/Anders_Sandberg

Happy Future Day (march 1st) : http://future-day.org

Freeman Dyson: https://en.wikipedia.org/wiki/Freeman_Dyson
Dyson Sphere: https://en.wikipedia.org/wiki/Dyson_sphere
Aestivation Hypothesis: https://en.wikipedia.org/wiki/Aestivation_hypothesis
Reversible Computing: https://en.wikipedia.org/wiki/Reversible_computing
Kardashev Scales: https://en.wikipedia.org/wiki/Kardashev_scale
Nikolai Kardashev: https://en.wikipedia.org/wiki/Nikolai_Kardashev

[Audio version] [Video here]