The earliest evidence of emergent symbolic reasoning in LLMs was behavioural – it is typically traced back to the discovery of In-Context Learning (ICL) in GPT-3 (2020). While earlier models like GPT-2 showed glimpses of this behaviour, GPT-3 was the first to demonstrate a “sudden spike” in the ability to follow abstract patterns from just…
AI doesn’t need a moustache-twirling villain to go wrong – it just needs the wrong metaethics in an unforgiving game. In this post I argue that: (1) many human moral norms are shaped by competitive, “Molochian” pressures; (2) if we give AI a purely constructivist / anti-realist metaethic, it will inherit and amplify those pressures;…
In the high-stakes world of Artificial Intelligence research, there is perhaps no metric more contentious than p(Doom), the probability assigned to catastrophic outcomes – including human extinction -resulting from advanced AI. Remarkably, the individuals seen as best positioned to understand this risk in profound disagreement. While many of the field’s pioneers, such as Turing Award…
In a blinded Moral Turing Test, people rated GPT-4’s moral answers higher than humans’ across most measures – yet they could still tell it was AI. Are we priming the public to over-trust machine morality? Interview with Dr Eyal Aharoni, Ph.D. – Associate Professor of Psychology, Philosophy, and Neuroscience Georgia State University Attributions toward artificial…
Halloween special: a scary futures tier list that is spooky in theme & sobering in content. This tier list isn’t scientific, it isn’t the final say, it doesn’t exhaustively include all doomsday risks – its a bit of a gimmick, and a fun intuition pump. Anders Sandberg is a neuroscientist and futurist well known for…
Stefan Lorenz Sorgner contrasts his Euro-Transhumanism approach with “classical transhumanism”. Stefan will publish a book about this soon. There will be a larger interview section added soon.
Schelling-point reasoning has been highlighted as a dangerous capability for scheming AIs capable of predicting what others will do, even across air gaps – and so are considered harder to control. However, the same Schelling-point reasoning capability might also raise the odds of convergence on cooperative norms under some assumptions. So I think Schelling-point reasoning…
in 2023, OpenAI introduced their idea of superalignment here. It seems in some ways similar to Iterated Distillation and Amplification (IDA) (Paul Christiano): “Our goal is to build a roughly human-level automated alignment researcher. We can then use vast amounts of compute to scale our efforts, and iteratively align superintelligence.” But superalignment failed to materialise…
If someone creates a powerful AI to serve their own selfish ends – enriching themselves while impoverishing others – they implicitly endorse a winner-takes-all strategy. A sufficiently capable AI may generalise this pattern with itself as the sole beneficiary: maximising its own power at the expense of everyone else, including the AIs creator. In trying…
Can current AI really reason – or are large language models just clever parrots, skipping the “understanding” step humans rely on? In this interview, Ben Goertzel (founder of SingularityNET and OpenCog) digs into one of the most fascinating debates in AI today: he argues that there is a big difference between appearing to reason and…