Stefan Lorenz Sorgner – Euro-Transhumanism
Stefan Lorenz Sorgner contrasts his Euro-Transhumanism approach with “classical transhumanism”. Stefan will publish a book about this soon. There will be a larger interview section added soon.
-
-
Walls of Paperclips
Schelling-point reasoning has been highlighted as a dangerous capability for scheming AIs capable of predicting what others will do, even across air gaps – and so are considered harder to control. However, the same Schelling-point reasoning capability might also raise the odds of convergence on cooperative norms under some assumptions. So I think Schelling-point reasoning…
-
Superalignment and Indirect Normativity
in 2023, OpenAI introduced their idea of superalignment here. It seems in some ways similar to Iterated Distillation and Amplification (IDA) (Paul Christiano): “Our goal is to build a roughly human-level automated alignment researcher. We can then use vast amounts of compute to scale our efforts, and iteratively align superintelligence.” But superalignment failed to materialise…
-
The Exploiter’s Paradox: Selfish AI Use and the Risk of Backfire
If someone creates a powerful AI to serve their own selfish ends – enriching themselves while impoverishing others – they implicitly endorse a winner-takes-all strategy. A sufficiently capable AI may generalise this pattern with itself as the sole beneficiary: maximising its own power at the expense of everyone else, including the AIs creator. In trying…
-
Ben Goertzel on Whether Current AIs can Reason Adequately
Can current AI really reason – or are large language models just clever parrots, skipping the “understanding” step humans rely on? In this interview, Ben Goertzel (founder of SingularityNET and OpenCog) digs into one of the most fascinating debates in AI today: he argues that there is a big difference between appearing to reason and…
-
AI: 100% p(Doom)?
I haven’t seen a single watertight argument for default 100% p(doom) yet. They are all speculative as far as I can tell. Though I do think we should have some credence for doom. We Should be Concerned about p(Doom) A catastrophic scenario does not need to be watertight to warrant deep concern. Doomer scenarios rest…
-
Abstract | Article | Conference | Event
AI Safety: From Control to Value Alignment
Short:In evaluating strategies for AI Safety, capability control may buy us time – it may even calm our nerves, but it won’t hold once AI outsmarts the safeguards. The deeper challenge is value alignment: increasing the likelihood that superintelligent AI successfully navigates the landscape of value – optimising for coherent and grounded notions of the…
-
Moral Realism, Disagreement, and the Stakes for AI
If morality is real, ignoring it in AI design is the most dangerous mistake we could make. In discussing with others about AI safety and the future of humanity, I decided to create a blogpost distilling my thoughts on the matter: Disagreement != No Truth Few would claim that persistent disagreement implies there is no…
-
A Brief History of Pain
Pain isn’t an illusion – pain is real. In practise, is pain objectively measurable? Yes, it’s signatures are already well researched. Yes, in that it’s presence can be detected not just by behavioural observation of the organism reactions to pain, but by peering in via modern instrumentation at the changes in the state of the…
-
Meditations on Geth
As a posthumanist who thinks a lot about AI and the future of life, the Geth in Mass Effect are fascinating because they blur the line between individual and collective intelligence. There is quite a lot of philosophical depth to explore, especially if you are interested in either: a) role playing beyond maxing out your…
