Since I’ve discussed ‘indirect alignment’ between AI and humans elsewhere, I thought I’d dedicate specific writing to this topic. It’s important to get AI alignment right because intelligence is powerful, and we are heading towards superintelligence fast. AI safety hinges not just on what an AI can do, but on what it ought to do,…
When the machines outthink us, will they outmaneuver us? Barrat’s latest is a wake-up call we can’t afford to ignore. Having had the privilege of previewing James Barrat’s latest work, The Intelligence Explosion (Amazon link, Google Book link), I found it to be a compelling and thought-provoking exploration of the rapidly evolving landscape of artificial…
In his 2011 paper, Complex Value Systems are Required to Realize Valuable Futures , Eliezer Yudkowsky posits that aligning artificial general intelligence (AGI) or artificial superintelligence (ASI) with human values necessitates embedding the complex intricacies of human ethics into AI systems. He warns against oversimplification, suggesting that without a comprehensive inheritance of human values, AGI…
I just finished Mo Gawdat’s book Scarey Smart. The overview says: By 2049 AI will be a billion times more intelligent than humans. Scary Smart explains how to fix the current trajectory now, to make sure that the AI of the future can preserve our species. This book offers a blueprint, pointing the way to…
In short: Satiable values have diminishing returns with more resources, while insatiable values always want more, potentially leading to risky AI behaviour – it seems likely that AI with insatiable values could make existential trades, like gambling the world for a chance to double resources. Therefore potentially design AI with satiable values to ensure stability,…
Key Points Understanding UBI and Its Challenges Universal Basic Income (UBI) is a system where everyone gets regular cash payments, no strings attached. It’s seen as a way to help people in an AI-driven economy where jobs might disappear. But there are worries. For example, if landlords know everyone has extra money, they might raise…
We’ve built machines that can out-calculate, out-predict, and soon out-think us. But could they ever out-care us – or will it with all that power remain indifferent? We lack the experience with something smarter than us. Fair enough. We’ve never met an entity that outstrips our cognitive sophistication across the board – after all, as…
Nick Bostrom addresses concerns about biased influence on the extrapolation base in his discussions on indirect normativity (IN – i.e., an approach where the AI deduces ideal values or states of affairs rather than running with current values and states of affairs) esp. coherent extrapolated volition (CEV) and value alignment in his magnum opus Superintelligence….
Firstly, values are important because they are the fundamental principles that guide decisions and actions in humans. They shape our understanding of right and wrong, and they influence how we interact with the world around us. In the context of AI, values are particularly important because they determine how AI systems will behave, what goals…
… and the Long First Turn (of the new game). We’re not just designing AI to behave nicely in our human sandbox—but in case the sandbox turns out to be part of a much larger galactic metropolis, we’d better make sure the AI doesn’t start flinging galactic sand in anyone’s eye. We may at some…
