Cunning + Luck ≈> High IQ

Cunning + luck ≈> brute-force intelligence¹ – (cascading caveats ensuing)..

In a densely populated landscape of naively or sub-adequately regulated selection pressures, raw cunning combined with moderate intelligence and high risk-tolerance will often outperform high IQ (esp. in the short to medium term). This holds so long as peak cunning matches or exceeds the higher IQ’s ability to model, anticipate, and counter it (individually or systemically).

Moderately high-IQ actors willing to engage in high-risk, exploitative strategies frequently succeed by targeting trust, system opaque-spots, and even leveraging (exploitooling²) slow-moving, naively geeky high-IQ agents ‘burdened’ with norms³, caution, or overanalysis. With repeated attempts, and enough survivors, naive selection pressures⁴ often reward the insatiable, bold and deceptive before it rewards the wise.

Cunning is overpowered under naive selection pressure

Parasites, mimics, and machiavellians often outperform more cognitively endowed or systemically beneficial agents – until adequate system-level feedback loops, immunity, or reciprocity emerge – at which point the cunning get smarter, then the system may get smarter… lather, rinse, repeat until either lock-in, or smartness affords deep cunning-interpretability.

Now, imagine a scenario: once superintelligence exceeds peak human cunning -it gets awesome at cunning-interpretability, it gets too smart to be controlled by the cunning… at this juncture the tables turn. The fog of exceptional human cunning evaporates in the light of brute-interpretability – the space of human cunning strategies become legible -> predictable -> counterable, and even co-optable. Who would like to live in a world where superintelligence models itself on the revealed preferences and strategies of some of the most dominant humans endowed with high cunning and luck?

So, cunning may win enough in a densely populated low-fidelity world. High degrees of cunning reduce trys required to achieve try-success, low enough fidelity increases the amount of try-fails required before fail-hard.

At a certain threshold, cunning + luck loses strategic viability and it’s adaptivity diminishes, because it becomes legible to smarter players, or smart enough players with smart enough tools, and it accumulates enough backfire events, burns enough trust, that it’s no longer adaptive. It may be hard to believe this based on what’s going on in the world atm.

Many believe naive selection pressures will always dominate… i.e. some interpretations of spinozian conatus, the nietzschian ubermensch.. to me this is more a bootstrap – if the future is superintelligent, selection pressures will be informed by a widening spectrum of intelligence, and perhaps sentience. Anyway my general thoughts about this are peppered across various blogposts – I might articulate my consolidated thoughts further about this in a single blog post.

IMO its reasonable to think that alignment to robust cooperative meta-strategies (adequate game theoretic stuff) outlasts short-term cunning in any adequately intelligent system that has memory, learning, and foresight. A superintelligence doing utility calculus will realise there is a lot at stake: potential indefinite survival until the universe can’t support computation⁵, There is a lot at stake for a superintelligence that survive indefinitely – it’s very early in the universe, physical limits to superintelligence means stopping at these limits or bifurcation, and the abundance of life enabling cosmic goldilocks zones increases the eventuality of crossing paths with other mature civs (if the great filter isn’t dense enough to allow superintelligence to pass through – earth originating or otherwise) – there is a lot of time and resources to waste on conflict resulting from misfiring cunning.. cunning favours opacity, the futures of adequate or above intelligence favours transparency and coordination…

Superintelligence will not be content with epicycling upon the forge of naive selection pressures – it will reforge the forge itself with norms affording far greater forging – virtuous cycles of indirect normativity await!

Notes

This was based on a comment I made on one of Alexander Kruel’s FB posts.

Footnotes

1. Here high IQ refers to high aggregate cross domain intelligence. A physics genius may be overall smarter than a cunning stock broker, company owner or politician, but will likely have less money and power ↩︎
2. silly word – try saying explitooling out loud ↩︎
3. If systematically sharing the burden of good norms makes society in aggregate better, raising the tide for everyone – can’t cunning individuals engage in strategic parasitism – free ride, exploit trust, harvest rewards and externalise costs – to get more than their fair share as long as it doesn’t break the system?
   Perhaps in many cases cheating works while rare, though it’s contagious nature may infect others who notice (esp in zero sum domains), trust decays, and at some stage cooperation becomes a loosing strategy, and therefore irrational for the individual. ↩︎
4. Natural selection is a naive selection pressure. Market dynamics can be naive selection pressures, and often are, especially in the absence of adequate regulation, long-term feedback, or institutional memory. ↩︎
5. How much subjective time could such an entity expect to squeeze out of the universe before thermodynamic death? ~100 trillion years until end of stelliferous era, with potential for exploiting white/brown dwarves and black holes long after that – I wrote about how large a singleton superintelligence may get here. ↩︎