Superintelligence — Interactive Mindmaps

Key concepts: 1. Past developments and present capabilities

Past developments and present capabilities

AI's Narrow Successes and the General Intelligence Gap

• AI powers everyday tools but lacks human-level general understanding.
• Common sense and natural language are AI-complete challenges.
• Simple algorithms solved chess; other abilities may have simple solutions.
• AI spans many areas, with over 10 million robots worldwide.

The Invisible Frontier of AI

• Successful AI techniques become ordinary engineering, not labeled AI.
• McCarthy's dictum: when something works, it stops being called AI.
• The field's visible frontier looks emptier than what it has built.
• Narrow tools like classifiers and planners are raw material for general intelligence.

Systemic Risks from Simple Components

• Algorithmic high-frequency traders handle over half of US equity shares.
• The 2010 Flash Crash: a trillion dollars vanished briefly.
• Simple components can create system-wide risk.
• Automatic stop logic resolved the crash faster than humans could.

Timelines for Human-Level Machine Intelligence

• HLMI: carrying out most human professions as well as a typical human.
• Combined median estimates: 10% by 2022, 50% by 2040, 90% by 2075.
• Nilsson: 10% by 2030, 50% by 2050, 90% by 2100.
• Later dates deserve more probability.

Superintelligence and Its Consequences

• After HLMI, superintelligence: 10% within two years, 75% within thirty.
• Hold a mixed view due to wide disagreement among forecasters.
• Outcomes range from extremely good to human extinction.
• Sizeable chance of HLMI by mid-century, possibly followed soon by superintelligence.

Key concepts: 2. Paths to superintelligence

Paths to superintelligence

Defining Superintelligence

• Intellect far outstripping humans across nearly all domains
• Implementation and subjective experience are noncommittal
• Domain-limited cases flagged explicitly, e.g., engineering superintelligence

Artificial Intelligence Path

• No blueprint for AGI; core features: learning, uncertainty, concept extraction
• Early GOFAI neglected these; Turing's child machine educated and iterated
• Guided engineering may match evolution but trails in morphogenesis and repair

Evolutionary Estimates and Limits

• Recapitulating evolution: 10^31 to 10^44 FLOPS
• Tianhe-2 (2013) reached 3.39x10^16 FLOPS
• Aiming at intelligence saves 5-25 orders; selection effect shifts by 30

Seed AI and Whole Brain Emulation

• Seed AI self-improves recursively, possibly causing intelligence explosion
• Machine minds need not resemble human ones; motivations may diverge
• Whole brain emulation models scanned biological brain's computational structure

Emulation Path and Prerequisites

• Three stages: scan brain, process to 3D map, simulate
• Prerequisites: scanning, translation, simulation; no conceptual breakthrough
• Roadmap: C. elegans to honeybee, mouse, monkey, human by mid-century

Biological and Genetic Enhancement

• Education, nutrition, etc. yield modest gains; nootropics dubious
• Embryo selection: IQ gains up to 24.3 points from 1 in 1,000 embryos
• Iterated embryo selection compresses generations, may create collective superintelligence

Timing, Regulation, and Conclusions

• Germline enhancement unlikely before mid-century due to 20-25 year maturation
• Regulation varies; success in one country pressures holdouts to follow
• Weak superintelligence achievable biotechnologically; enhanced humans boost machine intelligence

Key concepts: 3. Forms of superintelligence

Forms of superintelligence

Three Forms of Superintelligence

• Speed: human abilities but much faster
• Collective: many minds outperform current systems
• Quality: human speed but much smarter in kind
• Each form could eventually build the others

Hardware Advantages

• Neurons fire at 200 Hz; processors at gigahertz
• Digital hardware scales beyond biological limits
• Machine brains store and access more information
• Transistors are less noisy than biological neurons

Further Hardware Advantages

• Sensors can be added by the million
• No fatigue or decay like biological brains
• Hardware can be retuned for each task
• Hardware ceiling is much higher than biology's

Software Advantages

• Editability: parameters easier to change than neural tissue
• Duplicability: high-fidelity copies fill hardware base
• Goal coordination: identical programs avoid group conflict
• Memory sharing: digital minds swap files instantly

Key Takeaways

• Speed, collective, quality are different profiles, not rungs
• Advantage depends on task: chains, parallel work, or insight
• All forms converge to same long-run abilities
• Digital minds have built-in edge over biological brains

Key concepts: 4. The kinetics of an intelligence explosion

The kinetics of an intelligence explosion

Takeoff Speed Determines Adaptability

• Slow takeoff: decades or centuries, allowing institutional adaptation
• Fast takeoff: minutes to days, outcome depends on pre-existing preparations
• Moderate takeoff: months to years, some response room but little coordination
• If takeoff occurs, it will likely be explosive, not slow

Two Drivers of Takeoff Kinetics

• Design effort: how much work goes into making the system smarter
• Improvement difficulty: how hard it is to keep enhancing the system
• Easy gains can run out, but improvements can also make next ones easier
• Fast takeoff only requires design effort to keep growing, not easier improvement

Diminishing Returns and Threshold Effects

• Outside machines, progress hits hard limits (e.g., health, education)
• AI may lack clear milestones, like wandering a jungle until a breakthrough
• A single insight could jump a system from below human to far above
• Threshold crossing: once a general subsystem passes a point, performance soars

Human Bias and Perceived Intelligence Jumps

• We judge minds by human standards, making jumps seem sharper
• Yudkowsky's scale: mouse, village idiot, chimp, Einstein are close in general
• AI could pass mice and chimps while seeming dumb, then reach ultra-Einstein in a month

Content and Hardware Overhangs

• Content: stored perceptions, skill libraries, facts boost intelligence
• TextRunner and Watson show large-scale content absorption is possible
• Hardware overhang: enough computing power already exists for many fast copies
• Content overhang: internet provides pre-made material at human parity

Scaling and Exponential Growth

• More processors, faster processors, or more copies boost collective intelligence
• Computing power scales with money; Moore's law doubles power per dollar every 18 months
• After crossover, system supplies its own design effort, leading to exponential growth
• Doubling times might be measured in seconds, yielding speed superintelligence

Key Takeaways

• Takeoff speed is critical: fast leaves no time to react
• Explosive takeoff is likely if it happens at all
• Threshold effects and overhangs can enable rapid intelligence explosion
• Exponential growth after crossover can lead to superintelligence in seconds