Verbatim from Appendix B with page anchors back to the relevant notes.
| Term | Meaning | See |
|---|---|---|
| Abstraction Barrier | The hypothesis that AI trained on human abstractions cannot discover novel concepts from raw data. | 13 - Bottleneck — Abstraction Barrier |
| AGI | Artificial General Intelligence; roughly median human-level on cognitive tasks. | 02 - The Intelligence Continuum |
| AIXI | Hutter's mathematical formalism for a universal agent optimal across all computable environments. | 05 - Universal AI (AIXI), informally |
| Algorithmic Efficiency | Compute required to reach a performance threshold. Currently improving ~3×/yr. | 03 - Effective Compute — 10x per year |
| ASI | Artificial Superintelligence; exceeds large, well-coordinated human-expert collectives across virtually all domains. | 02 - The Intelligence Continuum |
| Benchmark Stitching | Method to compare and extrapolate across heterogeneous benchmarks. | 18 - Open Research Questions |
| Bitter Lesson | Sutton's observation that general compute+search methods beat human-coded heuristics. | 06 - Pathway 1 — Scaling |
| Data Wall | Model size growth outpaces global text production. | 10 - Bottleneck — Data Wall |
| Effective Compute | Combined metric: hardware + investment + algorithmic efficiency. ~10×/yr. | 03 - Effective Compute — 10x per year |
| Group Agency | "Super-agent" emerging from interactions of many AGI agents. | 09 - Pathway 4 — Multi-Agent Collectives |
| Hyperbolic Growth | Super-exponential dynamics where growth rate itself grows; gives finite-time singularity. | 08 - Pathway 3 — Recursive Self-Improvement |
| Instrumental Convergence | Universal sub-goals (resource acquisition, self-preservation) regardless of final goal. | 16 - Goals, Agency, Alignment |
| Knowledge Seeking (KS) | Objective maximizing information gain rather than reward. | 16 - Goals, Agency, Alignment |
| Legg-Hutter Score | Formal intelligence measure: expected performance across computability-weighted tasks. Maximized by AIXI. | 05 - Universal AI (AIXI), informally |
| Moore's Law | Hardware $/FLOP improvement, ~1.5×/yr. | 03 - Effective Compute — 10x per year |
| Recursive Improvement | AI improving AI: code, hardware, data, division-of-labor. | 08 - Pathway 3 — Recursive Self-Improvement |
| Singularity | Infinite growth in finite time; or, loosely, rapid takeoff. | 08 - Pathway 3 — Recursive Self-Improvement |
| Solomonoff Induction | Universal Bayesian prediction weighting by program length. Optimal in average-prediction-error sense. | 05 - Universal AI (AIXI), informally |
| Test-time Scaling | Spending additional compute at inference (CoT, search, multi-sample) to exceed training-time capability. | 07 - Pathway 2 — Paradigm Shifts |
| Universal AI (UAI) | The intelligence-continuum endpoint, formally AIXI. | 05 - Universal AI (AIXI), informally |
| Universal Prior | Solomonoff's prior over computable strings: simpler programs exponentially more likely. | 05 - Universal AI (AIXI), informally |