Capability progress might be deliberately capped by: - Public backlash from visible AI harms or accidents - Government regulation (EU AI Act, compute-threshold licensing, mandatory eval) - Voluntary moratoria from labs - Liability regimes shifting after high-profile failures - Geopolitical coordination (unlikely but possible)
This is the only bottleneck on the list that is a human choice rather than a technical limit.
The paper cites work on "normal accidents" in AI infrastructure (Perrow tradition extended to AI by Hadan et al., Maas) — the claim that failures in tightly-coupled socio-technical systems originate in organizational decisions, not isolated technical faults. A high-profile near-miss could shift public licence for AI deployment dramatically:
"Large, visible accidents or credible near-miss events could shift public preferences, liability regimes and regulatory thresholds in ways that render major further scaling steps towards ASI politically, legally or commercially infeasible, even where they remain technically and economically achievable."
This is the practical case for slowdown: technical possibility ≠ political possibility.
The paper invokes Dafoe's "military-economic adaptationism" / "Anarchy as Architect" framework:
"Sustained inter-group rivalry systematically favours the development and adoption of competitiveness-enhancing technologies, irrespective of their implications for human welfare."
In other words: even if individual states want to slow down, the international competition selects for those who don't. Unilateral slowdown loses; multilateral coordination is historically rare.
The interesting framing:
"Governance is not purely a brake on AI progress, but can also serve as a steering mechanism that shapes the direction and quality of development."
Examples of steering (not just stopping): - Compute-threshold licensing (EU AI Act) - Mandatory pre-deployment evals - International norm-setting (Bletchley Declaration) - Liability frameworks - Responsible-scaling policies from labs themselves
This is the one bottleneck that's most a function of choices nobody has made yet. The paper basically says: "this could matter a lot, we don't know how much it will matter, and it depends on factors outside our technical model."