Hinton, Bengio, and lab scientists warn an intelligence explosion could close the policy window
Two pioneers of modern AI and senior researchers at OpenAI and Anthropic on Monday warned governments to prepare for an "intelligence explosion," The Guardian's Dan Milmo reports.
Nobel laureate Geoffrey Hinton and Canadian computer scientist Yoshua Bengio co-authored the paper with more than 20 others. Named contributors include Anthropic co-founder Jack Clark and OpenAI chief scientist Jakub Pachocki. The paper is titled "What if automating AI R&D triggers an intelligence explosion."
Axios adds Microsoft chief scientific officer Eric Horvitz and Meta VP of AI Research Dawn Song among the listed authors.
An intelligence explosion, as they define it, is a dramatic AI-driven acceleration of AI progress that compresses advances that would otherwise take years into months or less. The focus is recursive self-improvement: systems that improve themselves in a loop, and automated AI research and development that runs with little human intervention. Once AI systems reach expert-level AI R&D, the authors write, a single developer could run a workforce equivalent to millions of top human researchers.
"An intelligence explosion could be the most consequential technological development in human history," the paper says, "threatening human control over AI systems, and severely eroding checks on power within and between states, companies, and branches of government." Another line lands harder for anyone waiting on Congress: "Once an intelligence explosion begins, the window for action may close."
The authors say impacts remain uncertain. Axios notes constraints that could slow the spiral: compute limits, hard automation problems, diminishing returns, and long training runs.
They also say productivity gains from AI R&D automation have not yet reached the threshold that would trigger an explosion, even as gains from newer systems look likely to approach it. Research projects that would take humans months could be fully automated by AI by 2028, the paper projects.
Inside the labs, the automation is already measurable. Anthropic says AI now produces 80% of its own code. OpenAI uses autonomous AI agents in areas such as training of new models.
Axios cites one scenario in which a year's worth of advances could arrive in a matter of weeks.
The policy ask is concrete. Authors want transparent progress reports on AI-related R&D, including independent auditors embedded in the work. They want tools that can constrain breakneck development, and preparation to adapt when acceleration arrives.
Axios lists related ideas such as mandatory incident reporting, independent evaluators, safety requirements for testing powerful systems, and ways to pause certain AI work.
That package sits beside a weekend of incident tallies and a morning harden-the-defenses speech from London. The same shops racing agent-assisted research are putting their names on a memo that says the rulebook may stop mattering once the research loop no longer needs them.
The signatures arrived from inside the companies building toward the threshold.