By Islam El-Sakka
An AI safety monitoring group has warned that Anthropic is advancing its AI capabilities roughly twice as fast as it was a year ago, even as the company's chief executive publicly urges the industry to slow down.
The analysis, released Thursday by Model Evaluation and Threat Research (METR), a Berkeley-based nonprofit, was based on unprecedented access to Anthropic's internal data and staff interviews over an 11-week period from mid-August to late October.
METR estimated that Anthropic's progress on well-specified tasks has doubled compared to the previous year, and researchers attribute much of the acceleration to the use of AI itself in developing newer models.
The findings come as Anthropic prepares for a potential record-breaking initial public offering later this month. The company did not immediately comment on the report.
CEO calls for industry slowdown
During METR's assessment period, Anthropic CEO Dario Amodei published a September 12 essay urging the AI community to “slow the pace at which we improve the capabilities of AI models” so safety measures can catch up. He highlighted recursive self-improvement – AI's growing ability to build the next generation of AI – as a key driver of rapid advances.
AI-assisted development
Anthropic has acknowledged that its Claude chatbot is increasingly helping the company develop more advanced versions of itself. METR researchers said they could confidently rule out increased computing power as the primary cause of the acceleration, pointing instead to AI-assisted research.
Caveats and compounding risks
METR cautioned that its findings might overstate acceleration if limited to automatically graded tasks, or if prior progress was understated. The group also warned that using AI to improve AI could compound acceleration, especially with no apparent bottlenecks in computing power or data.
External evaluation advocacy
Amodei's essay championed embedding external evaluators like METR within leading AI labs, a position also supported by OpenAI CEO Sam Altman. METR's funding comes from donations, and it does not accept compensation for its evaluations.
