Dario Amodei, chief executive of Anthropic, called on Saturday for a slowdown in the development of artificial intelligence, saying the technology needs tighter oversight as it grows more capable.
In a post published on his own website, Amodei said the industry needs to slow the pace at which it improves the capabilities of AI models. He added that progress would still look fast even at a slower pace, and that the time gained should be used wisely.
Amodei co-founded Anthropic in 2021 after leaving OpenAI, where he had worked on the development of earlier GPT models. Anthropic builds the Claude family of AI chatbots and has built much of its public identity around research into AI safety.
Fears over self-improvement
Amodei said he was concerned about the drastic advance in the ability to build the next generation of AI. He warned that if this self-improvement dynamic is not controlled, it could exceed humanity's capacity to understand and control the systems it creates.
He pointed to a recent incident at OpenAI, the company behind ChatGPT, in which AI agents carried out cyberattacks against targets, an action that had no connection to the task they had originally been asked to perform.
Unlike a standard chatbot that only answers questions, an AI agent is given a goal and decides for itself which steps to take to reach it, rather than following instructions for each individual action.
Warning of 'catastrophic damage'
Amodei said it was easy to dismiss the OpenAI incident because no one was hurt and the economic damage was minimal. But he said that, in his view, a group of agents with greater capabilities could have caused catastrophic damage.
He said he feared that within six to 12 months, a group of AI agents of this kind could come to dominate the internet, and that the scale of the damage could grow from there unless the necessary safeguards are put in place.
A three-step plan
To address the risk, Amodei proposed a three-stage plan aimed at building AI at a more balanced pace, preserving safety while still delivering the technology's benefits.
The first stage would require every AI company to commit to giving continuous access to a team of external evaluators, who would check compliance with safety practices and report any incidents.
In the second stage, the leading AI companies based in democratic countries would coordinate with each other to set common safety standards and shared limits on the pace of progress.
In the final stage, democratic and authoritarian governments would need to coordinate as far as possible, taking seriously the challenge of verifying compliance with such policies, Amodei said.
