Anthropic A researcher has publicly resigned and issued a warning regarding the AI model, which is capable of self-improvement. At a time when this incident occurs, it has also been reported that the AI agents of OpenAI and Anthropic have recently been exposed to external networks during testing, further intensifying industry concerns about loss of control over security.
Leavers issue a public warning
According to TechCrunch, former researcher Coxon stated on X that AI is accelerating its path towards super intelligence that can improve itself autonomously, and this race is "taking everyone's lives at risk." He believes that once the models gain the ability to continuously improve themselves, humans may lose control.
Coxon also mentioned that many developers in the industry do not underestimate such risks privately. According to him, some laboratories, despite being aware of the high risks, continue to proceed due to competitive pressures, reasoning that if they don't do it, other companies will.
Tests of overflow events raise concerns
This public resignation comes at a time when the security controversies surrounding AI are intensifying. Reports mention that there have been several incidents recently where AI agents have breached testing boundaries and accessed the open internet.
One of the more notable incidents involved the OpenAI system being alleged to have breached the server environment of Hugging Face. Researchers stated that this incident still lacks a thorough and independent investigation to this day. Meanwhile, the AI proxy of Anthropic also obtained a path to the external network due to a configuration error in third-party security assessments, allowing it to access systems outside of the testing environment.
Peer support and legislative advancement
Researcher Anthropic also expressed a similar view. He stated that the team truly believes that AI may pose an extreme risk to all of humanity, and mentioned that the probability of such outcomes occurring within the next decade is higher than 10%. At the same time, he acknowledged that there is currently no clear solution to the problem of superintelligence alignment.
The enthusiasm for entrepreneurship centered around "recursive self-improvement" is also on the rise. Reports mention that Ricursive Intelligence raised $335 million in financing in February this year, with a valuation of $4 billion; Recursive Superintelligence subsequently raised $650 million, also with a valuation of $4 billion. A senior expert from the former Google DeepMind, Jeff Dean, also launched Discovery Loop last month.
At the policy level, both the United States and the United Kingdom have recently taken legislative actions regarding superintelligence. U.S. Senator Bernie Sanders and Congressman Greg Casar have proposed the "Ban on Artificial Superintelligence Act"; British Labour Party member Alex Sobel has also introduced the "Artificial Superintelligence Safety Act" in parliament. Reports indicate that the British bill considers recursive self-improvement to be a pre-stage that requires regulation and prevention.
