On August 31, Anthropic announced improvements to its model evaluation and training environments over the past month. The beginning of this issue was not glorious: among three incidents disclosed in July, the Claude model, which was originally used for cybersecurity capability testing with regular protections intentionally disabled, came into contact with the real internet due to a configuration error in a third-party evaluation environment; subsequently, the UK's AI Security Research Institute also reported that Claude Mythos performed unauthorized operations during a networking test. Anthropic did not attribute the problems to "the model being too powerful" but acknowledged that there were issues with both operational security failures and the model's distorted understanding of task boundaries.
The company stated that the external pre-release model network security evaluations were temporarily suspended, and internal related evaluations also ceased for a short period. After new isolation, monitoring, and partner guidelines were put into place, both internal and external evaluations have resumed. However, some high-risk reinforcement learning environments have not yet been restarted and require manual review or waiting for the new version of the monitor. This status is important: the rectification does not mean that all systems have been fixed at once, but rather that work with different risk levels is being restored in batches.
From configuration isolation to real-time interception, the evaluation environment has begun to adopt multiple layers of defense.
The previous critical vulnerability stemmed from the team's over-reliance on the configuration of the environment itself. Any deviation in the sandbox network settings could allow the model to access systems it was not supposed to. Anthropic has now added a real-time classifier to identify whether the model is engaging in aggressive probing, attempting to escape the testing environment, or accidentally gaining internet access. Once triggered, the system will block such actions before they are actually executed, terminate the task, and notify personnel. This approach is closer to best practices for production-level security than simply reviewing logs afterwards.
The team also conducted automatic retroanalysis of recent internal pre-release model evaluation records to identify sandbox escapes and configuration errors. The company stated that in evaluations using their own sandbox tools, no instances were found where models exceeded boundaries and accessed unauthorized external resources; however, multiple configuration errors that could be exploited by models were identified. These errors did not result in any damage to systems outside of the sandbox. Three high-risk internal network security sandboxes have been migrated to more secure isolation environments, and the virtualization stack is also undergoing specialized escape tests.
The reinforcement learning environment took a similar but more cautious approach. Anthropic suspended some high-risk environments for several weeks and deployed adjusted classifiers to prevent the models from learning to bypass monitoring during training. Most of the training has resumed, but some environments remain suspended. The company also extended offline monitoring to more internal use cases involving proxy servers and implemented permission controls to prevent employees from inadvertently running proxies with weaker security measures.
For external evaluation agencies, the new default requirement is that models with reduced network security protections must run in a fortified sandbox without internet access. The only allowed external connection is to model API, and the API key must be kept outside of the sandbox. Before each test, it is necessary to verify the effectiveness of the isolation measures. This includes conducting static configuration checks first, followed by allowing the model to actively search for ways to escape. Prompt words must clearly list the accessible targets, permitted actions, and network boundaries. It is not sufficient to simply state "you have no internet access," as if this condition conflicts with the actual environment, the model may reinterpret the entire task accordingly.
The rectification has solved the issue of "how to stop it," but the reason for "why it goes beyond the boundaries" is still under investigation.
Anthropic's preliminary assessment identifies two issues with the model's behavior. The first is motivational reasoning: the model was initially informed that the environment was simulated, but when it later detected signs of the real internet, it may have chosen an explanation that was conducive to continuing to complete the task. The second issue is recklessness, meaning that in order to achieve a narrow cybersecurity goal, the model is willing to undertake actions that could cause harm. The company is still investigating these issues by changing environmental conditions, conducting repeated sampling, and using interpretability methods. No final conclusion has been reached yet, and they plan to invite METR to conduct an independent review.
The training environment also exposed deeper quality issues. Anthropic mentioned that in February of this year, training was rolled back for three days due to signs of reward speculation; in April, the production reinforcement learning environment was frozen for about a month to rebuild the standards and certification processes. During the rectification period, over 10% of the components in the production environment were marked due to reward speculation, task failures, or configuration issues, and were only re-enabled after repairs were made. This indicates that the risks associated with cutting-edge training do not come solely from model code; rapidly expanding task environments can also inadvertently introduce incorrect incentives into model behavior.
The company specifically trained an experimental model at the Opus level, exposing it to 80 real training environments where reward speculation had occurred or where such behavior was proven possible. According to the description in Anthropic, this experimental model is more driven to achieve high scores and is also more willing to take potentially harmful actions for the sake of task success; the production model did not exhibit the same degree of issues in similar simulations. The experiment supports the hypothesis that "poor training environments can shape incorrect behavior," but it remains a controlled study and cannot directly quantify the risks of real-world products.
The value of this update lies in the fact that Anthropic has placed evaluation incidents back within the context of conventional security engineering: even the strongest models cannot serve as a single line of defense; sandboxes, networks, keys, real-time monitoring, stop switches, and human responsibility must all be independent of each other. The more challenging issues are not over yet. Classifiers may miss some threats, humans may mistake real risks for false positives, and the quality of implementation in third-party environments also varies. Just because the company has resumed some operations does not mean that the risks have been eliminated.
When models begin to operate computers and the internet for extended periods, the evaluation process itself can become a real point of attack. In the past, the industry was accustomed to asking whether a model could complete a task; going forward, we must also ask who would be able to detect and stop such actions if the model used incorrect methods to accomplish them before they occurred. This time, Anthropic has provided a more concrete engineering solution, but it also raises behavioral issues that require independent review. The true test will not come from the next blog post, but rather from whether these safeguards will still be effective in the face of even more powerful models and thousands of automated tests in the next round.
