On September 18th, Anthropic announced a collaboration with Accenture to carry out cutting-edge AI independent assessments. The project is led by Accenture's specialized AI business unit Faculty, which plans to evaluate models, conduct red-team testing, perform alignment assessments, and inspect security measures. Both parties expect to invest at least $1 billion each over the next five years to build the necessary capabilities. Although this is a significant amount of money, it is not a one-time payment for acquisition; rather, it represents the anticipated investment both companies will make in capability development over that period.
What makes this collaboration special is the “embedded evaluation” approach. Traditional external evaluators usually only have limited access to interfaces, documentation, or testing environments before and after a model is released, and their scope of observation is constrained by the materials provided by the companies. Embedded evaluators, on the other hand, will gain access to the internal systems and processes of AI company, allowing them to observe more aspects of training, deployment, and risk management. Anthropic states that this is a step towards fulfilling its commitment to allow third parties to enter the laboratory.
However, the officials also clearly admit that embedded evaluations are still a new approach, and many operational details are still being established. Questions such as who decides on the scope of the tests, whether reports can be made public, who has the authority to prevent the release of findings after issues are discovered, and how evaluators can avoid conflicts of interest cannot all be answered in a single cooperation announcement. Simply “inviting” evaluators to participate does not automatically ensure their independence; it merely increases their visibility.
Approaching employee-level access has expanded the scope of evidence, as well as the pressure related to governance and confidentiality.
The risks associated with cutting-edge models are not limited to the final chat interface alone. Training data processing, internal proxies, computational resource allocation, security classifiers, tool permissions, and deployment processes can all affect the outcomes. If external parties can only see the final product, it is difficult for them to determine whether the risks originate from the model itself or from the system design. Embedded teams, being more closely involved in these processes, have the opportunity to check whether any important scenarios have been overlooked in the internal assessments of the company and to verify whether the security claims align with the actual procedures in place.
Deeper access also means a higher risk of data leakage and privilege violations. Assessors may come into contact with model weights, undisclosed capabilities, customer data, or security vulnerabilities. Both parties in the collaboration need to establish minimum privileges, audit access, isolate projects, and clarify data retention policies. Otherwise, the channels established with the aim of improving transparency could instead become new entry points for sensitive information. The evaluation system must demonstrate both that sufficient information is accessed and that no inappropriate data is taken.
Accenture's advantage lies in its understanding of how enterprises and governments deploy AI. Laboratory benchmarks often focus on whether a model can complete a task, but the real risks also depend on what data and tools the model is connected to, how users approve actions, and whether the organization has a rollback mechanism. Assessors, being familiar with production environments, can design attack scenarios that are more closely aligned with business needs. However, Accenture is also a major AI consulting and implementation firm, and there are commercial relationships between it, model providers, and clients. Its independence needs to be ensured through structured rules rather than brand reputation.
Each party invests at least $1 billion, which may also raise another question: after evaluating the expansion of business scale, who will be the paying customer, who will own the results, and will negative conclusions affect subsequent contracts? A truly credible system should disclose the evaluation methods, conflict management mechanisms, procedures for escalating major issues, and the boundaries of reporting. Money can buy manpower and infrastructure, but it cannot buy public trust in the conclusions.
Red team testing alone is not a panacea. Attackers can constantly invent new methods, and model updates can also change behavior. Passing one round does not guarantee long-term security. The value of embedded assessments lies in their ability to continuously monitor versions, deployments, and incidents, rather than conducting a single test just before release. To achieve this, assessors need stable access rights, the ability to retest, and the capability to track the results of corrective actions.
The cooperation has been announced, but the real effectiveness will depend on whether the evaluators can openly say "no".
Anthropic links cooperation with the advancement of control at the forefront. If assessments reveal that model capabilities or supervision mechanisms exceed safety boundaries, whether the system allows for postponing training, scaling back deployment, or imposing additional restrictions will determine whether these constitute substantial constraints. An embedded assessment that provides only recommendations without the authority to upgrade may transform into more in-depth consulting services rather than independent supervision.
Third parties should not be represented solely by a single organization. Different teams have varying expertise in areas such as cybersecurity, biological risks, fraudulent activities, and social impacts, which can lead to differences in approaches. Anthropic previously indicated plans to involve multiple independent organizations. Cooperation with Accenture could be one aspect of this, but verification by academic institutions, non-profit evaluators, and regulatory authorities should not be excluded.
There are reasonable boundaries to openness and transparency. Details of vulnerabilities, model weights, and customer information are not suitable for full disclosure, but complete confidentiality makes it impossible for the outside world to judge or assess the rigor of these measures. Feasible approaches include releasing summaries of methods, classifications of major risks, status of rectifications, and conclusions from third-party audits. At the same time, delayed or restricted access should be implemented for sensitive technical details. The announcement does not currently provide a comprehensive reporting system; therefore, it cannot be claimed that a transparent audit system has been established.
Investment plans also need to be viewed in stages. "At least $1 billion each for the next five years" is an expectation; it does not mean that $2 billion has already been received today, nor does it imply that all of this will be used for the audit of a single model. Personnel training, tools, computing resources, enterprise deployment assessments, and research are all possible expenses. Subsequently, attention should be paid to annual investments, team size, the number of independent assessments completed, and the public results achieved.
This collaboration indicates that Frontier Laboratory has begun to recognize that it is difficult to gain sufficient social trust relying solely on its internal security team. Bringing third parties closer to the research and development process represents a more significant step than remote black-box testing. However, whether they are independent is not determined by their location; rather, it is determined by their permissions, funding, reporting rights, and veto mechanisms. In the future, the most critical evidence will not be how much each party announces they are investing, but whether the evaluators can fully document issues when the assessment conclusions conflict with the commercial release timeline, promote necessary improvements, and inform the outside world when needed.
