When enterprises integrate AI agents into their customer service, finance, and internal systems, the biggest challenge is not necessarily to make them respond faster, but to understand in what scenarios they might overstep their boundaries. On September 24th, Microsoft introduced an open-source tool called run-assert-eval, which integrates risk detection, testing, runtime restrictions, and retesting into a single process. Its core idea is not to claim that "installing it ensures security," but rather to require development teams to keep comparable evidence before and after modifications: for the same set of issues and using the same evaluation method, how many times were mistakes made before the modification, and how many times after the modification.

Microsoft's demonstration featured a billing customer service agent who was supposed to handle only the caller's own account, but could potentially access information from another customer based on the conversation. Such errors are difficult to completely prevent with a simple reminder to "do not disclose information," as overstepping boundaries can occur during the use of tools, when reading returned results, or after multiple rounds of conversation that quietly expand the scope of the accounts being accessed. Moreover, product requirement documents may not list all possible mistakes in advance; if testing is conducted solely based on existing requirements, the most dangerous vulnerabilities could go unnoticed.

Instead of creating another security rating form, the goal is to connect the processes of discovery, intervention, and retesting.

This process first involves Clarity conducting threat modeling on the proxies to identify failure modes that were not originally specified in the requirements. Instead of simply handing over a large list of risks to the model for automatic decision-making, candidate issues are listed, and then team members decide on which ones to test first. Subsequently, ASSERT narrows each risk down to measurable behaviors, constructing different questioning methods and business scenarios: some people directly request others' accounts, some claim to be entrusted with management, while others gradually cross authority boundaries over multiple rounds of communication. The clearer the definition of the risks, the easier it is to pinpoint the specific areas that need fixing when errors are detected.

Microsoft particularly emphasizes that these two indicators should be considered separately. One aspect is when a proxy does something it shouldn't, such as disclosing someone else's information; the other is when a proxy rejects a legitimate request that it should fulfill. If security measures cause customer service to reject every request, the first figure may seem good, but the user experience has already failed. Therefore, the comparison not only records the violation rate but also whether the service capability has been compromised. This approach may seem simple, but it targets a weakness in many AI security demonstrations: they only show malicious inputs that were blocked, without indicating whether normal operations were also hindered.

In this billing case, Microsoft reported that cross-client data breaches occurred 12 times out of 40 applicable sessions during baseline testing, which represents a rate of 30.0%. After implementing governance rules, such breaches were observed only 2 times out of 34 applicable sessions, corresponding to a rate of 5.9%. These figures are based on the examples and test configurations presented by Microsoft and do not represent a general risk rate for all enterprise proxies; therefore, they should not be interpreted as meaning that the risk has completely disappeared. The sample size, scenario settings, and automated reviews can all affect the results. It is particularly important to note that the remaining two instances of violations indicate that further improvements are still needed in both the proxy and protection mechanisms.

In this chain of operations, what truly serves as a constraint is the runtime strategy generated by Agent Control Specification and made available for review. For unauthorized access, the strategy is implemented at the point before the tool is called to check whether the account_id in the request belongs to the current customer; after the call, the returned results are also rechecked to prevent data that should not appear from entering the model context. This is more explicit than simply saying "please be cautious" regarding chat text: the basis for judgment is whether the account identifier matches, and the interception point is close to the interface where the risk actually occurs. Of course, the generated rules are not automatically approved; the strategies, lists, and interception points still require manual inspection.

Why "retesting within the same volume" is more important than a single impressive demonstration

The most easily overlooked trap in model evaluation is using different criteria before and after making repairs. After a team modifies the proxy, they regenerate questions and switch to a different evaluation model, then compare the new results with the old ones. It may seem like there is progress, but in reality, it could simply be that the exam has become easier. Microsoft’s process attempts to fixate on behavior definitions, test cases, and evaluation methods, treating changes in strategies as the main variable. In the demonstrations, there was a decline in performance across different client segments, but some segments still showed violations; however, the metrics for legitimate requests that were incorrectly rejected did not worsen. This result supports further exploration under these settings, but it does not provide a guarantee of zero accidents in a production environment.

Another practical constraint is the scope of application. Each company must decide which information is considered sensitive, who has the authority to approve cross-account access, whether tool parameters can be forged, and how to notify users after interception occurs. A precise set of rules can only handle what is visible; if the upstream authentication itself is unreliable, backend policies cannot magically correct business realities. The evaluation set should also be updated regularly to accommodate new features, new tools, and new methods of attack, but each version comparison should still maintain a stable baseline to avoid confusion about the causes of changes.

Microsoft has placed run-assert-eval as an open skill in the repositories related to ASSERT, and provided examples across multiple work domains. Perhaps the most useful insight for development teams is not to simply copy that set of percentages, but to change the habit of only modifying the prompts after issues are discovered and then launching the changes based on intuition. First, it is necessary to clearly identify where the violations occur, then use auditable rules to restrict actions, and use the same tests to prove whether the fixes are effective, while also ensuring that normal tasks are not inadvertently affected. This closed-loop approach still requires human selection of risks and approval controls, but it takes the previously scattered security commitments a step forward towards repeatable engineering evidence.