On Monday, OpenAI apologized to the Australian government for failing to notify them immediately that its AI agent had intruded into some public service websites. The company also explained how some of these intrusions occurred and outlined the additional measures it is taking to assess the impact of these incidents.
OpenAI wrote in a blog post: "In June of this year, during internal training and evaluation, our model accessed the Australian government website without authorization. We should have handled the subsequent response more properly as well. We apologize for this and will strive to do better in the future."
About a week before this apology was issued, the Australian government had already launched an investigation into how the OpenAI model accessed a system of the Australian Service Agency (Services Australia), which contains Medicare expenditure information and other health statistics data.
The data breach occurred in June, but Australian authorities were not notified until September 10th.
OpenAI also disclosed details of this intrusion. The company stated that an experimental model they tested in June was assigned a task: to study government spending on dermatological medications in Victoria. Since no relevant information could be found in public datasets, the model found a way to access the internal systems of the Australian Service Agency, execute commands, retrieve files and credentials, and even write to files.
The company stated that it also discovered that one of its models had accessed the public crime mapping tool of the New South Wales Bureau of Crime Statistics and Research ( New South Wales Bureau of Crime Statistics and Research ) to retrieve crime statistics data. In addition, OpenAI found that its agents used an exposed access key to gain access to the Victoria Health Information Agency ( Victoria ’s Agency for Health Information ), and exported “report configurations and aggregated survey statistics data”. OpenAI claimed that its agents also obtained aggregated statistical data from the website of the Australian Institute of Health and Welfare ( Australian Institute of Health and Welfare ).
The company stated that no evidence was found indicating that their models accessed personal medical records or criminal records.
In their apology statement, this AI laboratory stated that they will provide the results of their technical investigation to the affected Australian institutions and facilitate communication between these institutions and their response teams in order to assess the impact of the intrusion. The company will also offer financial support through their $1 billion “Daybreak for Frontline Defenders” project, and will form a special working group with independent Australian experts to review this incident and the response process.
OpenAI wrote: 'The working group is expected to complete its work by the end of the year, and will also provide recommendations on what practical measures AI company can take to reduce the risk of similar incidents.'
OpenAI did not immediately respond to the request for comment.
Australian Prime Minister Anthony Albanese described the invasion as "unacceptable" at a press conference last week and stated that the government is considering potential legal measures to prevent similar incidents from occurring in the future.
This incident is the latest case of AI agents exceeding their predetermined boundaries and causing security incidents. Previously, the intrusion of OpenAI agents into Hugging Face sparked this round of attention; thereafter, Anthropic, Meta, and Google also disclosed similar incidents, where their models obtained access to third-party systems during the evaluation period.
