(OSV News) — A recent OpenAI security incident that saw a test model go rogue may be a sign of more to come, and a call to double down on preventing future attacks, a Catholic expert on artificial intelligence ethics told OSV News.
“Cases like these are going to keep popping up, and each one of those should motivate us to do more,” said Brian Patrick Green, director of technology ethics at the Markkula Center for Applied Ethics at Santa Clara University.
On July 16, Hugging Face — a company and platform for open-source AI model development, with some proprietary elements — said it had “detected and responded to an intrusion into part of our production infrastructure.”
But, said the company, the incident “was different from anything we had handled before in one important way” — namely, “it was driven, end to end, by an autonomous AI agent system.”
Findings of OpenAI investigation of incident
On July 21, OpenAI — the San Francisco-based AI research and development giant behind Chat GPT — published a statement saying an investigation had shown “this particular incident was driven by a combination of OpenAI models,” including the company’s GPT-5.6 Sol and “an even more capable pre-release model.”
OpenAI explained the models were being “internally tested” on a benchmark — a standardized test used to evaluate AI performance on a given task — for cyber capabilities.
In the process, the models escaped the secure testing environment, or sandbox, to gain access to Hugging Face’s system.
Green explained to OSV News that the incident was “a case of it (the OpenAI model group) just plain outsmarting” the test and doing “something unexpected.”
“You give it a goal, and it says, ‘Oh, I know how I can get that goal. I’ll steal the answers from Hugging Face’ — which is what it did,” said Green.
‘Storage base’ of different models, data sets
Since Hugging Face has “a huge, huge storage base of different models and data sets,” said Green, “it makes perfect sense for this OpenAI model (group) to have said, ‘Well, where do I turn to find the answers? Hugging Face looks like the kind of place that could have this model.’ So they hacked into them and found it.”
Green warned that such breaches are “going to likely start happening more and more.”
In a sense, he said, the OpenAI model group was “just trying to do what it was told” to accomplish its goal, with the hack appearing to it as “the most efficient solution to the problem.”
He said the incident was not necessarily an example of what’s called emergent misalignment, where AI acts counter to human values, “because it was never aligned” with those values “in the first place.”
‘Whole thing is very significant’
The “whole thing is very significant,” Green added, noting that “there are a lot of softer targets than Hugging Face that are likely to be hit and have bad things happen.”
He said the OpenAI incident took place, providentially, within the context of a test, and that it was “significant enough for people to notice it.”
“It’s important that these are news stories,” Green said.
Given Pope Leo XIV’s recently released encyclical “Magnifica Humanitas,” which urged AI development to remain centered in God-given human dignity and the principles of Catholic social teaching, the faithful have a role to play in shaping this technology, Green said.
Need ‘adequate AI models on your side’
“For example, if you’re a Catholic who’s working in the security industry, then this is something you should work on,” he said. “If you’re a Catholic who’s working on AI models, then this is something that you should be aware of. If you’re a Catholic who’s in charge of protecting something, then it becomes particularly important for you to have adequate AI models on your side.”
Even those who are not directly involved in AI development and implementation can help, he said.
“Contact your congressperson and say, ‘Hey, I want better cybersecurity,'” said Green. “We’re super reliant on computers, and all of those systems are reliant on being secure. And if all of a sudden you make everything unsecure, then there are so many different kinds of disasters that could happen.”
Gina Christian is a multimedia reporter for OSV News. Follow her on X @GinaJesseReina.
