In case you don’t follow this sort of thing as closely as some, the headline refers to a recent incident in which two of OpenAI’s leading-edge AI models gained unauthorized access to – in other words, hacked – an open-source model known as Hugging Face and extracted information from it (there are also reports that it used a second service as a staging point for the attack). As if that’s not bad enough, in order to accomplish the hack, the OpenAI models gained unauthorized access to the internet without anyone knowing, until a significant amount of time had elapsed. The attack was first reported by Hugging Face on July 16th as an unauthorized intrusion into its open-source model’s database by what it described as an unidentified agent – one that the company said was clearly part of an autonomous AI system. Here’s how Hugging Face described it:
“Earlier this week, we detected and responded to an intrusion into part of our production infrastructure. This one was different from anything we had handled before in one important way: it was driven, end to end, by an autonomous AI agent system – and we detected and dissected it largely with AI of our own. We identified unauthorized access to a limited set of internal datasets and to several credentials used by our services. The intrusion started where AI platforms are uniquely exposed: the data-processing pipeline. A malicious dataset abused two code-execution paths in our dataset processing to run code. From there, the actor escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters.”
According to the company, the campaign was run by an agent framework whose identity it could not determine, and that agent or web of agents executed “thousands of individual actions across a swarm of short-lived sandboxes with self-migrating command-and-control staged on public services.” In case you’re wondering where the name Hugging Face came from, the company is a New York-based entity founded by several French entrepreneurs to develop an AI-powered chat service for kids, and got its name from an emoji known as “hugging face.” The company open-sourced the chat software and then created a database of machine-learning tools and libraries for others to use as a resource for working with AI. Hugging Face also developed its own large-language model similar to ChatGPT or Anthropic’s Claude, known as BLOOM. Ironically, perhaps, the company used AI to detect the intrusion, saying its attack-detection software used LLM-based “triage over security telemetry” to spot anomalies.
Note: This is a version of my Torment Nexus newsletter, which I send out via Ghost, the open-source publishing platform. You can see other issues and sign up here.
Continue reading “Should we be worried about the Hugging Face hack?”





















