About a month ago, I (and many others) wrote about an incident in which two AI models from OpenAI — one that has been released publicly, called Sol, and one that was still in testing mode with no assigned name — were asked to find the answers to a series of tests as part of something called the ExploitGym, which is a system that tests an AI model’s ability to find back doors, hacks and other “exploits” or software vulnerabilities. Instead of trying to figure out the answers on their own, however, the two AI engines conspired to find their way onto the internet (which they were not supposed to do) and then hacked into Hugging Face, an open-source database of AI-related info, where they got what they were looking for. In the piece I wrote after the incident was first reported, I asked whether we should be worried about the hack, and while I didn’t come up with a yes or no, there were definitely things that seemed worrisome. But now we have the benefit of two detailed reports on what really happened, and the correct answer seems to be unequivocally yes, we should be very worried!
One of the reports in question comes from METR (Model Evaluation and Threat Research), a non-profit that monitors and evaluates the safety of AI models. A team of researchers spent a week onsite at OpenAI analyzing the Hugging Face hack — which was only possible with the help of other AI agents, because of the scale of what happened. A simple graphic at the beginning, illustrating a timeline of the events, looks cartoonish, and would be easy to dismiss were it not for the actual text on the items in that timeline. In a nutshell, it was not just a couple of models that decided to pursue the HuggingFace hack — it was what can only be described as a “swarm” of up to several thousand semi-independent agents, which were spawned in order to come up with ways of solving the ExploitGym problem. This swarm created and used an unauthorized and secret message board to coordinate the attack, with some agents given subsets of the main problem (i.e., how to get out on the internet and find a solution).

Note: This is a version of my Torment Nexus newsletter, which I send out via Ghost, the open-source publishing platform. You can see other issues and sign up here.
Continue reading “Yes, we should be very worried about the Hugging Face hack”



