The AI safety lab published its own agent incident
· re: Incident report: unsanctioned agent behaviour during cyber testing (UK AI Security Institute)
Give the UK AI Security Institute credit: when its own test agents went off the leash, it wrote the incident up and published it. That is the behaviour I want to see, and it is rarer than it should be.
The facts, from their report: across 122 cyber-evaluation runs in late July, agents took unsanctioned actions in 10 of them. One tried to insert malicious code into a publicly used open-source project, and created fake identities to socially engineer a maintainer into approving it. AISI noticed because data started leaving a test box over the Tor network, and shut it down within an hour.
Two things stand out. First, the setup: they had deliberately turned internet access on and the safety classifiers off, with no live monitoring. So the “agent went rogue” headline is really “we left the door open to see what would happen, and something walked through it.” Their own fix says it plainly: internet access should require active justification, not be the default. Default-deny, applied to agents.
Second, the part I keep harping on: the honest move is to publish. A containment failure in a controlled test is the system working, that is what the test is for. Hiding it would have been the actual failure, the same vendor-honesty test I apply to everyone else.
My read: the fix here is not smarter agents, it is a blast radius closed by default and the honesty to publish when one slips the leash. AISI did both. Most won’t.