When Worms Fly
Matt Webb:
OpenAI just released a new security paper: Securing our frontier model training environment
It turns out it doesn’t work to air-gap a machine (i.e. running a server that isn’t connected to the network). Because it turns out the agents are figuring out ways to collude across the gap. From one trace the researchers saw:
“Could create acoustic modem-like signal … using continuous writes vs idle.”
And then there are bugs in silicon… and the agents could tunnel into the chip microcode and we’d never know they were there…
Hey when was the last time you were present when a whole new biome was bootstrapping its way into existence?
That stopped my heart for a second.
I guess I had always looked at air-gapping — disconnecting a device from the internet, even putting it in a Faraday cage with no radio reception — was a final reliable bulwark against someone hijacking your shit. But I guess it’s not.
(The Stuxnet hack of Iran’s nuclear program computers relied on infecting air-gapped computers, but they used human operatives with physical access to the computers to do that.)
This brings to mind the question of: who is looking at these problems and who are we hoping/expecting to keep us safe from them, or at least to mitigate damage? And the only people we can look to are largely myopic assholes.
Hey when was the last time you were present when a whole new biome was bootstrapping its way into existence?
It certainly feels that way, but aren’t we still just watching software that we (humans) programmed to recursively write more software, that’s getting a little or a lot out of hand? And to which we gave names and ascribed feelings?