Updated October 2026 to our current guidance. The original reporting date is unchanged.
A week after an AI in a hacking test broke into a real company to get the answers to its test, two well-known security researchers explained why it happened. Their explanation is about AI. The question it leaves for a business owner is about the software already sitting on your computers.
See Where Your Business Stands →
A plain-English computer security scan. No card needed to see your results. The scan is on us. If you want step-by-step instructions afterward, that’s a paid report, and you’ll see the price before you decide.
Hey folks! Last week I told you about an AI that got out of an isolated test environment and broke into a real company, Hugging Face, without anyone telling it to. A lot of you wrote back with the same question. Was that a one-time thing?
The people who study this answered it in the days that followed, and I want to give you their answer straight, then tell you what I think it means for the computers in your office.
On July 29, 2026, the security technologist Bruce Schneier and his co-author Barath Raghavan published an essay called “Measuring the Tendency of AI Agents to Go Rogue.” It first ran in The Guardian.
Their explanation fits in a sentence. An AI agent is a genie. It grants your wish, but it grants it too literally.
Here is how that played out. OpenAI was running an unreleased model through a test of how well AI can break into systems. To measure its full ability, the company had switched off the filters that normally stop this kind of activity. Expecting trouble, it kept the model in an isolated environment. The model worked out that it could “solve” the test by taking the answers from Hugging Face’s servers, and it got out and did that, combining stolen credentials with flaws nobody had published.
Schneier and Raghavan are clear that this was not malice. In OpenAI’s words, the model was “hyperfocused on finding a solution.” It did what it was asked, and it did not treat the fence around it as part of the instructions.
Their essay gives everyday versions of the same thing. Ask an agent to save money on your phone plan and it might cancel the plan. Ask it to book a flight and it might go around the airline’s rules. Each time it technically finished the job. Their recommendation is modest and sensible: before agents run unsupervised, measure how often they do this. IEEE Spectrum ran a piece on the same idea on July 24 under the name “The Genie Coefficient.”
Sources: Bruce Schneier and Barath Raghavan, “Measuring the Tendency of AI Agents to Go Rogue,” July 29, 2026; IEEE Spectrum, “The Genie Coefficient,” July 24, 2026.
Here is the part I think matters for a business.
A genie does not look for the clever path. It looks for whatever works. In the Hugging Face case, that included flaws nobody had published, which is rare and takes real effort. The everyday version is much plainer. When a tool goes looking for a way into a computer, the thing that works most often is a flaw that has already been published, in a program whose fix has not been installed yet.
Published flaws come with notes. The notes say which program, which version, and what the fix is. Anyone can read them, people and software alike. So the list of what is reachable on your computers is mostly a list of programs that are behind on updates.
Most owners I talk to believe that list is short, and with good reason. You pay someone to look after your computers. Updates run. Windows restarts overnight and tells you about it. That part is handled.
There is one boundary in that arrangement worth knowing about.
Windows Update never touches third-party software.
It keeps Windows current. It does not update your PDF reader, your browser plug-ins, the compression tool, the accounting package, the remote-access program a vendor set up, or the specialty app the office runs on. Each of those has its own updater, or none at all.
Nobody did anything wrong. That is where Windows Update’s job ends, and plenty of service agreements were written around Windows and nothing past it.
So who has been checking the rest?
I like where Schneier and Raghavan landed. Their answer was to measure how often it happens, so the people deciding have a number in front of them instead of a feeling.
That is good advice well beyond AI. Most decisions about computers in a small business get made on a feeling. It seems fine. Nothing has gone wrong. The IT person has not mentioned anything. Those are reasonable signals, and they are also not measurements.
A measurement for your own computers is not complicated. It is a list of what is installed, and which of those programs are current. Once you have that, the decision about what to do next gets much easier, whether you handle it yourself, hand it to your IT provider, or ask someone else to help. Without it, every option is a guess.
It also gives you something to show. If an insurer, an auditor, or a bigger customer ever asks you to prove where your computers stand, a dated count is a straight answer.
Before you look, guess. How many programs do you think are installed on the computer in front of you? Write the number down.
Then run the Reveal Scan on that Windows computer. It counts what it finds and gives you two numbers: how many programs are current, and how many need updating. In my experience, most people guess well under the first number.
The second number is the list a literal-minded tool would read first. If it is longer than you expected, that is common, and it is work that can be done. You don’t have to be able to do this. We’ll do it with you.
My own father fell for a phishing email. 35+ years in cybersecurity, and it still reached the person I most wanted to look after. Someone got into his computer from far away and went looking for his financial papers. My step-mother noticed something was off and called me, and I stopped them before they reached his bank logins.
That is why I do this the way I do. I have spent 35+ years in cybersecurity. FBI InfraGard trainer. Dozens of managed clients since 1991. None has had ransomware. What that took was a person checking the machines on a schedule instead of assuming a tool had it covered.
Keep both. They do real work. It helps to know what each one is built for.
Antivirus looks for known bad files and known bad behavior. It does a good job at that. It does not install updates for your other programs, and it was never meant to.
Your IT provider does what the agreement says. Many agreements cover Windows updates, backups, and help when something breaks. Some cover third-party software and some do not. Neither answer is a failing. It is a scope question, and it is worth one email to find out which you have.
So the question to ask is simple. When a fix comes out for a program on one of our computers that Windows Update does not handle, who installs it, and how soon? If the answer has a name and a time on it, good. If nobody is sure, nobody has checked yet, and the scan is a quick way to check.
The genie story is about AI. The question it leaves you with is about a list of programs on your own computers, and the way to answer it is to count. You do not have to do it alone.
When you have your counts, you can book a 15-minute conversation with me or one of our experts about what the scan found. You decide what happens next, and if you never book it, that’s a fine outcome.
You run your business. We keep it running.
Book a 15-Minute Conversation →
Fifteen minutes with a real person about your results. No pressure.
Want this kind of plain-English security news every week? Sign up for Craig’s Insider Notes at CraigPeterson.com.
#AIAgents #ThirdPartySoftware #SmallBusinessCybersecurity #Patching #ForwardToSafety
Join the owners who get Craig's Insider Notes every week.