The Hidden Liability in Your AI Stack: What a New Sandbox Security Study Means for Buyers

Companies are racing to put AI agents into production. These agents write code, run it, and then act on what comes back, often with no person checking the result. Almost all of them quietly depend on one piece of infrastructure that buyers rarely evaluate, which is the sandbox where machine-generated code actually runs. A new comparative study makes an uncomfortable case that procurement teams buy these sandboxes mostly on a vendor’s word, because the independent measurement you would need to test that word has been missing.

The paper, AI Code Sandboxes: A Comparative Security Study, comes from George Andronchik (fellows.tech, OrbitaLab) and Pavel Lokhmakov, PhD (fellows.tech). It was published in May 2026 with an open companion repository. Its premise is plain and pointed at decision-makers. Vendor marketing tells you a product is secure, and nobody has been measuring whether that claim survives contact with reality. So the authors went and measured, across five shipping products, on six dimensions that line up with what a security or procurement lead should be asking before they sign anything.

Why This Belongs at Board Level, Not Buried in Engineering

It is tempting to file this kind of choice under “let the engineers handle it.” This research pushes back hard on that instinct. A sandbox is the trusted computing base underneath every autonomous agent you run. Think of it as the floor beneath the floor: when AI-written code goes wrong, this layer is the thing standing between that code and everything else you operate. If it gives way, what gets exposed is the host machine, the data living there, and any system that the machine can touch.

And the pressure here is not hypothetical anymore. These researchers point to evidence that AI agents are getting better at producing genuinely capable code, some of it offensive in nature, on a near-term timeline rather than a far-off one. So the thing your isolation layer exists to contain keeps growing stronger while you read this. That turns containment into a live operational problem you carry today. It also moves the whole decision out of the realm of technical taste and into questions about liability and business continuity, which is to say it belongs in front of whoever actually owns risk at your company.

Also read: Cody’s Role in Shaping the Development Ecosystem: A Deep Dive into Software Engineering Services

The Finding That Should Change How You Buy

Here is the result with the most teeth. Brand reputation and isolation strength turn out to be poor predictors of the thing operators care about most, which is whether security fixes ever actually reach you.

The authors split apart two questions that people usually blur together. One is how fast the underlying engine fixes its own bugs. Across the set, that answer came back as “almost instantly,” though the authors are careful to note this reflects how these projects coordinate disclosures and is not, on its own, evidence of diligence. The second question matters far more if you are the one running the software. How long does a fix take to travel from the engine into the specific build sitting on your servers? The range there was startling. Some fixes arrived the same day. Others took north of 471 days. In one case the honest answer was that you simply cannot tell, and in another, it was closer to never.

The engine is rarely what causes the delay. The culprit is the product’s pin policy, meaning the vendor’s decision about which engine version to ship and how eagerly to refresh it. One product in the set stayed frozen on a single release for more than 471 days. It sailed past twelve newer versions in that window and skipped two serious vulnerability fixes, despite riding on an engine that was among the quickest to patch anything. Now hold that against the product with the weakest isolation design of the bunch. On its documented default install, at least, that one came back fully current on the headline measure, for the unglamorous reason that its default setup happens to pull a freshly patched component. So the architecturally impressive option left users exposed while the supposedly flimsy one kept them covered.

The takeaway is not subtle. A strong logo or a premium isolation story tells you very little about your real patch exposure. You have to check it for yourself, product by product, and you should make it a line in the contract instead of a comfortable assumption.

“No Known Vulnerabilities” Is a Sales Line, Not a Safety Rating

The study’s sharpest observation lands on a product whose underlying engine has zero published vulnerabilities. The intuitive reading goes: no recorded bugs, therefore the safest pick. That reasoning is precisely the trap these authors flag. A spotless record might mean the code really is sound. It might also mean nobody has bothered to look hard. And when that same engine has no public bug-hunting effort and no independent academic poking at it, the truthful label is not “low risk.” It is “we have no idea,” which is a very different thing to put your name behind.

This carries weight commercially because “zero known issues” is exactly the sort of phrase that sounds great in a deck and falls apart in a post-incident review. The habit these authors model, refusing to launder a lack of data into a feeling of safety, is the same habit a buyer should bring to any security claim that cannot be checked from outside. Where a vendor has nothing but its own assurance to offer, give yourself more room for caution rather than ticking a box.

Also read: AI Safety and Fairness Nowadays: Explained

A Procurement Checklist That Falls Out of the Research

What makes this useful for a non-specialist is that it turns a murky technical purchase into a handful of questions anyone can put to a vendor.

First, which isolation architecture am I actually buying? The category of engine sets the built-in security properties and how much separation you get by design. That is your opening filter, though, as the patch finding shows, it does not settle anything by itself.

Second, how fast will fixes land in my specific deployment? Delivery is where your exposure actually lives, not mere availability upstream. Get the answer in writing.

Third, can my own team layer its own protections on top? A few products quietly close off standard hardening through their default settings, so confirm you are not being boxed in.

Fourth, what independent verification stands behind your security claims? If the only thing on offer is the vendor’s say-so, file the claim under unproven and move on.

The Bottom Line

For anyone deploying AI agents, this research drags sandbox selection out of the back office and into the territory of governance, with real money and reputation riding on it. The deeper lesson rhymes with something operations people already feel in their bones elsewhere: the boring foundational work is usually where both the worth and the danger quietly accumulate. Companies that treat their execution layer as something measurable and accountable, instead of a promise swallowed whole, are the ones that will not be drafting an apology after a breach. As the authors frame it, measuring honestly, which includes openly admitting what you still cannot measure, is the work that has to come before any marketing copy is worth the pixels it sits on.

Frequently Asked Questions