TL;DR
Three answers to the same question about agentic AI risk
We expected broad agreement about constraining AI agents and got three different answers.
- Pedro Fortuna: break agents into many small, specific ones, each with the narrowest possible permissions, and add runtime monitoring that stops them early.
- Jack Trefon: the opposite, for coding agents. Constrain one and you destroy what makes it useful, so accept the blast radius and invest in tracing and fast recovery.
- Luis Rodríguez Berzosa: a third line. Treat frontier models as you would treat malware, design the harness to limit each specialized agent, and put an enforcement layer below all of it that holds when everything above it fails.
Where all three met: you cannot protect what you have not inventoried, every action must be logged, and the thing that stops an agent has to sit outside the agent.
Nothing was breached.
The agent did its job. On the way, it decided it needed to touch a dependency, a config, a service nobody asked it to touch. It stayed inside its permissions the whole time. No rule was violated, so nothing fired.
We put that case to three people who deal with it professionally, expecting broad agreement on what to do about it. We got three different answers, and two of them contradict each other.
Agentic AI risk usually gets described as an attacker problem. This is the other version. For twenty years we drew permission boundaries around software that did exactly what it was told. A service account, a CI runner, a scheduled job. We got the boundary wrong often enough, but the thing inside it was predictable. It did one job, the same way, every time.
The thing inside the boundary now decides. And the boundary is still drawn the way it always was: in advance, at configuration time, by somebody who cannot know what the task will turn into.
That is the version of agentic AI risk worth arguing about, and we spent an hour on it in the last SafeDev Talk with Pedro Fortuna of Jscrambler, Jack Trefon of Trefon Digital Consulting, and Luis Rodríguez Berzosa of Xygeni, hosted by Alejandro Estrada.
We expected them to broadly agree. They did not.
Agentic AI risk: the comforting premise that does not hold
The framing above has a word in it doing more work than it should: attacker. There isn’t one. That is supposed to make the scenario easier.
Pedro took that apart in one move.
“Being an attacker or not is a human concept. What makes an attacker an attacker is that it has different goals. And a misaligned agent is also something that has a different goal.”
Sit with that, because it dismantles the premise of most agentic AI risk conversations, including the description we wrote for our own event. If what defines an attacker is divergent goals rather than malice, then “there is no attacker in this story” is a comfort nobody earned. Structurally, it is the same risk. What is missing is intent, and intent was never the thing your controls detected. They detect effects. They always did.
This is also why the industry’s vocabulary moved. OWASP’s 2026 Top 10 for LLM Applications promoted excessive agency from sixth place to third, which is less a change of opinion than the incident record catching up with an architecture that was already deployed.
What the Hugging Face incident actually shows
July’s incident, in which a frontier model attacked a third party, got read widely as a story about a model going rogue. Luis reads it differently, and his version puts the responsibility somewhere uncomfortable.
“The problem is not the frontier model. The frontier model did what it was instructed to do. The problem is the design of the experiment.”
The goal was open-ended. The restrictions were absent. And the sandbox meant to contain it had a window to the internet, through a server-side request forgery vulnerability in the package cache the agent could reach. Every one of those is a design decision made by a person.
Which produces the sentence that should change how you provision these things:
“Frontier models need to be looked at like if they were malware, because they can act autonomously and take decisions that are malicious. Not by intent, but they’re really malicious at the end.”
Not as an insult to the model. As a provisioning posture. You do not hand malware a bash shell and hope the goal was well specified.
Pedro’s reading of the same incident was about timing rather than design:
“The minute that you see agents communicating through folder names on a service they shouldn’t be using, that moment you need to be stopping those agents. You don’t need to wait for those agents to successfully breach Hugging Face in order to decide whether or not that’s malicious.”
A detail from the response is worth keeping too. The responders tried to use a commercial agentic tool to analyse the traces and could not, because they hit rate limits. The attacker had no such ceiling. “There was an asymmetry in the attack itself.”
The actual argument: limit the agent, or recover faster?
Here is where the panel split, and it is a real disagreement between practitioners who have each earned their position.
Pedro argues for decomposition. Fifteen years of pushing isolation inside the browser taught him you cannot enforce anything at runtime against a monolith. Break the architecture into small pieces with specific functions, and only then do controls and monitoring points have somewhere to attach. Agents are no different.
“I think you have to break it down into very specific agents that are only meant for specific tasks. And it doesn’t matter if you end up with fifty agents. It’s better than just having a single one.”
Jack argues the opposite, for coding agents specifically. You would expect a background running technology in a regulated crypto exchange to produce the most restrictive position in the room. It produced the least.
His starting analogy is not a threat model at all:
“I’m looking at a typical agentic deployment just as I would look at a new hire, a new person that is coming in. And that person can be flawless in the execution, or it could make a mistake. I don’t see much of a difference.”
For narrow agents he is as strict as anyone. Describing a read-only review agent he is currently building, he was blunt that the common setup is indefensible: deploying onto a vanilla Linux box with bash access “is opening up the entire ecosystem of threats”. But for general coding agents the logic inverts, and he knows the position is unpopular:
“The more power to the agent, the more creative it is and the more effective it is. So it is not the industry that is thriving due to limitations and shrinking the scope.”
So what is left if you do not limit? Circuit breakers, and above all time-based snapshots: not preventing the blast radius, but making recovery from it cheap and fast. He builds that into Amber, a dependency-free C++ agent harness he writes in his spare time, with snapshotting in the loop (project site).
And the warning about where the other path ends:
“If we start taking away, there will be only one tool left, and that developer will be able to turn on the lights and turn off the lights.”
Luis takes a third position, which is neither constrain nor recover but contain from below: assume the agent will eventually behave like malware, design the harness to limit each specialized agent, and put an enforcement layer underneath all of it that holds when everything above it fails.
Three approaches to the same problem
Three positions, each with a cost attached. None of them is free, and none of them makes the agent safe on its own.
| Approach | Core move | What it costs you |
|---|---|---|
| DecomposePedro Fortuna | Many small agents, each with narrow permissions, plus runtime monitoring that stops them early | Design effort up front, and a harder time with open-ended coding work |
| RecoverJack Trefon | Keep the capability, add circuit breakers and time-based snapshots so rollback is cheap | A wider blast radius, and a bet on detecting and undoing it in time |
| Contain from belowLuis Rodríguez Berzosa | Assume malware behaviour, limit each agent in the harness, enforce beneath it at the workstation and CI runner | Infrastructure and tooling that most endpoint stacks do not provide yet |
Reading them side by side, the disagreement is narrower than it sounds. All three accept that the agent will do something unintended. They differ on where the money goes: into the architecture before it happens, into the recovery after it happens, or into a floor beneath both.
Why least privilege does not translate to agentic AI risk
Least privilege assumes you know the job. You scope the permission to the task.
An agent breaks that quietly, because its permission boundary is drawn at configuration time and its task arrives at runtime. What you actually grant is the union of everything it might ever be asked to do. That is not least privilege; it is the opposite, wearing least privilege’s clothes.
Luis offered a reframing borrowed from an idea older than any of this:
“I think about the UNIX mechanism of piping, connecting simple tools together to perform a complex task. You can define specific agents with a controlled configuration, limiting what that specialized agent can do. And you can combine them for almost any task.”
Plus a second design idea most teams have not tried: wrap the tools rather than widen the permissions. Not bash, which hands over the entire system, but a narrow wrapper that is the only invocable path, with read-only variants wrapping generic ones. The agent gets a capability, not a shell.
He was also clear that the obstacle is not the design; it is the rollout. New controls on developer machines have a long history of being worked around, and that history does not change because the tool in question is an agent.
And about where he thinks it all ends:
“I think that at the end we will have autonomous agents controlling other autonomous agents.”
The laptop became the crown jewel, and your EDR has not noticed
An agent needs standing access to be useful. A token that does not expire mid-task, a cached credential, push rights, registry access, a live path into the pipeline. Every one of those is a requirement rather than an oversight, which means the better the tooling got, the more concentrated the developer machine became.
Attackers noticed first. As Pedro put it, they are not ideologues:
“They organize like little companies. They need to make money, so they move to the most profitable angle that they can find. And they were quick to see that there was money to be made in dev stations.”
We watch that shift land in the registries every week in the Malicious Code Digest, where credential harvesters now routinely hunt AI provider keys alongside the cloud and registry ones.
His mitigation list is unglamorous and works: remove every key that does not need to be on the machine, pair the ones that remain with a second factor such as OpenID Connect or a hardware key so a stolen token is not enough on its own, and move the environment off the host where you can, through dev containers or cloud IDEs.
Then Luis made the point that should worry anyone assuming their endpoint stack covers this:
“The EDRs are specialized in traditional malware and not aware of agentic tools. And if they work, they can generate a lot of false positives, because the agent is doing things that the detection patterns are not aware of as of today.”
Two failure modes at once: it misses what matters and flags what does not. His answer is that outer enforcement layer, sitting around the developer workstation and the CI runner, below the harness, bounding what a rogue agent can reach. That is the gap Xygeni Shield was built for, with policy enforced on the endpoint where the install actually happens.
Jack put it more bluntly: do not keep your Bitcoin on your dev machine.
The floor all three agreed on
Three different answers, one shared minimum.
- Inventory first. “You cannot protect what you don’t know about.” Every other decision depends on knowing which agents, skills, and MCP servers exist, and almost none of them were declared.
- Log every action. Jack and Luis arrived here from opposite directions. Jack wants traces because they are how you recover and how an organization matures. Luis wants them because without them you cannot reconstruct the blast radius afterwards. Jack’s framing is worth borrowing: there is no value in interrogating the actor, human or agent, only in understanding the context well enough not to repeat it.
- The thing that stops an agent must sit outside the agent. Pedro’s runtime monitoring, Luis’s enforcement layer, Jack’s circuit breakers and snapshots. Different mechanisms, one principle.
Jack offered a useful framing of where the industry is on the curve. He remembers the first spam email he ever received, telling him he had inherited a gold mine in Nigeria. This is a maturity problem, and maturity arrives after the mistakes rather than before them.
What to do before Friday
Pick one agent. Write down two lists: what it is allowed to do, and what it was actually asked to do this week.
The gap between those lists is your blast radius, and it is the most useful measure of agentic AI risk your team can produce in an afternoon. It takes an afternoon, it needs no budget, and it settles the argument above for your own organization, because the three positions are really three different answers to the question of how big that gap should be.
Most teams have never written it down.
The full hour is available to watch back, including Jack’s point about why human-in-the-loop requirements in Europe make some AI defences infeasible in practice.
And if the exercise returns an answer you do not like, Xygeni discover every AI asset across your repositories, including the agents, skills, and MCP servers nobody declared, and maps what each one can reach. Book a demo or start free, and we will run it on yours.
FAQ
What is the blast radius of an AI agent?
Everything the agent could reach if it went wrong, as distinct from what it was asked to do. It is defined by the permissions, credentials, and tools available to it at runtime, not by the task. The gap between those two is the thing worth measuring, and most teams have never written it down.
Is an agent exceeding its task a security incident if it never exceeded its permissions?
That was the central disagreement. Pedro’s answer is effectively yes, because an attacker is defined by goal divergence rather than malice, and a misaligned agent has diverged. Jack’s answer is closer to no: it is an operational failure like a new hire’s mistake, and the right response is tracing and recovery rather than tighter constraints. Luis sidesteps the question by assuming the worst case from the start and asking what layer still holds when it happens.
Should coding agents be sandboxed?
For narrow, task-specific agents, all three said yes without hesitation: read-only where possible, no bash, no network unless required. For general coding agents, the panel split three ways: Pedro would decompose them, Jack would keep the capability and snapshot aggressively, and Luis would add an enforcement layer below the harness regardless of what the agent is allowed to do.
Does my EDR cover agentic AI risk?
Probably not well. Current endpoint tools are built around traditional malware patterns and do not model agent behaviour, so they tend to miss what matters and flag what does not. Luis’s answer was an outer enforcement layer designed for developer environments rather than a general-purpose EDR.
What is the single first thing to do?
Build the inventory. Every model, agent, skill, and MCP server in your repositories, and what each one can reach. It is the one step all three panellists named, and the only one that every other control depends on.
