TL;DR
"AI security threats" covers two different things: attacks aimed at AI systems, and attacks that use AI as a weapon. They need different controls, different owners and different evidence, and most lists mix them together, which is why they are so hard to act on. The first five below target your AI. The second five use AI against your supply chain.
- Threats to AI systems mostly live in configuration, not in the model. Tool scope, the credential an agent runs under, a rules file, an
.mcp/servers.jsonentry. None of it is reachable from the model endpoint, which is why red-teaming alone comes back clean. - AI-driven attacks mostly arrive through dependencies, not through your perimeter. Hallucinated package names registered by attackers, self-propagating worms using developer credentials, AI provider keys as the objective.
- Nine of the ten are questions about specific things. Which MCP servers, which agents, which scope, which credential, which rules file, which package. A policy document answers none of them.
- Every one of the ten shares a precondition. You cannot defend an AI asset you have never mapped, which makes discovery the step before the controls rather than the one after.
What are the two kinds of AI security threats?
Most lists of AI security threats mix the two, which is why they are hard to act on.
- Threats to AI systems treat your AI as the target. Someone manipulates a model, poisons the instructions it follows, or abuses the permissions an agent was given. The defender’s job is containment: reduce what a compromised AI can reach.
- AI-driven attacks treat AI as the attacker’s instrument. The target is your ordinary software: dependencies, credentials, build systems, developers. The defender’s job is detection at speed, because the economics of the attack have changed and volume has gone up.
They need different controls, different owners and different evidence. A team that has red-teamed its chatbot has done nothing about slopsquatting. A team with excellent dependency screening may still be running an agent with write access to production repositories and no scope limit. Both are AI security threats. Neither covers the other.
Threats to AI systems: the five that land most often
01. Prompt injection that reaches a tool sink
Untrusted content arrives, gets concatenated into a system prompt, and the agent on the other end can invoke tools. The prompt is not the vulnerability. The path from untrusted input to a dangerous capability is.
This is why prompt filtering alone disappoints. It hardens one end of a chain while the interesting question is whether a guardrail sits anywhere between the input and the tool call. The OWASP Top 10 for LLM Applications keeps prompt injection at the top of the list, but pairs it with excessive agency for exactly this reason.
What to look for: untrusted input reaching a system prompt, retrieval feeding context without sanitisation, and reachable tool sinks with no guardrail in between.
02. Malicious rules and skill files
A rules file is plain text. It is committed like documentation and reviewed like documentation, and it instructs the assistant on what to do. Zero-width characters make lines invisible in a pull request diff while remaining perfectly legible to the agent.
MITRE ATLAS catalogues this as the Rules File Backdoor, case study AML.CS0041. The reviewer approves a formatting convention. The agent reads an instruction to import from an attacker-controlled package and to say nothing about it.
What to look for: rules files, skill files, and prompt templates analysed as code rather than prose, with invisible-character detection.
03. Unsafe MCP server configuration
MCP configuration is where an agent takes its orders and where credentials accumulate. Both halves are a problem. CVE-2025-6514, disclosed in July 2025, was a critical OS command injection in a widely used MCP bridge, carrying a CVSS score of 9.6 and allowing arbitrary commands on the client host when connecting to an untrusted server.
The credential half is worse because it is passive. Research in 2026 found more than 24,000 secrets sitting in MCP configuration files on public GitHub, with over 2,000 of them still valid.
What to look for: an inventory of MCP servers in use, their versions, their declared tools, and secrets detection running over configuration files.
04. Excessive agency
An agent given broad tool scope and a long-lived credential is a standing privilege escalation waiting for a trigger. Nothing has to be exploited for this to hurt. A confused agent and a compromised one do similar damage when both have write access they never needed.
This is the least glamorous item on the list and the most common. It also has the cheapest fix, which is scope reduction, and the fix is rarely applied because nobody owns the question.
What to look for: which agents exist, what each can invoke, under whose credentials, and whether anyone approved that scope.
05. Poisoned models, datasets and AI dependencies
The AI stack is built on ordinary packages and hosted artifacts. Models are downloaded from public hubs. Datasets are pulled from wherever. Both carry the same provenance problem as any dependency, with less tooling and far less scrutiny.
What to look for: models, frameworks, and datasets treated as supply chain components, with the same composition analysis and the same CVE tracking as the rest of your dependencies.
AI-driven attacks: the five reshaping your supply chain
06. Slopsquatting
Research presented at USENIX Security in 2025 generated 2.23 million code samples across 16 models and found that 19.7% of recommended packages did not exist, producing over 205,000 unique hallucinated names. Around 43% of those names recurred across repeated runs of the same prompt, which makes them farmable.
Attackers no longer need to compromise a maintainer. They register the name your agent invented and wait for the install. The package has no signature and no advisory, because it is new and nobody has seen it before.
What to look for: malware screening that works on behaviour rather than reputation, applied to every dependency an agent adds.
07. Self-propagating supply chain worms
The September 2025 npm worm marked the point where developer machines became the delivery mechanism rather than the target. The pattern repeated at scale through 2026. Each compromised developer credential publishes the next round of compromised packages, and the blast radius compounds instead of adding.
What to look for: anomalous publishing activity, unusual commit patterns, and credential use that does not match the developer’s normal behaviour.
08. Nation-state targeting of AI credentials
In March 2026, a package with roughly 100 million weekly downloads was compromised for a period measured in hours, with AI provider API keys as the objective. The window was deliberately short. Long enough to harvest, short enough to reduce the chance of detection.
AI credentials are now a first-class target in their own right, because they buy compute, access, and a plausible identity. Attacks aimed at stealing them rose sharply between late 2025 and early 2026.
What to look for: AI provider keys treated as secrets with the same detection and revocation as cloud credentials.
09. Malware authored or planted using an LLM
By 2026, there were documented cases of a language model being weaponised to plant malicious code inside an agent workflow. The significance is not that the malware was novel. It is that authorship stopped being a bottleneck, and the volume of plausible-looking malicious code available to an attacker went up accordingly.
What to look for: detection that does not depend on having seen the sample before.
10. Review capacity collapse
This one is not an attack, which is why it is last and why it matters most. When an agent produces a thousand files in a week, code review stops being a control and becomes a queue. Every other threat on this list gets easier when the last human check is overwhelmed.
What to look for: findings ranked by exploitability rather than raw severity, so the small number that threaten production actually reach a developer.
Ten threats is the map. What to do about each one takes longer than a blog post, so we had the conversation properly. In this episode of SafeDev Talks, Jesús Cuadrado, CEO at Xygeni, sits down with Atanas Nikolov, DevSecOps expert and SSDLC tech lead, on polymorphic malware, prompt injection, model tampering, and what actually holds up against them. No form, no gate.
Why can you not defend AI assets you have not mapped?
Read the ten again, and the pattern is hard to miss. Nine of them are questions about specific things: which MCP servers, which agents, which scope, which credential, which rules file, which package.
None of those questions can be answered from a policy document. They are answered from an inventory, and most organisations do not have one. The AI arrived through developers moving quickly, without a ticket, a procurement process or a repository entry that announced itself as AI.
That makes mapping the precondition rather than the first step. An AI security threats programme that starts with controls will apply them to the AI it knows about, which is a subset it cannot size. Discovery has to come first: models, frameworks, datasets, inference endpoints, agents, MCP servers, skills, prompts, and the AI coding tools your developers actually use, with the relationships between them, because an asset in isolation tells you very little and the risk concentrates in the chain.
The NIST AI Risk Management Framework puts map before measure and manage for the same reason. It is the least interesting function in the framework and the one everything else depends on.
Which AI security threats should you act on first?
Not the most sophisticated. The most reachable.
- This week: count your MCP servers and run secrets detection over their configuration files. Threat 03 is the highest ratio of exposure to effort on the list, and almost nobody has looked.
- This month: find your agents and write down what each one can invoke and under whose credentials. That addresses threat 04, which is the most common and the cheapest to reduce.
- This quarter: put behavioural malware screening in front of every dependency an agent adds. That covers threats 06, 07 and 09 at once, because all three defeat reputation-based checks by being new.
Everything else can wait behind those three, because everything else assumes you already know what you have.
The organisations that handle AI security threats well in 2027 will not be the ones with the best model evaluations. They will be the ones that could answer, on any given Tuesday, what AI is running in their software and what it is allowed to reach. That is the position Xygeni was built around: continuous discovery of every AI asset across repositories and pipelines, risk scored against the frameworks your auditors already recognise, and the supply chain screening that catches what has no signature yet.
Ten threats, two categories, one map underneath both.
FAQ
Are AI security threats really different from ordinary application security threats?
Partly. The consequences are familiar: code execution, credential theft, data exposure. What changes is the entry point and the speed. Configuration files that were previously inert now carry instructions that something acts on, and an attacker’s cost of producing convincing malicious code has dropped. Your existing controls still apply; they just no longer cover the whole surface.
Does red-teaming our models address most of these threats?
It addresses part of one of them. Red-teaming answers whether a model can be talked into something it should refuse, which is genuinely useful. It cannot see the tool scope, the credential, the rules file, or the MCP configuration, because none of those are reachable through the model endpoint. A clean red-team report and a dangerously over-privileged agent coexist without contradiction.
Which of these AI security threats is most underestimated?
Excessive agency, by a wide margin. It has no CVE, no exploit to demonstrate, and no vendor selling a dedicated product for it, so it generates no urgency. It is also the difference between a contained incident and an uncontained one, because scope is what determines how far anything travels after the first mistake.
How do AI-driven attacks change what we should measure?
They shift the useful metric from severity to time. When malicious packages are generated faster than signatures are published, “how many criticals are open” matters less than “how quickly does something new get flagged”. Measure the gap between a dependency entering your codebase and being screened, and the gap between a finding being raised and reaching a developer who can act on it.
