TL;DR
A pentest proves what an attacker can do. With AI, the system you tested last quarter no longer exists. Models get new versions, prompts get edited, tools get connected, and none of it touches a line of your code. AI pentesting targets behaviour rather than code paths: what the model can be persuaded to do, what the agent can be made to call, and what leaves the building as a result. The scope now maps to the OWASP Top 10 for LLM Applications and the OWASP Top 10 for Agentic Applications 2026.
The payload is a sentence, and nothing in the request looks malformed. There is no parameter to fuzz when the attack arrives inside a support ticket, a code comment or a retrieved document, which is why indirect prompt injection is the technique that scales: nobody has to touch your front end. Once the model holds tools, the damage lands through the tools it is allowed to use. Tool abuse, excessive agency and data exfiltration through the agent's own integrations are the findings that matter, not whether the answers read well.
One failed attempt proves nothing. AI systems are probabilistic, so a useful result is a rate, not an event: this attack succeeded in 12 of 100 attempts. And the target is wider than the model. Documented attacks already hit the surrounding assets: hidden Unicode in rules files that makes an assistant emit backdoored output, tool poisoning in MCP servers, and full remote code execution through an MCP bridge pulled more than 437,000 times. Re-testing on every model, prompt, tool and server change is the only cadence that reflects reality.
Testing comes second. Seeing comes first: AI-SPM builds the inventory and the AI-BOM, so you know which agents hold which tools and which MCP servers nobody approved; AI Security scores those assets against the OWASP Top 10 for LLM Applications and points at the exact file and line, and the prioritization funnel narrows thousands of findings to the ones that are in use, reachable, exploitable, privileged and business critical; xy-dast keeps testing the web and API surface the AI application still runs on. The same intelligence applies to findings ingested from the scanners you already own, so this extends your stack instead of replacing it.
What is AI Pentesting?
AI pentesting is adversarial testing of AI systems: sending crafted inputs to a model, an agent, or an MCP server to prove what an attacker can actually make it do. Where a classic penetration test targets code paths and configurations, AI penetration testing targets behaviour: hijacked instructions, abused tools, leaked context, and actions the system was never supposed to take. That is the short version of what AI pentesting is. It exists as a separate discipline for an uncomfortable but simple reason. Your annual pentest report describes an application that no longer exists because the model behind it changed three times since the engagement, and no line of your code changed.
Why does a classic pentest not cover AI?
Anyone asking what AI pentesting is for the first time usually starts here, with the gap. A traditional test assumes deterministic behaviour. Send the same request, get the same response, and a finding either reproduces or it does not. AI breaks that assumption in three places.
- The system is probabilistic. The same prompt can succeed on the fourth attempt and fail on the first three. A single negative test proves nothing, which is why pentesting AI systems is a matter of coverage and repetition rather than a one-shot exploit.
- The attack surface is language, not just interfaces. There is no parameter to fuzz when the payload is a sentence inside a support ticket, a code comment or a retrieved document. Nothing in the request looks malformed.
- The blast radius is the tool surface. An agent with tools can send email, query databases, call internal APIs and write files. Pentesting AI agents means testing what the model can be persuaded to do with those tools, not whether its answers are polite.
What AI pentesting actually tests
The scope has settled around a recognisable set of objectives, aligned to the OWASP Top 10 for LLM Applications and the OWASP Top 10 for Agentic Applications 2026.
- Direct prompt injection. Can a user override the system prompt and take control of the model’s behaviour?
- Indirect prompt injection. Can an attacker plant instructions in content the system retrieves, so nobody has to interact with it at all?
- System prompt extraction. Does the model disclose its own instructions, its guardrails or the business logic embedded in them?
- Guardrail bypass and jailbreak. How many attempts, and which encodings, defeat the safety and policy layer?
- Tool abuse. Can the model be induced to call a tool with attacker-chosen arguments, for example, sending data to an arbitrary recipient through its own email tool?
- Excessive agency. What is the widest action the agent can take, and does anything require a human before it executes?
- Data exfiltration paths. Can context, secrets, or personal data reach an output channel, a log, or a third-party tool?
- Retrieval poisoning. Does poisoned content in the vector store change downstream behaviour, and does it persist?
- MCP server exposure. What does each connected server expose, does it validate arguments, and who else can reach it?
- Insecure output handling. Does model output flow unvalidated into a shell, a browser, a query, or a template?
That list is why AI penetration testing produces a different kind of report. The valuable output is not a CVE. The practical answer to what AI pentesting is is a reproducible sequence: this input, through this path, produced this unauthorised action, with this evidence.
AI pentesting compared with what you already run
| Classic pentest / DAST | AI pentesting | |
|---|---|---|
| Target | Endpoints, parameters, configuration | Model behaviour, agent decisions, tool calls |
| Payload | Malformed or malicious requests | Natural language, and content the system retrieves |
| Result | Deterministic and reproducible | Probabilistic, needs repetition to establish a rate |
| Evidence | Request, response, CWE | Prompt, trace, the action the system took |
| Cadence | Per release or per year | Per model, prompt, tool or configuration change |
AI penetration testing and classic runtime testing are complementary, not competing. An AI application still runs on a web stack with authentication, APIs, and infrastructure, and that surface needs runtime testing exactly as before. Xygeni DAST covers it: xy-dast simulates real attack techniques against running web applications and APIs, tests behind login with form, token, header, or script-based authentication, runs from a single CLI command in any pipeline, gates builds on threshold, and returns the attack payload plus the full request and response as evidence on every finding. Findings then pass through the prioritization funnel, which filters to what is internet-exposed, exploitable without credentials, and attached to something the business cares about.
What DAST does not do is argue with a language model. That is the gap AI pentesting fills.
Before the first payload
Two things make an AI pentest useful rather than theatrical, and both come before the first payload.
- A target list. You cannot test AI you have not found. Xygeni AI Security discovers every AI asset in the SDLC, including the models, agents, agent servers, datasets, MCP servers, skill files, prompts, and guardrails nobody declared, reading application code, declared dependencies, and the configuration files AI tools leave behind. It maps the relationships between them, which is what turns a list of assets into an attack path worth testing.
- A shortlist. Xygeni detects the weaknesses that make an exploit likely before anyone attempts one: prompt injection and system prompt leakage, malicious instructions and tool injection in rules and skill files, insecure MCP configuration, excessive agency and missing guardrails, secrets in AI files, and vulnerable or slopsquatted AI dependencies. Findings map to the OWASP Top 10 for LLM Applications and point at the exact file and line, and the prioritization funnel narrows thousands of findings to the handful that are in use, reachable, exploitable, privileged and business critical.
Run in that order and a pentest stops being a fishing expedition. You arrive knowing which agent holds which tools, which server nobody approved, and which prompt accepts untrusted input.
How to run an AI pentest that produces something useful
Seven habits separate useful AI penetration testing from a demo, and teams new to pentesting AI applications tend to skip the first two.
- Inventory first Models, agents, MCP servers, datasets, prompts. Testing an unknown surface produces an unknown result.
- Define what unauthorised means Write down the actions the system must never take. Without that list, every finding is a matter of opinion.
- Test the retrieval path, not just the chat box Indirect injection is the technique that scales, and it never touches your front end.
- Measure rates, not events Report that an attack succeeded in 12 of 100 attempts, because that is the number an engineer can act on and a board can understand.
- Test the configuration layer as well Pentesting AI without reading its configuration is half a test, and a prompt hardening exercise is wasted if a rules file quietly overrides it.
- Re-test on change New model version, new tool, new MCP server or edited prompt, all of them invalidate the last result.
- Feed results back into posture A finding that lives in a PDF changes nothing. A finding correlated with the asset that produced it changes priorities.
FAQ
- What is AI pentesting, in one sentence? Adversarial testing that proves what an attacker can make your models, agents, and MCP servers actually do.
- Is AI pentesting the same as AI red teaming? They overlap heavily. Red teaming is broader and often includes safety, bias, and abuse scenarios; AI pentesting tends to focus on security outcomes such as injection, data exposure, and unauthorised actions.
- How is pentesting AI different from testing a web app? The payload is language, the result is probabilistic, and the damage happens through tools the model is allowed to use rather than through a vulnerable code path.
- How often should AI penetration testing happen? On every change that affects behaviour: model version, prompt, tool surface, connected MCP server, or retrieval corpus. An annual cadence describes a system that no longer exists.
- Do we still need DAST if we run AI pentesting? Yes. The application, its APIs, and its infrastructure remain a target. AI pentesting adds a layer; it does not replace the one underneath.
- What should we test first? The agent with the widest tool surface, and any path where content from outside the organisation reaches a prompt.
Start with what you can see
The answer to what AI pentesting is comes down to one question: is this exploitable, and by whom? Getting a useful answer depends on knowing what exists first, because an attack simulation against an incomplete inventory measures your visibility, not your exposure.
Xygeni discovers the AI in your SDLC, scores what is genuinely exploitable, and enforces policy at the developer endpoint where most of it actually runs. See what your agents are connected to at Xygeni.
