TL;DR
Every board is now asking about AI risk, and most security teams answer with adjectives instead of numbers. The governance frameworks tell you what to govern. They do not tell you what to count. These AI risk metrics all come from the same place: the code your organisation already ships.
- An agentic AI strategy has three layers, and only one of them is the model. What the agent writes, what the agent is wired to, and what the agent pulls in. Most programmes measure the first and ignore the other two.
- The blast radius is not in the model. It is in the skill file, the MCP configuration, the tool scope and the credential the agent runs under. A clean model evaluation and an over-privileged agent coexist comfortably.
- Nine AI risk metrics, each one a number rather than an adjective. Unapproved AI assets, prompt injection paths that reach a tool sink, secrets in
.mcp/servers.json, agent-added dependencies flagged before a signature existed. - If you cannot produce the number this quarter, the metric is not the problem. The inventory is. Start with three counts, not a programme.
What does AI risk actually mean when you are the one shipping the software?
Ask ten people what AI risk means, and you get ten answers. Model bias. Hallucination. Data leakage. Regulatory exposure. Job displacement. All real, all worth attention, and none of them answerable by a security team on a Tuesday afternoon.
There is a narrower version of the question that a security team can actually own: what AI is running inside the software we build, what can it reach, and what happens when someone manipulates it. That version has an answer, and the answer lives in your repositories and pipelines rather than in a policy document.
This is the version of AI risk this post is about. Not because the governance questions do not matter, but because the governance questions cannot be answered until this one is. A framework asks you to inventory your AI systems. It does not tell you that the inventory is sitting in a .mcp/servers.json file that nobody has opened since it was committed.
Why do most AI risk programmes measure the wrong thing?
Three failure patterns, and most organisations run at least two of them.
- Measuring adoption instead of exposure. “72% of our developers use AI tooling” is a productivity metric wearing a security badge. It tells you nothing about what those tools can reach.
- Measuring the model instead of the system. Red-teaming an endpoint answers whether a model can be talked into saying something it should not. It does not answer what happens next, because the blast radius lives in the tool scope, the credential the agent runs under, and the configuration files that define both. A clean model evaluation and a wildly over-privileged agent coexist very comfortably.
- Measuring what is easy to count. Number of AI policies published. Number of staff who completed AI training. These numbers go up reliably and correlate with nothing.
The NIST AI Risk Management Framework is explicit that measurement is one of its four core functions, alongside govern, map and manage. The framework is deliberately outcome-based rather than prescriptive, which is correct for a standard and unhelpful on a Tuesday. You still have to decide what to count.
Which AI risk metrics actually tell you something?
Nine metrics. Each one is a number, each one changes when your exposure changes, and each one can be produced from code rather than from a survey.
| Metric | Why it matters | Where the number comes from |
|---|---|---|
| 01Total AI assets discovered, and the change since last month | You cannot report on AI risk you have never counted. The trend matters more than the absolute number | Continuous discovery of models, frameworks, datasets, inference endpoints, agents, MCP servers, skills and AI coding tools |
| 02Unapproved AI assets as a percentage of the total | This is your shadow AI rate. It is the single most quotable AI risk metric for a board | Approval state tracked per asset against your policy |
| 03AI assets with no named owner | An asset nobody owns is an asset nobody will fix. Usually the ugliest number in the set | Ownership tracked against the asset |
| 04Prompt injection paths that reach a tool sink | Untrusted input reaching a system prompt only matters if something dangerous sits at the other end | AI risk detection mapped to the OWASP Top 10 for LLM Applications |
| 05MCP servers in use, and how many carry findings | MCP configuration is where the agent takes its orders. It is also where credentials pile up | MCP server configuration analysed as a security artifact |
| 06Secrets found in AI configuration files | Research in 2026 found over 24,000 secrets sitting in MCP config files on public GitHub, with more than 2,000 still valid | Secrets detection across code, configs, containers and pipelines |
| 07Agent-added dependencies flagged before a public signature existed | Agents pull packages fast, and the name your model invented is a farmable attack surface | Malware Early Warning, evidence detection followed by AI validation |
| 08Share of AI findings that are reachable and in actively developed code | Separates the backlog you will never clear from the work that changes your exposure | The AI prioritization funnel |
| 09Percentage of agent-authored pull requests merged with a human approval | The governance metric a regulator will ask for, and the one nobody tracks | Pull request analysis and approval gates |
If you take only one, take metric 02. The ratio of unapproved to total AI assets is the number that survives translation from a security team to an audit committee, and it moves when you actually do something.
What does an agentic AI strategy have to cover?
Three layers. The mistake almost everyone makes is starting at the first one and stopping there.
- Layer one: what the agent writes. Generated code carries the same vulnerability classes as human code, and research going back to 2022 has consistently found that a substantial share of AI-generated programs contain security flaws. This layer is the most familiar, which is exactly why it absorbs all the attention.
- Layer two: what the agent is wired to. Skill files, rules files, MCP server configurations, tool scopes, the credential the agent runs under. This is the layer that determines how far a successful attack travels, and it is invisible to anything that tests the model through its endpoint. MITRE ATLAS catalogues this ground directly: the Rules File Backdoor case study, AML.CS0041, describes zero-width characters in a rules file turning documentation into instructions an agent obeys silently. A reviewer reading the pull request sees nothing.
- Layer three: what the agent pulls in. Dependencies, models, datasets, external tools. Research presented at USENIX Security in 2025 generated 2.23 million code samples across 16 models and found that 19.7% of recommended packages did not exist, producing more than 205,000 unique hallucinated names. Attackers no longer need to compromise a maintainer. They register the name your agent invented and wait.
An agentic AI strategy that covers only layer one produces a clean report and an unchanged AI risk profile. The OWASP Top 10 for LLM Applications reflects this in its own structure: excessive agency sits at the top of the list precisely because it is a wiring problem, not a model problem.
How do you implement this without slowing the agents down?
The instinct is to gate everything. It fails because the whole point of the agent is throughput, and a control that halves throughput gets removed within a quarter.
Three implementation principles that hold up:
- Measure continuously, gate selectively. All nine AI risk metrics can be collected on every scan without blocking anything. Reserve hard gates for the small set where the failure is unrecoverable: a malicious dependency, a live secret, an unapproved model handling regulated data.
- Put security where the agent already is. An agent connected over MCP calls the security platform while it generates, scans what it just wrote, and iterates on the fixes until the code meets your thresholds. No human in the loop at that stage, and nothing to install where the agent runs. That is the only intervention point that scales with agent throughput rather than against it.
- Keep the human at the pull request. For anything that reaches production code, a person approves. This is not caution, it is the design position the EU AI Act pushes you toward anyway, and it is far cheaper to build in now than to retrofit under audit pressure. It is also metric 09, which is why that metric is worth tracking from day one.
The row that goes unassigned in most organisations is the second one. Configuration files fall between application security, which treats them as infrastructure, and platform engineering, which treats them as documentation. That gap is where the interesting attacks land.
How do you start measuring AI risk this quarter?
Do not build the full programme. Produce three numbers. Count your AI assets. Count how many of them nobody approved. Count how many secrets are sitting in AI configuration files. Those three fit on one slide; they are all defensible, and they will be worse than whoever asked for them expects.
Then pick the one that is worst and fix it before the next board meeting. That is a functioning AI risk programme. Everything else is elaboration.
The organisations that will struggle with AI risk in 2027 are not the ones that moved fast. They are the ones that spent 2026 writing AI policies and never counted anything. A policy nobody can measure against is a document. AI risk metrics are what turn it into a control.
See your own numbers. Connect a repository and Xygeni returns your AI inventory, the unapproved assets, the risks mapped to the frameworks your auditors use, and an AI-BOM you can export. Start free or book a demo.
FAQ
What is the difference between AI risk and AI safety?
AI safety is mostly about what a model does: whether it behaves as intended, whether its outputs are harmful, whether it can be manipulated into something it should refuse. AI risk, in a security context, is about what a compromised or misbehaving AI system can reach inside your environment. The two overlap, but they are owned by different teams and measured with different instruments. A security team can act on the second one today.
How many AI risk metrics should we report to the board?
Three to five. A board cannot act on nine numbers, and a deck with nine tends to hide the one that matters. Report your total AI asset count, the unapproved percentage, and one exposure metric such as reachable AI findings in active development. Keep the rest for the operational review.
Do we need an AI-BOM if no regulation requires one yet?
No regulation mandates an AI-BOM at present, which is exactly why generating one is cheap right now and expensive later. Two machine-readable formats exist today, and an auditor will accept either. The value is not compliance; it is that producing the file forces you to have the inventory, and the inventory is what every AI risk metric depends on.
Our developers use AI tools we never approved. Where do we start?
Count before you police. Announcing a policy against tools you cannot see drives usage underground and destroys the visibility you need. Run discovery first, publish the number internally without attribution, and use it to agree an approved list that reflects what people actually need. Enforcement lands far better when it follows evidence than when it precedes it.
