[Governed AI Transformation 03] If AI Can Read It, AI Can Leak It

Key Takeaways

• The first two articles in this series dealt with AI that answers questions. This one deals with AI that reads your systems and acts on them — and agentic AI security is a fundamentally different problem from prompt security.
• Gartner predicts that by 2028 a quarter of enterprise breaches will be traced back to AI agent abuse; the average Fortune 500 company will run more than 150,000 agents by 2028, up from fewer than fifteen in 2025 — what analysts call AI agent sprawl.
• IBM's 2025 data shows the most common AI-related security incident involves a compromised application, API or plug-in, and 97% of organisations that suffered one had no proper AI access controls in place.
• Two mechanisms are quietly breaking the permissions you already have: retrieval flattens access rights, and delegation gives agents more privilege than the people who trigger them — the decades-old confused-deputy problem in new clothes.
• The fix is four controls that must travel with every request: identity and access management, scope, content and evidence — evidence being the practical expression of GDPR Article 30 and the AI Act's Article 12 record-keeping duty.
• Testing whether an agent architecture is ready to scale comes down to three questions: does the path exist, is it controlled, and can you prove what happened.

The first two articles in this series dealt with AI that answers questions. This one deals with AI that reads your systems — and, increasingly, acts on them.

The moment an agent is given a connector into the ERP, the CRM, the mailbox or the file share — and a second connector out to a SaaS tool or an external API — the security question changes shape. It is no longer “what might an employee paste into a prompt?” It is “what can this piece of software reach, on whose authority, and who will know?” Most organisations answer the first question with training and the second with hope.

From assistants to agents: data now crosses the trust boundary

In the assistant era, data moved along one short path: a person typed something, a model replied. The risk sat at the keyboard, and the controls — training, a usage policy, a data-loss filter on the prompt — sat there too.

Agents change the topology. An agent is authorised to read from several internal systems, reason about what it finds, and then act — by calling an external API, updating a SaaS tool, sending a message, or handing context to another agent. Each hop is individually approved. The sequence is not. Customer records read from the CRM in step one are still in the agent's working context when it calls a third-party service in step four, and no single permission check ever saw both ends of that journey. The agent has quietly become the only entity in the company that routinely carries data across the trust boundary in both directions, on its own initiative, at machine speed.

Figure 1. An agent sits astride the boundary between systems of record and the outside world. Six risk patterns follow: three concern where data goes (boundary expansion, delegated authority, context contamination) and three amplify them (compromised tooling, non-deterministic paths, speed and scale).

The new attack surface is access, not the model

Analysts have been unusually direct about this. Gartner predicts that by 2028 a quarter of enterprise breaches will be traced back to AI agent abuse, from external attackers and malicious insiders alike. [1] Later research from the same firm warns that agent permissions must be scoped specifically for the agent rather than inherited from human user roles. [2] Gartner also expects the average Fortune 500 company to be running more than 150,000 agents by 2028, up from fewer than fifteen in 2025 — what analysts call AI agent sprawl. [3]

IBM's 2025 breach data explains why the risk is concentrated at the point of connection: the most common AI-related incident involved a compromised application, API or plug-in, and 97% of organisations that suffered one had no proper AI access controls in place. [4]

And the gap is widening rather than closing. Deloitte's 2026 survey finds 74% of organisations expect to be using AI agents by 2027, while only 21% have a mature governance model for them. [5]

Figure 2 — Agent risk, by the numbers.

Put simply: the industry is wiring AI into its systems of record several times faster than it is deciding who is allowed to pull the wires.

Why agents break the permissions you already have

Two mechanisms do the damage. The first is retrieval. Retrieval-augmented generation works by indexing documents into a vector store and letting the model retrieve relevant passages at query time. The convenience is enormous. So is the side effect: the index flattens permissions. A contract that was visible to three people in a locked folder becomes a chunk of text that a retriever will happily surface to anyone who phrases the question well. Unless access rights are re-evaluated at query time, in the user's context, a knowledge agent becomes the most efficient data-exfiltration interface the company has ever built — and one that leaves no trace in the file server's logs, because the file server was never asked.

The second is delegation. An agent given a service account to “read the CRM” typically holds far more privilege than the person who triggered it. That person may not be allowed to see the board's pipeline, but the agent is, and a well-crafted request — or a prompt injected via an email the agent was asked to summarise — can steer it there. This is the confused-deputy problem, three decades old, arriving in a new costume. Combine the two with the cross-boundary flows in Figure 1, and a low-privilege user, a helpful agent and a legitimate connector can move regulated data outside the company without anyone having done anything forbidden.

Figure 3 — Two mechanisms that break the permissions you already have: retrieval and delegation.

Figure 4. Each hop in the access path — user, workspace, agent, connector, system — needs its own control. Permission and accountability must travel with the request rather than stop at the front door.

Four controls that must travel with the request

Put those two failure modes together and there is only one direction the answer can go: identity and access management can no longer be a check made once at the front door — it has to travel with every request, across the whole access path. That breaks down into four controls:

These four are the design constraints we built into AnyInsight. Its three-tier IAM ties every request to an organisation, a workspace and a user; workspaces are walled, with knowledge, agents and connectors assigned per workspace and never shared by default; the zero-trust firewall inspects every prompt and response in line; and five log types — system, dialogue, automation, shared conversation and connector — plus an immutable Data Vault provide the evidence trail. The 500-plus MCP connectors that make agents useful are the same reason the scoping has to be strict, and we treat the two as inseparable.

A practical test for your architecture

Ask three questions of any agent deployment, existing or planned — does the path exist, is it controlled, and can you prove what happened:

Figure 5 — A practical test for your AI architecture.

1. Can any agent in your environment both read an internal system of record and call an external service within the same task? If so, what sits between those two steps?
2. When an agent reads customer data in one step, what prevents that data from travelling — in its working context — into a later call to a SaaS tool, an external API or a model provider?
3. Can I produce, for a named individual and a named date, every system the AI read on their behalf and every action it took?

If any answer is “no” or “I'd have to check”, the deployment is not ready to scale, however impressive the demo.

Conclusion

As AI moves from answering questions to reading systems and acting on them, the centre of gravity in security shifts from the model itself to the access path. Retrieval flattens permissions, delegation amplifies them, and together they can move regulated data out of a company without anyone doing anything wrong. Identity and access management, scope, content and evidence are the four controls that have to travel with every request, rather than stopping at the front door.

If your organisation is planning or already running AI agents, AnyInsight's team can walk you through what identity, scope, content and evidence controls that travel with every request would look like for your architecture.

The final article in this series turns these principles into a checklist: ten questions to ask any enterprise AI platform vendor before you sign — including how it governs agents that move data between systems.

Figure 6. This series runs four parts; this is Part 3. Part 1 covered the governance bottleneck; Part 2 covered shadow AI; Part 4 will cover the ten questions to ask before you sign with an AI platform vendor.