AI Agents in the Enterprise: why governance is more urgent than ever

A new kind of colleague

Imagine an employee who is available 24/7, never sleeps, has access to all your business systems and makes decisions independently. Not a hypothetical scenario. This is what AI agents are already doing in thousands of organizations worldwide.

AI agents are fundamentally different from the chatbots and AI assistants we've grown accustomed to. Where a chatbot waits for instructions, an agent takes initiative. A chatbot answers questions. An agent plans, acts, queries data, and communicates with other agents to complete complex tasks.

Adoption is accelerating at breakneck speed. According to Microsoft's Cyber Pulse report, leading sectors including software and technology (16%), manufacturing (13%), financial institutions (11%) and retail (9%) use agents for tasks such as drafting proposals, analyzing financial data, triaging security alerts and automating customer processes. And here's the thing: building agents is no longer reserved for developers. Employees across all functions create and use agents with low-code and no-code tools.

That changes everything.

The governance gap: bigger than you think

The speed at which organizations adopt AI agents stands in stark contrast to the speed at which they set up governance. And that gap is growing by the day.

Shadow agents: the invisible threat

Microsoft reports that 29% of employees already use unsanctioned AI agents for work tasks. Not with malicious intent, but simply because it makes their jobs easier. The problem: these agents operate outside the view of IT and security teams, with all the risks that entails.

Shadow IT has been around for decades. But shadow AI introduces an entirely new risk dimension. Agents can inherit permissions, access sensitive information and generate outputs at scale. IBM's Cost of Data Breach Report shows that shadow AI is now responsible for 20% of all data breaches, with costs averaging $670,000 more per incident.

Organizations meanwhile struggle with questions that could not be more basic:

• How many agents are actually running across our organization?
• Who is responsible for which agent?
• What data do they touch?
• Which agents are sanctioned and which are not?

If you can't answer those questions, you don't have governance. You have hope.

The CISO paradox

This is where it gets truly alarming. MachineLearningMastery describes a paradox playing out across the entire industry: most Chief Information Security Officers express deep concern about AI agent risks, yet only a handful have implemented mature safeguards.

The numbers back this up. Vectra AI estimates that 40% of enterprise applications will contain autonomous AI agents by end of 2026, while only 6% of organizations have an advanced AI security strategy. And the 2026 CISO AI Risk Report reveals that 71% of organizations say AI tools have access to core systems like Salesforce and SAP, but only 16% say that access is effectively governed.

Organizations are deploying agents faster than they can secure them.

Agents as insider threats

Palo Alto Networks warns of a shift that many security teams don't yet have on their radar: "The AI agent is a potent insider threat." These agents have privileged, always-on access. They are the most valuable target an attacker can compromise. Rather than targeting humans, attackers will focus on taking over agents that are already deep within systems.

The difference from a human insider? A compromised agent operates at machine speed. The damage that can occur before anyone notices is exponentially greater.

What does the EU AI Act say about agents?

The EU AI Act was not specifically written with AI agents in mind. The law was largely designed during a period when AI systems were still primarily passive tools. But that does not mean agents fall outside its scope. Quite the opposite.

The Future Society published the first comprehensive analysis of how AI agents are regulated under the EU AI Act, with three key findings:

First: agents fall under both GPAI and high-risk provisions. Most current agents run on general-purpose AI models that may carry systemic risk. Depending on the specific application, agents can also be classified as high-risk AI systems. Agents intended for multiple purposes are even assumed to be high-risk, unless the provider demonstrably takes sufficient precautions.

Second: governance must span the entire value chain. Model providers must build the fundamental infrastructure for safe agents. System providers adapt these for specific contexts. And deployers, the organizations that use agents, must comply with rules during operation. This shared responsibility is crucial, but also complex.

Third: four governance pillars. Risk assessment, transparency tools, technical deployment controls and human oversight design. Within each of these pillars, the report identifies specific requirements for providers and deployers.

When is an agent high-risk?

The practical question for organizations: when does my AI agent fall under high-risk classification? The answer is more concrete than you might expect.

Annex III of the AI Act lists the domains: employment and HR decisions, education, credit scoring, law enforcement, critical infrastructure, and access to essential services. Are you deploying an AI agent that influences who gets hired, who receives a loan, or who has access to a service? Then it is likely a high-risk system.

Kennedys Law points out that the European Commission may weigh the degree of autonomy as a relevant factor when determining risk level (Art. 6). The more autonomous the agent, the higher the risk profile. And that is precisely what distinguishes agents from traditional AI tools.

The deadline for high-risk AI system conformity requirements: August 2026. That is six months away.

Five things organizations must do now

Based on the research and best practices, five concrete priorities:

1. Create an Agent Registry

You can't protect what you can't see. Microsoft's Cyber Pulse report emphasizes the need for a centralized registry as a single source of truth: which agents are running, who owns them, which are sanctioned and which are not? This is step one, no exceptions.

2. Apply Zero Trust to agents

Treat AI agents like employees or service accounts. That means: least privilege access (only the minimum required permissions), explicit verification for every access request, and the design principle that compromise is always possible. No agent should have more access than strictly necessary. Period.

3. Classify your agents under the EU AI Act

Map out which agents operate in high-risk domains. Start with a risk assessment, work on documentation and design human oversight. This is not optional: the high-risk compliance requirements take effect in August 2026.

4. Invest in observability

Real-time dashboards and telemetry. Where are your agents operating, with what data, what behavior do they exhibit? Microsoft identifies five capabilities: registry, access control, visualization, interoperability and security. None of these are luxury. They are hygiene.

5. Make governance cross-functional

AI governance cannot rest solely with IT or the CISO. It is a shared responsibility across legal, compliance, HR, data science and the board. Deloitte confirms: organizations where senior leadership actively shapes AI governance achieve significantly greater business value than those delegating it to technical teams alone.

Treat AI risk as enterprise risk. Not as an IT problem.

The paradox we need to solve

There is a fundamental tension in how we deal with AI agents. On one hand, they are incredibly powerful. They boost productivity, accelerate processes and can perform tasks that were previously impossible. On the other hand, they introduce risks that existing governance frameworks cannot fully address.

The EU AI Act provides a foundation, but it is not the complete answer. The law assumes relatively static AI systems with predetermined configurations. AI agents are dynamic, adaptive and operate in chains of interactions that are difficult to predict in advance.

What organizations need is not just compliance with a law, but a fundamental reconsideration of how they deal with autonomous systems. How do you give a "digital employee" responsibilities without losing control? How do you audit decisions made at machine speed? How do you prevent the tools that make your organization more efficient from simultaneously becoming your greatest vulnerability?

These are not questions for next year. These are questions for right now.