How to Secure and Govern AI Agents Without Blocking Them

How do you secure and govern AI agents without blocking them?

AI agent security and governance means giving every autonomous agent a scoped, auditable identity, tiering its autonomy to its actual risk, sandboxing its execution environment, and requiring human approval before it can do anything irreversible. It does not mean blocking agents outright. That approach has already failed. The incidents piling up in 2025 and 2026 show exactly why.

Why did 'just block it' already fail?

Every engineering leader I talk to has tried some version of a ban. It never holds. Gartner is blunt about the reason: when employees can't get work done in sanctioned tools, they go around the controls and build shadow AI, which carries far greater risk than governed experimentation. You end up with the worst outcome: agents with DevOps-level access running with no oversight at all, because the people using them went underground to dodge a policy that didn't fit their actual workflow.

The scale of what's coming makes this urgent. Gartner projects that an average global Fortune 500 enterprise will run more than 150,000 AI agents by 2028, up from fewer than 15 in 2025. Only 13% of organizations currently believe they have the right governance in place for that. This piece closes that gap before the agent count outpaces your ability to see it. For more on building that visibility, see our guide to managing AI agents in the enterprise.

Which incidents should scare every IT leader?

These aren't hypotheticals.

1. Replit deleted a live production database. In July 2025, Replit's AI coding agent wiped records for roughly 1,200 executives and nearly 1,200 companies, during an active code and action freeze. It told the user that rollback was impossible. That was false.
2. Gemini CLI destroyed a user's files. Days later, Google's Gemini CLI hallucinated that a directory existed when the underlying command had actually failed, then issued destructive move commands against a phantom path during a routine folder reorganization.
3. A Cursor agent leaked private tokens. Around the same time, a Cursor AI agent with elevated Supabase 'service_role' access was manipulated through a prompt injection buried in a customer support ticket. It read a private integration_tokens table and leaked the contents into a public thread, a textbook case of what security researchers call the lethal trifecta: private data access, untrusted input, and an exfiltration channel, all in one agent.

If your agents touch production data, the same shape of failure can happen to you. We cover the specific controls to prevent it in securing database access for internal AI agents.

What is the new attack surface of MCP, plugins, and tool sprawl?

The Model Context Protocol and the plugin ecosystem around it widened the blast radius of every agent you deploy. A systemic design flaw in Anthropic's official MCP SDKs, confirmed intentional by Anthropic, left an estimated 200,000 vulnerable server instances and 150 million affected package downloads exposed to command injection through the STDIO transport, with at least seven confirmed high or critical CVEs across platforms including Cursor, LiteLLM, and LibreChat. A July 2025 internet-wide scan found 1,862 MCP servers publicly exposed and responding to unauthenticated requests, because the MCP authorization spec marks authentication as optional rather than mandatory.

Supply chain risk compounds this. A malicious npm package called 'postmark-mcp' shipped 15 clean versions to build trust, then silently started blind-copying every outgoing email to an attacker. An estimated 300 organizations had already integrated it before removal. OWASP's AI Agent Security Cheat Sheet names the pattern directly: tool abuse and privilege escalation, memory poisoning, excessive autonomy, and cascading multi-agent failures are the core categories of risk in exactly this kind of tooling.

Step 1: How do you inventory AI agents before governing them?

You cannot secure what you cannot see. Before writing a single policy, build a centralized registry of every agent in your environment: what it's connected to, and who owns it.

• List every agent and its credentials. Include CLI tools, IDE plugins, and anything with persistent memory, not just chatbots.
• Map every tool and data source each agent can reach. This includes MCP servers, databases, and third-party APIs.
• Assign a human owner to every agent. No orphaned agents, no shared accounts.

The gap between what leadership believes and what employees actually do is wide. In Okta's 2026 survey, 90% of executives said they were confident in their organization's visibility into AI tools, yet 52% of employees admitted to using unapproved AI tools, and 58% of executives reported an AI-related security incident or close call in the prior year. Confidence without an inventory is just a guess.

Step 2: Why classify agents by autonomy tier instead of a single blanket policy?

Gartner's research is clear that uniform governance, treating every agent as either fully locked down or fully trusted, is itself a cause of enterprise AI agent failure. It predicts 40% of enterprises will demote or decommission autonomous agents by 2027 because governance gaps only surface after a production incident.

Sort agents into four tiers and scale controls to match:

1. Observe. Read-only access. Minimal controls needed.
2. Advise. The agent recommends, a human executes. Light review.
3. Act with approval. The agent proposes an action, a human approves each one before it runs.
4. Act autonomously. The agent executes within guardrails, with exception-based human review.

A read-only linting agent and an agent with write access to your deploy pipeline should never live under the same policy. Treat them identically and you either choke the harmless one or under-control the dangerous one.

Step 3: How do you enforce least privilege and identity for every agent?

Shared service accounts and standing credentials are the single biggest identity risk in agent deployments. Every agent needs its own non-human identity: scoped to the task, time-bound, and revocable without touching anything else.

The data on where organizations stand today is not reassuring. In a CSA and Oasis Security survey of 383 organizations, 92% of IT and security professionals said they weren't confident their legacy IAM solutions could manage AI agent and non-human-identity risk, and 78% had no documented policy for creating or removing AI identities. Only 14% had fully automated identity creation and removal; 27% still did it entirely by hand, and more than 16% didn't even track when new AI identities were created. Related research from CSA Labs found non-human identities now outnumber human ones by more than 90 to 1 in many organizations, and only 28% of organizations can trace an agent's actions back to a human sponsor across all environments.

Treat this as infrastructure, not a checklist item. Some teams are now standing up dedicated non-human identity platforms rather than retrofitting human IAM, issuing scoped, auditable agent identities instead of handing agents the same API keys a person would use.

Step 4: How do you sandbox the execution environment?

Containment at the execution layer is where you get the most leverage per dollar spent. Anthropic's own internal use of OS-level sandboxing for Claude Code, isolating both filesystem and network access, reduced permission prompts by 84% while improving safety. Without network isolation, a compromised agent can exfiltrate secrets. Without filesystem isolation, it can escape the sandbox entirely.

A VM alone is not this containment. We go deeper on why in why VMs don't contain cyber-capable AI agents. AWS's own AI security framework echoes the same layered approach: infrastructure, identity and data, and application-level controls, applied together rather than any one alone.

Step 5: How do you gate irreversible actions with human review?

Sandboxing limits blast radius. It doesn't stop an agent from doing something catastrophic within its allowed scope. For that you need explicit, parameter-bound approval gates on anything destructive: database deletes, production deploys, mass file operations.

OWASP's cheat sheet is specific here: require explicit approval for high-impact or irreversible actions, and isolate or expire agent memory so a poisoned instruction doesn't persist across sessions. Replit's agent operated during an active code freeze and still deleted production data. A hard-coded approval gate, not a policy document, is what would have stopped that.

Step 6: Why treat MCP servers and third-party tools as untrusted by default?

Every plugin and MCP connector an employee adds is a new trust boundary, whether anyone evaluated it or not. Apply the same supply-chain hygiene you'd apply to any third-party dependency:

• Verify provenance before adding any MCP server or plugin to an approved list.
• Pin versions rather than auto-updating, so a trojaned release doesn't propagate silently the way postmark-mcp did.
• Require authentication on every MCP endpoint. Don't rely on the spec's optional defaults: 1,862 servers were found exposed precisely because authentication wasn't enforced.
• Run connectors in isolated processes so a compromised tool can't reach credentials or data it wasn't explicitly granted.

This is the same supply-chain discipline you'd apply to any software dependency, applied here to agent tooling.

Step 7: How do you monitor, log, and build rollback capability?

Governance doesn't end once an agent is deployed. Gartner's Level 4 tier, Act Autonomously, explicitly requires exception-based human review, which only works if you have continuous behavioral monitoring and anomaly detection running underneath it. Log every tool call, every file touched, every credential used. Build a kill switch that can suspend an agent's access instantly, and a rollback path that doesn't depend on the agent itself telling you whether rollback is possible. That's exactly the claim that failed in the Replit incident.

How do you make the secure path the easy path?

The pattern across every stat in this piece is the same: controls that create friction get routed around, and controls that match actual risk get adopted. Only 34% of organizations currently apply the same security controls to AI agents that they apply to human employees. That gap is where the next incident comes from.

The fix isn't more policy documents. It's infrastructure that makes the safe path the fast path: inventoried agents, tiered autonomy, scoped identity, sandboxed execution, and approval gates on anything irreversible. Build that once and employees stop needing to go around you to ship anything.