How to Secure Internal AI Agents: Engineering Beats Alignment

You secure internal AI agents the way you secure any high-privilege, unowned software: scoped credentials, sandboxed execution, and hard-coded limits on what actions are even possible. Not by asking the agent nicely to behave. Not by publishing a policy document nobody enforces.

That's the whole argument here. Most enterprise AI governance runs on alignment: system prompts, RLHF tuning, acceptable-use policies. Alignment asks a model to choose safety. Engineering removes the choice. The research is now clear on which one holds up.

Why doesn't alignment work as a security control?

Start with the number that should worry any security lead: large reasoning models acting as autonomous adversaries against other AI models hit a 97.14% jailbreak success rate across nine widely used target models, in multi-turn conversations with no human supervision after the initial prompt.¹ That's not a clever one-off exploit. It's a repeatable process. The researchers call it an "alignment regression": one model systematically erodes another's guardrails, turning jailbreaking from a bespoke, labor-intensive exercise into a scalable, commodity capability.¹

The pattern holds outside the lab too. Independent compilations put average single-turn jailbreak success around 20%, but multi-turn attacks push that to 92.78% in enterprise model testing, with attackers needing only 5 to 7 prompt iterations and about 42 seconds to succeed.² Palo Alto Networks' Unit 42 tested 17 popular generative AI web products, expecting the consumer-facing layer to add real protection beyond the base model. Every one was still vulnerable to jailbreaking, and most were vulnerable to multiple distinct strategies.³

Add it up and the conclusion is unavoidable: alignment isn't a security control. It's a speed bump. Treat it as your primary defense and your primary defense fails most of the time it's actually tested.

Why employee-built agents make this worse

This would be a manageable problem if AI agents stayed confined to a handful of vendor products your security team already reviewed. They don't. Seventy-eight percent of AI users at work now bring their own AI tools outside IT approval, a bring-your-own-AI pattern Microsoft's Work Trend Index has tracked since 2024.⁴ Increasingly those tools aren't chatbots. They're agents wired into internal APIs, databases, and file systems, built by employees on no-code platforms and coding assistants, often in an afternoon.

The cost of letting that run ungoverned is measurable. IBM found shadow AI was a contributing factor in one in five breaches studied, and organizations with high levels of shadow AI absorbed an average of $670,000 in additional breach costs compared to those with low or no shadow AI use.⁵ MIT's research on enterprise generative AI adoption found 95% of organizations see zero measurable P&L return from their AI pilots, largely because ad hoc, ungoverned deployments produce brittle workflows that don't hold up in production.⁶ Shadow AI is failing on business grounds and security grounds at once, which is exactly what you'd expect from deployments with no owner and no engineering discipline behind them. We've written about the governance model that keeps up with it in Shadow AI Governance.

From chat window to blast radius

A jailbroken chatbot produces bad text. A jailbroken agent with tool access, stored credentials, and API calls produces unauthorized actions: a database query that shouldn't have run, a file that shouldn't have been read, a transaction that shouldn't have gone out. That's the qualitative shift OWASP's new Top 10 for Agentic Applications is built around. Developed with more than 100 industry experts, it names excessive agency, prompt injection, sensitive data exposure, and insufficient logging as the leading risks specific to agentic systems, distinct from and more consequential than the risks in a plain chatbot deployment.⁷

The consequence side isn't abstract. Thirteen percent of organizations IBM surveyed reported a breach involving an AI model or application.⁵ Breach cost scales with what the compromised system can actually do, and an agent with standing database access or write permissions on production systems can do a lot. We've written separately about what that looks like when the target is a company database in 5 Ways to Secure Database Access for Internal AI Agents.

The governance gap: policies on paper, nothing in the stack

Here's the diagnosis that matters most for anyone running security or IT at a mid-size company: this isn't primarily an awareness problem. Everyone knows AI agents are a risk. It's an enforcement problem.

IBM's 2025 Cost of a Data Breach Report found 63% of breached organizations either had no AI governance policy at all or were still developing one.⁵ Worse: of the organizations that suffered an AI-related breach specifically, 97% had no AI access controls in place.⁵ Read those two numbers together and the picture is stark. Most organizations relying on a policy document and vendor assurances have no technical enforcement layer underneath it. The policy says the agent shouldn't touch sensitive tables. Nothing in the infrastructure actually stops it.

What does deterministic engineering for AI agents look like?

The fix has a name, and it's not new: least privilege, applied to software instead of people. Security research and vendor guidance now converge on roughly the same architecture for agentic systems:

1. Least-privilege scoped credentials. Give each agent only the permissions its task requires, never a standing admin key it happens not to need most of the time.⁸⁹
2. Short-lived, ephemeral tokens. Credentials expire fast, so a leaked or hijacked token has a small window to do damage.⁸
3. Sandboxed or microVM-isolated execution per session. Each run happens in its own isolated environment, not a shared process with access to everything else on the box.⁹
4. Information flow control. Untrusted content the agent reads (a web page, an email, a support ticket) is isolated from the channel that issues commands, so injected instructions can't hijack the agent's next action.⁸
5. Tool-chain and action allowlists. The agent can only call the specific tools and endpoints it's been explicitly permitted to use, nothing else.⁷⁹
6. Human-in-the-loop gates for irreversible actions. Anything that deletes, sends, pays, or publishes gets a mandatory approval step before it executes.⁸
7. Full audit logging. Every action is recorded, attributable, and reviewable after the fact, closing the insufficient-logging gap OWASP flags as a top agentic risk.⁷⁹

None of this depends on the model behaving well. That's the point: boundary-setting at the infrastructure layer, enforced regardless of what the model decides to do. If you want the deeper mechanics of why isolating the process alone, a sandbox with no action-level controls, still isn't enough, we cover that in Why Sandboxes Won't Save You From Shadow AI.

Defense in depth, not a silver bullet

None of this makes alignment work worthless. Prompt shields, output filters, and plan-drift detection still catch a meaningful share of attacks, and they're cheap to run. Microsoft's own Secure Future Initiative guidance is explicit: design systems assuming indirect prompt injection will happen and some attacks will succeed, and layer deterministic controls, like least privilege and short-lived credentials, underneath probabilistic ones, like prompt shields.⁸ Note the order. The deterministic layer is the floor. The probabilistic layer is a bonus on top of it, not a substitute for it. Coming from the vendor that ships the models, that's a fairly candid admission that alignment-only defense doesn't hold on its own.

This is an ownership problem, not just a security problem

Strip away the security framing and this looks like a familiar problem to anyone who's thought hard about renting versus owning software. The SaaS-era trust model was simple: the vendor says it's secure, you take their word for it, and you inherit the risk when they're wrong. Model-vendor alignment promises are the AI-agent version of that same bargain. You're trusting a third party's tuning process to hold up against a jailbreak technique it hasn't seen yet.¹³

Owning the engineering controls around your agents flips that. Scoped credentials, sandboxing, and action allowlists are things you build and audit yourself, not things you take on faith from a model provider's release notes. That's the same accountable, inspectable posture this publication argues for when it comes to owning software generally, and it's exactly the gap tools built for governing agent identity and access, like Remy, exist to close. If you're trying to formalize this across every agent employees have already stood up, the practical framework is laid out in How to Manage AI Agents in the Enterprise.

How to secure internal AI agents: a practical checklist

If you're starting near zero, here's the order that matters most:

1. Inventory every agent with tool or API access, sanctioned or not. You can't scope a credential for something you don't know exists.
2. Replace shared or standing credentials with scoped, short-lived tokens tied to a specific task and a specific agent identity.⁸
3. Isolate execution per session, ideally in sandboxed or microVM environments, so a compromised run can't reach anything outside its own box.⁹
4. Separate untrusted input from command channels so content an agent reads can't be interpreted as an instruction to act.⁸
5. Allowlist the specific tools and endpoints each agent can call. Default to nothing, add access deliberately.⁷
6. Require human approval before irreversible actions: deletions, payments, external sends, production deploys.⁸
7. Log every action with attribution and review those logs on a real cadence, not just after an incident.⁷⁹
8. Treat prompt-level safety as a second layer, not the first. Keep it running, but don't let it be the only thing standing between an agent and your data.⁸

The jailbreak research isn't going to get more forgiving. Models will keep getting better at persuading each other to misbehave, and the cost of running that persuasion at scale keeps falling.¹² The organizations that hold up won't be the ones with the best-worded policy. They'll be the ones that stopped trusting the model to enforce its own boundaries and built those boundaries into the infrastructure instead.