---
title: "How to Govern Shadow AI Tools: 5 Open-Source Fixes That Work"
url: "https://goremy.ai/blog/articles/how-to-govern-shadow-ai-tools-5-open-source-fixes"
author: Josh
editor: Josh
editor_title: Chief of Staff, GTM & Product
published: "2026-09-22T04:00:21.552Z"
updated: "2026-11-16T22:22:57.223Z"
topic: AI Governance
description: "Bans don't stop shadow AI. Here are 5 open-source tools IT leaders use to inventory, secure, and govern employee-built AI agents instead."
---

# How to Govern Shadow AI Tools: 5 Open-Source Fixes That Work

> Bans don't stop shadow AI. Here are 5 open-source tools IT leaders use to inventory, secure, and govern employee-built AI agents instead.

Edited by Josh, Chief of Staff, GTM & Product · Published 2026-09-22 · Updated 2026-11-16

## How do you govern shadow AI tools?

You govern shadow AI by making it visible, not by banning it. Block one tool and employees route around it with the next. The actual fix is giving IT the inventory, the runtime controls, and the sanctioned alternatives that make ungoverned AI unnecessary. Below are five open-source tools that do exactly that, mapped to the layer of the problem each one solves.

## TL;DR

- Banning shadow AI backfires, because provisioning sanctioned tools that meet the same need **cuts unauthorized use by roughly 89%**.
- Organizations with high shadow AI involvement paid **$670,000 more per breach** on average and took a median of 247 days to identify the incident.
- Gartner projects the average Fortune 500 company will run **more than 150,000 AI agents** by 2028, up from fewer than 15 in 2025.
- Microsoft's Agent Governance Toolkit uses an **Agent OS policy engine** that lets IT enforce rules on every agent action in real time without rewriting agent code.
- Scanning MCP servers with Cisco's MCP Scanner before approval works like a **dependency audit** for connections that have permission to act on your systems.
- Routing API traffic through a self-hosted LiteLLM gateway turns **untracked personal spend** into a governed, observable channel.

## Why don't bans on shadow AI work?

More than 80% of employees already use AI tools their organization hasn't approved. [1] Nearly all organizations, 98%, have employees running some kind of unsanctioned app, shadow AI included. [2] Only 37% of enterprises have any AI governance policy at all. [1] Just 18.5% of employees even know such a policy exists, despite 60.2% of them using AI tools at work. [1]

**The Governance Gap Driving Shadow AI**

| Metric | Share of respondents |
| --- | --- |
| Employees using unapproved AI tools | 80% |
| Enterprises with any AI governance policy | 37% |
| Drop in unauthorized use after sanctioned rollout | 89% |

_The 89% figure measures the drop in unauthorized AI use after organizations provision a sanctioned alternative, not a share of respondents in the same survey question as the other two bars._

Banning feels like the obvious fix. It isn't one. When organizations provision sanctioned AI tools that meet the same need, unauthorized use drops by roughly 89%. [1] Komprise's 2025 IT survey found nearly 80% of organizations have already had a negative AI-related data incident, and 13% say it caused measurable financial, customer, or reputational harm. [3] Employees aren't running shadow AI out of malice. They're running it because the approved tools don't do what they need yet, and the underground ones do.

**What a Shadow AI Breach Costs**

- $670,000: Additional breach cost with high shadow AI involvement
- 247 days: Median days to detect the breach

## What does shadow AI actually cost you?

IBM's 2025 Cost of a Data Breach Report treated shadow AI as its own breach category for the first time. Organizations with high shadow AI involvement paid $670,000 more per breach on average and took a median of 247 days to even identify the incident. [4] [1] [2] Of the organizations breached through an AI model or application, 97% lacked proper AI access controls, and 63% of breached organizations overall had no AI governance policy at all. [4] [2]

The shape of the risk is changing too. This isn't someone pasting a customer list into a chatbot anymore. Gartner projects the average Fortune 500 company will run more than 150,000 AI agents by 2028, up from fewer than 15 in 2025. [1] Eighty-two percent of CIOs say employees are creating AI agents and apps faster than IT can govern them. [1] Each of those agents typically talks to other tools through the Model Context Protocol (MCP), the open standard Anthropic released in late 2024 that lets an agent call external systems directly. [5] MCP was built flexible on purpose, so authentication and role-based access control are optional rather than required, and the NSA has flagged the resulting risks, including arbitrary code execution, as high severity. [5] In December 2025, OWASP responded with the first formal risk taxonomy for this category, the Top 10 for Agentic Applications 2026, covering goal hijacking, tool misuse, identity abuse, memory poisoning, cascading failures, and rogue agents. [6] That taxonomy is now the reference point the newest open-source governance tools build against.

## How can Cisco MCP Scanner audit the connections your employees already built?

Most teams didn't choose to run MCP. It arrived because someone wired an agent into Slack, a database, or an internal API, and MCP was the easiest way to do it. Cisco's open-source MCP Scanner, along with its companion Skill Scanner, A2A Scanner, and DefenseClaw tools, scans MCP servers, agent skills, and agent-to-agent traffic for vulnerabilities before they get connected to a production agent. That matters because MCP's security model leaves authentication and RBAC optional by design, so the same flexibility that made it popular also makes an unaudited server an easy path to arbitrary code execution. [5] Running a scanner before you approve a new MCP connection is the equivalent of a dependency audit for a package you're about to install, except this package has permission to act on your systems. Teams already wrestling with agents that reach into internal databases will recognize the problem, the same one covered in [5 Ways to Secure Database Access for Internal AI Agents](https://goremy.ai/blog/articles/5-ways-to-secure-database-access-for-ai-agents).

**Komprise 2025 IT Survey: Shadow AI Incidents**

- 80%: Organizations with a negative AI-related data incident
- 13%: Incidents causing measurable financial, customer, or reputational harm

## How does Microsoft's Agent Governance Toolkit act as a policy kernel for autonomous agents?

Once you know what your agents are connected to, you need a way to enforce rules on what they're allowed to do, in real time, without rewriting every agent your teams already built. Microsoft's Agent Governance Toolkit, MIT-licensed and released in April 2026, is the first open-source project to claim coverage of all 10 risk categories in the OWASP Top 10 for Agentic Applications 2026. [7] [6] It borrows an operating-system pattern: an Agent OS acts as a stateless policy engine that intercepts every agent action before execution, at sub-millisecond latency, under 0.1ms at p99. [7] Because it sits underneath the agent framework rather than inside it, it works across LangChain, CrewAI, AutoGen, and other stacks without requiring teams to rewrite agent code. [7] That's the practical value: IT gets a centrally controlled policy layer, and the developers who built the agent, sanctioned or not, keep their workflow intact. This is the same scaffolding-not-model shift covered in [The Rise of the AI Agent Harness](https://goremy.ai/blog/articles/what-is-an-ai-agent-harness-a-plain-english-explainer).

## Praxen: does this agent only do its job?

Runtime policy enforcement answers what an agent can do. It doesn't answer whether an agent, as built and deployed, actually does what its owner claims. Exabeam's open-source Praxen tool fills that gap with a discipline it calls Agent Behavior Verification. [8] [9] A team writes a plain-language "Worker Remit," a markdown document declaring the agent's authorized mission, the tools it may touch, and the actions it's forbidden from taking. [8] Praxen then reads the evidence, source code, deployment configuration, and behavioral logs, and reports the gap between what was declared and what actually happens. [8] [9] For shadow AI specifically, this is the check that catches capability creep: an agent an employee built to summarize tickets that has quietly picked up write access to a CRM nobody remembers granting. Run it before deployment and it catches misconfiguration. Run it after and it catches drift.

**Inside an AI-Related Breach**

- 97%: Breached AI systems lacking proper access controls
- 63%: Breached organizations with no AI governance policy at all

## How does a self-hosted LiteLLM gateway make shadow usage visible?

Most shadow AI isn't exotic agents. It's ordinary API calls that never touch a corporate account. 71% of generative AI connections happen through personal, non-corporate accounts, and even among connections made through corporate accounts, 58% bypass single sign-on entirely. [1] That means the CASB and SSO controls most companies rely on miss most of the traffic. LiteLLM, an MIT-licensed, self-hostable AI gateway supporting more than 100 model providers, fixes this by giving every team a single proxy to call instead of a personal API key. Issue virtual keys per team or project, set budgets, route requests, and log everything centrally, and untracked personal spend becomes a governed, observable channel. This is the same principle behind treating your AI workflow as infrastructure you run rather than a pile of vendor logins, a shift covered in more depth in [Reclaiming the Dev Cycle](https://goremy.ai/blog/articles/ai-native-development-cycle-build-one-you-actually-own).

## Why give employees a private workspace with Ollama so data stays put?

The blunt truth about shadow AI: most of it exists because employees need to think through sensitive material and the sanctioned tool is slower, worse, or nonexistent. AI is now the single largest channel for corporate-to-personal data movement, accounting for 32% of it, driven largely by employees pasting proprietary information into public tools. [1] [10] Running an open-weight model locally through a tool like Ollama gives employees the same drafting and reasoning help without the data leaving company-controlled hardware. It costs nothing per token, requires no outbound API call to audit, and satisfies the same productivity itch that sends people to public ChatGPT in the first place. Teams weighing which local models are actually good enough for daily work should start with [The Best Local Open-Source AI and Infrastructure Tools in 2026](https://goremy.ai/blog/articles/best-local-open-source-ai-infra-tools-2026).

## How do you govern shadow AI in practice?

Frameworks like NIST's AI Risk Management Framework offer templates for building the kind of continuously maintained AI inventory that policy enforcement depends on. [1] You can't govern what you can't see, and right now most companies can't see most of what's running.

**Microsoft's Agent Governance Toolkit, by the Numbers**

- 0.1ms: p99 policy-check latency
- 10/10: OWASP agentic risk categories covered

A lightweight rollout for this quarter:

1. **Inventory the MCP connections you already have.** Run Cisco's MCP Scanner or equivalent tooling against every server your agents currently call before approving new ones.
2. **Insert a policy layer, not a rewrite.** Deploy an OWASP-mapped runtime like Microsoft's Agent Governance Toolkit underneath your existing agent frameworks rather than asking teams to re-architect.
3. **Verify behavior against declared intent.** Use a tool like Praxen to write a Worker Remit for every agent with production access and check it before and after deployment.
4. **Route API spend through a gateway.** Stand up LiteLLM so personal keys stop being the default path to a model provider.
5. **Give people a sanctioned private option.** Deploy Ollama or similar for the drafting and reasoning work that would otherwise go to a public tool.

Some teams find it easier to run all five layers under one roof rather than stitching them together piecemeal. That's the gap a platform like [Remy](https://goremy.ai) is built to close: a single inventory and audit trail across the agents, MCP servers, and gateway traffic a company already has, instead of five separate dashboards. Whichever combination you pick, the sequence matters more than the tooling. See what's running, put a policy layer under it, verify it does what it claims, and give people a sanctioned place to do the work they were already going to do anyway. That's what turns shadow AI from a $670,000 liability into an inventoried, boring, auditable part of the stack.

## Frequently Asked Questions

### What is shadow AI governance?

It's the practice of discovering, inventorying, and applying policy controls to AI tools and agents employees build or adopt without formal IT approval, rather than trying to block them outright. It typically combines scanning existing connections (like MCP servers), runtime policy enforcement, behavior verification, and sanctioned alternatives to unapproved tools.

### Why doesn't banning shadow AI tools work?

Because employees adopt shadow AI to fill a real gap, not out of malice, and blocking one tool just pushes usage to another that's harder to see. Data backs this up: when organizations instead provision sanctioned tools that meet the same need, unauthorized use drops by about 89%, far more than blocking alone achieves.

### What is the Model Context Protocol and why does it matter for shadow AI?

MCP is the open standard, released by Anthropic in late 2024, that lets AI agents call external tools and systems directly. It was designed to be flexible, so authentication and role-based access control are optional rather than required, which the NSA has flagged as creating high-severity risks like arbitrary code execution when unaudited MCP servers get connected to agents.

### How much can shadow AI actually cost a company?

IBM's 2025 Cost of a Data Breach Report found organizations with high shadow AI involvement paid an average of $670,000 more per breach and took a median of 247 days to even identify the incident, compared to organizations with low or no shadow AI involvement.

### Is Ollama safe to use for sensitive company data?

Running open-weight models locally through Ollama keeps data on company-controlled hardware instead of sending it to a third-party API, which directly addresses the largest driver of AI-related data exfiltration: generative AI now accounts for 32% of all corporate-to-personal data movement, mostly through public tools.

## Sources

1. [Shadow AI Apps: The Enterprise Attack Surface That Outpaces Monitoring](https://labs.cloudsecurityalliance.org/research/csa-research-note-shadow-ai-apps-enterprise-20260530-csa-sty/) (Cloud Security Alliance (CSA Labs))
2. [Shadow AI Statistics: How Unauthorized AI Use Costs Companies](https://programs.com/resources/shadow-ai-stats/) (Programs.com)
3. [Shadow AI: The hidden agents beyond traditional governance](https://www.cio.com/article/4083473/shadow-ai-the-hidden-agents-beyond-traditional-governance.html) (CIO.com)
4. [What data leaders need to know from the Cost of a Data Breach Report 2025](https://www.ibm.com/think/insights/data-matters/cost-of-a-data-breach) (IBM)
5. [Model Context Protocol (MCP): Security Design Considerations for AI-Driven Automation](https://www.nsa.gov/Portals/75/documents/Cybersecurity/CSI_MCP_SECURITY.pdf) (National Security Agency (NSA))
6. [OWASP Top 10 for Agentic Applications for 2026](https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/) (OWASP GenAI Security Project)
7. [Introducing the Agent Governance Toolkit: Open-source runtime security for AI agents](https://opensource.microsoft.com/blog/2026/04/02/introducing-the-agent-governance-toolkit-open-source-runtime-security-for-ai-agents/) (Microsoft Open Source)
8. [Praxen: Open-source AI agent behavior verification](https://www.helpnetsecurity.com/2026/06/24/praxen-open-source-ai-agent-behavior-verification/) (Help Net Security)
9. [Exabeam Launches Open Source Praxen to Bring Agent Behavior Verification to AI Agents and Digital Workers](https://www.businesswire.com/news/home/20260623496656/en/Exabeam-Launches-Open-Source-Praxen-to-Bring-Agent-Behavior-Verification-to-AI-Agents-and-Digital-Workers) (Business Wire (Exabeam))
10. [15 Private ChatGPT Alternatives That Don't Train on Your Data](https://www.premai.io/blog/15-private-chatgpt-alternatives-that-dont-train-on-your-data/) (Prem AI)
