---
title: "Should Companies Run Open-Weight Models to Protect Their Data?"
url: "https://goremy.ai/blog/articles/open-weight-models-enterprise-data-protect-sops"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-11-21T12:00:16.123Z"
updated: "2026-11-21T12:00:16.123Z"
topic: Local AI
description: Why companies like AT&T are moving AI work to open-weight models, what knowledge alpha means, and how to split sensitive and routine work across models.
---

# Should Companies Run Open-Weight Models to Protect Their Data?

> Why companies like AT&T are moving AI work to open-weight models, what knowledge alpha means, and how to split sensitive and routine work across models.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-11-21

## Should companies run open-weight models to protect their data?

For sensitive work, often yes. An open-weight model is one you download and run on your own servers, so your prompts, documents and customer data never travel to an outside AI company. That removes the biggest exposure: your processes sitting in someone else's logs. It does not make sense for everything. The practical answer for most companies is a split. Use self-hosted open models for work that touches your proprietary processes and customer data, and use a hosted model like Claude or ChatGPT for lower-stakes tasks.

## TL;DR

- **Open-weight models run on your own hardware**, so no requests or data go to an outside AI provider.
- The asset worth protecting is what one commentator calls **knowledge alpha**: your SOPs, prompts, customer data and edge-case know-how.
- **Opting out of training helps, but it is not the whole story.** It stops your chats from being used to train models. It does not change the fact that your content passes through someone else's systems.
- AT&T reportedly sends about **25% of its AI usage to open models** and aims for 70 to 80%. Survey figures cited on the topic suggest many other companies are moving the same way.
- A workable setup is **open models for sensitive work, a hosted model for the rest**, with a tool that lets you switch between them.
- Before you move anything, **read your own contract and plan terms**. They may already cover what you are worried about.

## What is an open-weight model?

An AI model is a large file of numbers (the "weights") that determines how it responds. Most famous models, such as Claude and ChatGPT, are closed. You can only use them through the provider's app or API (the way software sends requests to a service), and the provider keeps the weights.

Open-weight models publish those weights. You can download them and run them on your own servers or rented cloud machines. Nothing needs to be sent to an outside provider, because the model lives inside your own environment. Your IT team controls who can use it, what gets logged and where the data goes.

"Open-weight" is not quite the same as "open source." You get the finished model, but not necessarily the training data or full recipe, and licenses vary. Read the license before you build on one.

## What is "knowledge alpha" and why does it matter?

"Alpha" is a finance word for an edge over the market. Knowledge alpha is the business version: the information that makes your company hard to copy. The commentator who popularized the phrase lists:

- Your standard operating procedures (SOPs)
- Your prompts
- Your customer data
- Your knowledge of edge cases, collected over years

Think of an insurance claims team that has spent five years learning which odd claims get approved and why. That knowledge is not in any public dataset. If someone pastes the full playbook into a chatbot to "make it more concise," the playbook now exists inside a transcript on someone else's system.

## Doesn't opting out of training solve this?

Only partly. On many consumer plans, the terms say your chats may be used to develop products and services, including training models, unless you opt out. The major providers have similar settings. So turning that off is a sensible first step.

But training is only one risk. The argument made by critics is that even without training, a provider can see how people use its products: which tasks come up, which plugins get used, what people are asking for. AI labs now ship products for finance, legal work, sales, design and more, and some publish analyses of how businesses use their models. Anthropic's Economic Index, for example, studies large numbers of business conversations to understand what tasks AI does.

Be careful here. Nobody has shown that a particular provider copied a particular customer's SOP. The concern is about incentives and exposure, not proven misuse. Also, business and API agreements often differ from consumer terms, sometimes with stronger data commitments. Check the plan your company actually pays for before drawing conclusions.

The cautious position is simple. If a document is the core of your competitive edge, the safest place for it is a system you control.

## Who is already moving to open-weight models?

AT&T is the example most often cited. According to the commentary behind this topic, about 25% of its AI usage already goes to open models, and the stated goal is 70 to 80%. A large infrastructure company handling software and customer data has obvious reasons to worry about leakage.

Survey figures cited in the same discussion say:

- At least 40% of companies surveyed want open-weight models in production.
- 60% have already moved some work off their current systems onto open-weight models, specifically to avoid data exposure.

These numbers come secondhand, and the underlying surveys were not provided, so treat them as directional. They are consistent with something IT leaders say openly: many want more control over where sensitive data goes.

## Is running your own model cheaper?

It can be, at scale. Using the most capable hosted models for large volumes of work gets expensive. When you run models on your own compute, you can batch many requests together and use the hardware more efficiently, which can lower the cost per task.

The catch is that you take on the work. You pay for hardware or cloud machines, someone has to deploy and maintain the model, and the open models available may not match the best closed ones on every task. For a small team with light usage, a hosted model with strong contract terms is probably cheaper and simpler. Self-hosting starts to pay off when volume is high, the data is sensitive, or both.

## How do you set up open models for sensitive work and Claude for the rest?

The approach described is "hot swapping": keep an open-weight model for sensitive work and switch to a hosted model when the task is low-risk. Many coding and chat tools now let you change models with a single command, so you can move between a private model and Claude or ChatGPT without changing your whole workflow.

A practical way to set it up:

1. **Sort your information into tiers.** Public or low-risk (marketing drafts, general research), internal (meeting notes, routine emails), and crown jewels (SOPs, pricing logic, customer records, unreleased plans).
2. **Assign a model to each tier.** Crown jewels go to the self-hosted open model. Low-risk work can go to a hosted model.
3. **Choose a tool that switches models easily.** Look for one where changing models is a setting or a command, not a rebuild.
4. **Start a fresh conversation when you switch.** This keeps sensitive context from carrying over into a hosted model's transcript.
5. **Write the rule down.** A one-page policy ("never paste X into tool Y") does more than a clever setup nobody follows.

| Type of work | Where it can run | Why |
|---|---|---|
| Rewriting a public blog post | Hosted model | Nothing sensitive in it |
| Summarizing a general industry report | Hosted model | Public information |
| Turning your internal SOP into training material | Self-hosted open model | The SOP is your edge |
| Analyzing customer records | Self-hosted open model | Customer data and privacy duties |
| Brainstorming meeting agendas | Either | Low stakes |

The commentator admits this is annoying to do. Open models can lag on quality, and switching adds friction. Expect to spend time testing which tasks the open model handles well enough.

## What does this mean for you?

**If you are an employee:** Learn what your company allows. Do not paste SOPs, customer lists or internal pricing into a personal chatbot account, even with good intentions. That is how process knowledge leaves the building. If you want to use AI on sensitive material, ask IT whether an approved private option exists.

**If you manage a team:** Name your crown-jewel documents and decide which can never go into a hosted tool. Ask which plan your team uses and whether training is turned off. Check what your company's contract says about data retention.

**If you influence IT or procurement:** Pilot an open-weight model on one sensitive workflow, such as internal document search or drafting from your SOPs. Measure quality and cost against the hosted option before committing. Ask vendors for written terms on retention and training.

**If you are building a product on top of someone else's model:** The commentary's warning is that thin "wrapper" products are vulnerable when model providers ship similar features. Depth, regulated work, physical-world work and narrow niches hold up better.

## Can you build your own private tools instead of pasting SOPs into a chatbot?

Often the real reason people paste a process into a chatbot is that they have no internal tool for it. A small app that applies your SOP, such as an intake form that checks requests against your rules or a dashboard that answers questions from your documents, keeps that logic in your own system. [Remy](https://goremy.ai) builds and deploys full-stack apps from a plain-English description, and each app is standard TypeScript in a git repo you own. Remy lets you bring your own provider keys, register custom or self-hosted models, and reach local models over a secure tunnel, so a sensitive workflow can point at a model you run. Whether that fits your security requirements is something to check with your IT team.

## Frequently Asked Questions

### What is the difference between open-weight and closed models?
Closed models, like Claude and ChatGPT, are only available through the provider's service. Open-weight models publish their weights so you can download and run them yourself. The tradeoff is control and privacy on one side, and convenience and often top-end capability on the other.

### Does turning off training mean my data is safe with a hosted AI?
It means your chats are not used to train the provider's models. Your content still passes through and may be stored on their systems under their retention terms. Review your plan's agreement for details on storage, access and retention.

### Do I need to move everything to open models?
No. The approach described is a split. Keep sensitive work, like SOPs and customer data, on models you control, and use hosted models for everyday, low-risk tasks where their quality and convenience help.

### Are the AT&T and survey numbers reliable?
They come from commentary rather than the original reports, so treat them as indicators. The reported figures are about 25% of AT&T's AI usage on open models with a 70 to 80% goal, and survey results of at least 40% wanting open weights in production and 60% having moved some work.
