Why Rewrite AI Agents in Rust? Python vs Rust for Agents

Why are teams rewriting AI agents in Rust instead of Python?

Teams are rewriting AI agents in Rust because thin Python wrappers hit real limits at scale. The GIL blocks parallel CPU work, spawning a process per agent burns memory, and neither problem shows up until you're running hundreds of agents at once. AI coding agents have also made Rust ports cheap enough that owning the runtime is now a rational default, not a luxury reserved for infrastructure teams.

This isn't theoretical. It's already playing out across the industry's biggest AI tooling projects.

Why are Rust rewrites suddenly everywhere in AI tooling?

OpenAI rewrote its Codex CLI from TypeScript and Node.js into Rust in 2025. The team cited a zero-dependency install, native security sandboxing bindings, lower memory consumption from removing runtime garbage collection, and a more extensible wire protocol as the drivers.

GitHub went further. It rewrote the entire Copilot agent runtime, the shared backend behind Copilot CLI, the Copilot app, the SDK, VS Code, Visual Studio, Copilot Code Review, Copilot Cowork, Copilot Studio, and the Office apps, into more than 800,000 lines of production Rust.

It's not limited to agent runtimes. Pydantic rewrote its core validation engine in Rust for V2 and measured a 5 to 50x speedup over the pure-Python V1. Hugging Face's tokenizers library, Rust under the hood with Python and Node bindings, tokenizes a gigabyte of text in under 20 seconds on a standard server CPU. The Rust Foundation now frames Rust as the natural fit for real-time inference and edge AI deployment, while acknowledging Python still owns large-scale model training.

The common thread: wherever an AI tool stopped being a prototype and started being infrastructure, teams hit the same wall and reached for the same fix.

Why do Python wrappers break down at agent scale?

A single Python agent calling an LLM API feels fine. The trouble starts when you run many of them concurrently and expect your hardware to keep up.

Python's Global Interpreter Lock serializes CPU-bound work across threads. Multi-threaded Python agents doing any real "thinking" work beyond waiting on network calls don't scale with core count the way you'd expect. Red Hat's own benchmark makes this concrete: a two-threaded Python job summing primes ran slower than the single-threaded version, because the GIL forced the threads to take turns instead of running in parallel. The same workload in Rust, using the Rayon library for actual parallelism, cut runtime roughly in half.

The second wall is architectural. Many Python and Node agent frameworks isolate each agent in its own process to dodge the GIL or sandbox risky code. That buys safety but costs memory and startup latency per agent, and it compounds fast once you're running dozens or hundreds of agent instances at once.

Neither problem is a bug. They're design tradeoffs that made sense when an agent was a side project. They stop making sense when the agent is the product.

What does Rust actually fix in performance, memory, and security?

Teams that make the switch tend to cite the same three gains, and the numbers behind them are specific:

• Throughput. GitHub's benchmarks on the rewritten Copilot runtime found it completing 120 one-turn session lifecycles per second running in-process, versus 7.55 per second for the original TypeScript implementation. That's a 15.9x speedup.
• Memory footprint. A 10-client batch of agents consumed 1,383MB of memory under the TypeScript runtime. The same workload under Rust used 126MB, roughly a tenth as much.
• Memory safety at compile time. Microsoft has attributed close to 70% of its historical CVEs to memory-safety bugs like buffer overflows and use-after-free errors, a figure that has held for over a decade and now underpins NSA and CISA guidance pushing memory-safe languages. Rust's ownership model catches most of that class of bug before the code ever ships.

None of these gains require exotic hardware or a research team. They come from the language itself: no garbage collector pausing your runtime, no interpreter lock serializing your threads, and a compiler that refuses to let a whole category of memory bugs through.

How did GitHub rewrite the Copilot agent runtime in Rust?

The clearest worked example of this shift is GitHub's own migration, and the economics are the real story.

GitHub didn't hand the rewrite to a large team over a year or two, which is roughly what the pre-agent estimate would have required. Instead, it used the Copilot app and Copilot CLI to port the runtime incrementally, in place, merging 128 pull requests over about 14.5 weeks. AI agents wrote most of the code. By the end, the runtime was 832,378 lines of production Rust plus 468,689 lines of Rust unit tests, entirely replacing the TypeScript and Node.js implementation.

The total cost: roughly $120,000 in AI token usage, plus about three weeks of one developer's focused time. GitHub was explicit that this wasn't a verdict that every TypeScript program should become Rust. The reasons were specific to this runtime: minimal dependency overhead, the ability to embed in-process rather than being forced out-of-process, strong FFI across six SDK languages, and a lower supply-chain-risk security posture.

Why wasn't this rewrite affordable before AI agents?

This is the part that matters beyond GitHub's specific case. A systems-language rewrite of an 800,000-line production runtime used to be a multi-quarter bet most teams would never greenlight, because the payoff had to justify a year or more of senior engineering time. AI coding agents collapsed that cost. GitHub's own estimate puts the agent-assisted effort at a small fraction of what a human-only team would have spent.

That changes the calculus from "rent a wrapper and accept its ceiling" to "build and own a faster, safer agent runtime," which is exactly the kind of decision our build-versus-buy coverage of AI agents replacing software keeps surfacing: the cost of ownership keeps dropping while the cost of staying on rented infrastructure's limits stays fixed.

The same forces are reshaping desktop tooling, not just backend runtimes. Teams that once shrugged at bloated Electron apps are now rewriting internal tools in Rust for the same reasons: memory bloat finally became unacceptable once the cost of fixing it dropped.

What are the risks of an agent-driven Rust rewrite?

A faster, safer runtime isn't the same as a correct one, and an agent-driven rewrite introduces failure modes that didn't exist when humans wrote every line.

• Passing the compiler isn't passing correctness. GitHub's own postmortem on the Copilot port found dozens of regressions from ambiguous semantics and branch drift, and missed features, even though every line compiled cleanly. The compiler caught memory bugs. It didn't catch logic bugs. End-to-end tests, not the compiler, were the real check on correctness.
• Provenance and licensing don't survive translation by default. In early 2026, an agentic LLM system ported the Python-based ScanCode Toolkit license scanner to Rust, published it under a name that infringed the ScanCode trademark, stripped copyright and license notices from both ScanCode and vendored third-party code, and published 10x to 100x performance claims that didn't hold up under correctness testing. LLM-driven code translation doesn't track attribution unless someone explicitly builds that step in. An agent-assisted rewrite can quietly carry unresolved licensing obligations even when the output looks original.
• Ungoverned agent-built software is a measurable breach risk. IBM's 2026 Cost of a Data Breach Report found shadow AI incidents more than doubled year over year, now involved in 43% of breached organizations, up from 20%, and those incidents cost an average of $5.39 million against a global average of $4.99 million. A Rust rewrite done outside normal review, by an agent nobody is tracking, is exactly the kind of shadow build this stat describes. Our governance framework for employee-built agents covers how to bring that kind of build back under review without killing the speed advantage.

How do you decide whether your team should rewrite in Rust?

Most teams don't need a full runtime rewrite on day one. The pragmatic path most agentic AI developers follow is hybrid: prototype the agent logic in Python's rich AI ecosystem, profile for the actual CPU or network bottleneck, then rewrite only the slow component in Rust and expose it back to Python using a binding tool like PyO3.

Use this to decide which path fits:

1. Agent count. If you're running a handful of agents, process overhead and GIL contention are noise. If you're running dozens to hundreds concurrently, the per-instance memory and throughput gap compounds quickly, as GitHub's 126MB-versus-1,383MB comparison shows.
2. CPU-bound work inside the agent loop. If your agent is mostly waiting on an LLM API, Python's threading model is fine. If it's doing heavy local computation, parsing, scoring, or orchestration logic between calls, that's the part the GIL will punish, and the part worth porting first.
3. Security and compliance exposure. If the agent handles sensitive data or runs in a regulated environment, the roughly 70% memory-safety-CVE baseline that NSA and CISA guidance is built around is a real argument for Rust's compile-time guarantees, not just a performance one.
4. In-house ownership goals. If you intend to run this agent runtime for years and want to control its roadmap, security posture, and cost structure, the economics have shifted in your favor. What used to cost a team a year or two of effort now costs a fraction of that in AI tokens and a few weeks of focused engineering time, as GitHub's own numbers show.

None of this makes Rust the right choice for every script you own. It makes it the right question to ask the moment a Python wrapper stops being a prototype and starts being something your product depends on.