The Best Tools for AI Coding Agents Aren't the Fancy Ones

The best tool for most AI coding agent tasks is still grep. Not a language server, not embeddings, not a semantic index. Plain text search. That's not a hot take. It's what three separate measurement efforts found when they watched agents work, and it's what the teams behind Claude Code, Cursor, and Codex have said outright about their own products.¹²³

Why do agents keep choosing grep over LSP?

Give an agent a codebase and a hard question, and it will grep first. Give it a choice between grep and a language server protocol (LSP) integration that understands the actual symbol graph, and it will still grep most of the time.² This holds even though LSP is objectively more precise at some tasks. The pattern shows up in a 150-cell benchmark across ten major open-source repos, in a dedicated token-efficiency study across three model tiers, and in public statements from the people building these agents for a living.

Boris Cherny, who works on Claude Code, put it plainly: "Claude Code's agentic search is really just glob and grep, and it outperformed RAG."³ That's not a limitation the team is embarrassed about. It's a design decision backed by internal testing.

What the data actually shows

The clearest picture comes from a benchmark that ran Claude Sonnet across ten major repos — Redis, Django, Bitcoin, Laravel, Rails, TypeScript, webpack, Spring Boot, Tokio, and Hugo — at five complexity tiers, producing 150 harvested test cells comparing grep, tree-sitter-based structural navigation, and LSP.¹ The headline result: grep gets you almost the same answer quality as the fancier tools. Grounding scored 0.97 and completeness 0.99, a near-tie across all three arms.¹ Grep is enough to be correct.

Where it isn't enough is efficiency. At the hardest task tier, plain grep-and-read search burned roughly 2.8x more context tokens than the structural tree-sitter approach — 1.5 million tokens against 534,000.¹ So the story isn't "grep wins." It's "grep wins on correctness, loses on cost, at scale."

A second study sharpens that further. Researchers ran Claude Opus, Sonnet, and Haiku through localization tasks (find the right spot to make an edit) and reference-completeness tasks (find every call site), tracking both forced and free-choice tool use.² On localization tasks, LSP actually cost more tokens than grep — 6% more for Opus, a startling 118% more for Sonnet.² Agents figured this out on their own: given free choice, they used LSP tools on localization tasks only 0-6% of the time.² But flip the task to reference-completeness, and the same agents reached for LSP voluntarily 45-57% of the time.² The grep habit isn't a fixed bias. It's a learned, task-shaped policy.

Why do agents default to grep?

Put the benchmark data and vendor disclosures together, and five structural reasons explain the default:

1. Zero setup. Grep works the instant a repo is checked out. No indexing step, no language server to configure, no risk of a broken toolchain blocking the agent's first move.
2. Universal file coverage. Grep doesn't care if it's scanning source code, YAML configs, log files, or a stray comment. LSP only understands the language it's configured for.⁴
3. Benign failure mode. A bad grep pattern returns zero or noisy results, and the agent tries again. A misconfigured or crashed language server can fail silently, quietly bouncing the agent back to grep without anyone noticing until the token bill shows up.⁴
4. Shell-native composability. Grep chains naturally into find, sed, awk, and pipes — the exact vocabulary agents were trained on from millions of terminal sessions and shell scripts.
5. Training-data prior. Every major agentic coding product surveyed — Claude Code, Codex CLI, OpenCode, Cursor, Continue, and Aider — defaults to grep/ripgrep as its primary search mechanism.³ Agents trained on transcripts of these tools inherit that same default.

Augment's engineering team, whose agent was a top performer on SWE-bench Verified, found the same thing from a different angle: "We explored adding various embedding-based retrieval tools, but found that for SWE-Bench tasks this was not the bottleneck — grep and find were sufficient," according to their engineer Colin.⁵ Cursor hit a related wall on huge monorepos, where single ripgrep calls exceeded 15 seconds, and their fix wasn't to switch to vector search or LSP. It was to build a custom sparse n-gram index that makes regex search itself faster.³ The industry keeps optimizing the grep layer instead of replacing it.

Where LSP actually earns its keep

None of this means LSP is useless. It means LSP's value is precision, not efficiency, and that value is concentrated in specific task shapes.

On reference-finding tasks — find every place a function is called — LSP-based retrieval hit perfect precision, 1.00, with zero false call sites, against grep's 0.76 precision, meaning roughly a quarter of grep's matches were false positives.² It didn't improve recall, and it cost 12-19% more tokens for capable models. The token savings from LSP showed up only for the weakest model tested, Haiku, where semantic retrieval acted as a crutch against lexical noise, saving 7-26% of tokens.²

There's also a codebase-quality signal in the data that matters more than language choice. Across two same-language TypeScript repos, LSP delivered zero accuracy improvement on a clean, well-named codebase (remeda) but the single largest accuracy gain in the whole study, +0.246 F1, on a noisier one (hono).²⁶ The rule isn't "turn on LSP for TypeScript." It's "turn on LSP when grep starts returning garbage."

And forcing the issue backfires. When researchers mandated LSP-first search on localization tasks, task success dropped from 100% with free choice to 89%.² Heavy-handed tool mandates can hurt agents more than they help, which is the same lesson behind why rules baked into a CLAUDE.md or AGENTS.md file don't guarantee agent behavior — dictating a path doesn't make it the right one.

The real lesson: layered retrieval, not tool maximalism

The practical takeaway for internal platform teams isn't "buy the semantic tooling" or "skip it entirely." It's building a layered toolchain that matches how agents actually search, then letting the agent pick the right layer for the task:

• Text layer (grep/ripgrep). The default for everything that isn't structured code — logs, configs, string literals, comments, feature flags — and for broad, whole-tree sweeps.⁴
• Structural layer (tree-sitter, ast-grep). A middle ground that understands syntax without a full symbol graph. It reached the same grounding and completeness as LSP in the benchmark at meaningfully lower token cost.¹
• Symbol layer (LSP). Reserved for tasks that actually need call-site precision — renames, reference-completeness checks, and noisy or poorly-named codebases where grep's false-positive rate climbs.²
• Semantic layer (embeddings/vector search). Least useful for source-code navigation on small-to-medium repos, per Augment's findings, but this changes at scale — beyond roughly a million documents, grep-only search degrades on latency, recall of conceptual synonyms, and signal-to-noise.⁵⁷ It's the same threshold that pushes teams toward hand-rolled retrieval pipelines rather than renting the whole stack from a heavyweight RAG framework.

This is worth stressing: the grep-beats-everything result is a source-code-specific finding, not a blanket argument against retrieval infrastructure. For enterprise document search or large unstructured corpora, the calculus flips.⁷ A related arXiv study running grep versus vector retrieval across Claude Code, Codex, and Gemini CLI on a memory-recall benchmark found grep generally more accurate, but also found that results swing heavily based on which harness and tool-calling style is used, even on identical underlying data.⁸ The harness matters as much as the retrieval method, which is the whole idea behind treating the scaffolding around a model as the real product rather than the model itself.

What should you actually install for your coding agents?

1. Start with grep/ripgrep, always. It's free, zero-config, and covers non-code content LSP will never touch.
2. Add a structural layer (tree-sitter or ast-grep) before you add LSP. It closes most of the token gap at far lower setup cost.¹
3. Turn on LSP for your primary language stack, but verify it's actually firing. Silent fallback to grep after an LSP misconfiguration is the single biggest hidden cost teams report.⁴
4. Don't force tool order. Let the agent choose. Mandating LSP-first on simple tasks measurably hurt success rates in testing.²
5. Save embeddings for scale, not source code. They earn their cost past roughly a million documents, or in large, less-structured codebases well beyond what SWE-bench-style benchmarks capture.⁵⁷

Do you need to buy semantic search tooling, or build your own?

There's a broader lesson here for teams evaluating "AI-ready IDE" products that advertise elaborate semantic indexing, embeddings pipelines, or proprietary code-graph engines as their headline feature. The measurement studies above suggest a lot of that infrastructure is solving a problem coding agents already handle cheaply with commodity tools. If your agent's real behavior is grep-first with LSP as a targeted assist, you don't need to rent a heavyweight platform to get there. You need ripgrep, a structural parser, and a properly wired language server, the same kind of stack a team can run and own on its own infrastructure instead of paying a subscription to abstract away. That's the same logic behind building an AI-native workflow you actually own instead of stacking unmanaged vendor tools: match the toolchain to what the agent actually does, not to what the sales deck promises. Teams running agents through platforms like Remy can apply the same layered logic directly, configuring the retrieval stack around observed agent behavior instead of a vendor's assumptions about what "semantic" search should look like.

The agents already figured out what works. The job now is building infrastructure that matches their habits, not fighting them.