Framework-Free RAG and AI Agents: Why Developers Are Ditching LangChain
Chunk, embed, search, ask. That's the whole pipeline. Here's why more developers are building it in plain Python instead of renting it from a $1.25 billion abstraction layer.
- 01Anthropic found the most successful AI agents avoid complex frameworks and use direct API calls.
- 02Teams using agent frameworks often hit a hard 70-80% quality ceiling that is difficult to debug.
- 03LangChain's $1.25 billion valuation signals a shift from open-source tool to metered product.
- 04Framework-free RAG requires only five plain Python libraries to chunk, embed, search, and ask.
Framework-free RAG and agents means building retrieval and reasoning pipelines with plain code: direct API calls, a vector database, and your own control flow, instead of routing everything through a framework like LangChain or LangGraph. It's not a fringe move. Anthropic's engineering team documented it after watching dozens of production agent teams work.1 The 12-Factor Agents project turned it into a named methodology with 25,500 GitHub stars.2 Developer tutorials now walk through full RAG builds that never import an orchestration library.3 This is a pattern with a paper trail, not a hot take.
Why is 'framework-free' suddenly the answer?
The search interest tracks a real shift in how production agents get built. Anthropic's team, after working with dozens of teams shipping LLM agents, found something counterintuitive: the most successful implementations weren't using complex frameworks or specialized libraries at all.1 They were composing simple patterns directly against the model API.
The 12-Factor Agents author put it more bluntly. He'd tried every major agent framework, then advised more than 100 startups building customer-facing agents. His conclusion: "Most of them are rolling the stack themselves. I don't see a lot of frameworks in production customer-facing agents."2 That's not an opinion piece. That's a field report.
The vendor lock-in problem hiding inside agent frameworks
LangChain started as a free, open-source library. It isn't just that anymore. In October 2025, LangChain raised $125 million at a $1.25 billion valuation to build what it calls "the platform for agent engineering": observability, deployment, sandboxes, an LLM gateway, the works.4
The commercial layer sits on top in LangSmith, its evaluation and tracing product. Pricing runs like this:5
- Developer tier: $0 per seat per month, with usage limits.
- Plus tier: $39 per seat per month, then pay-as-you-go compute.
- LangChain Compute Units (LCUs): $1.50 each, billed for trace and eval workloads.
- LangChain Storage Units (LSUs): $1.00 each, billed for retained data.
- Enterprise: custom pricing, negotiated per contract.
That's a seat-and-usage model layered on top of an "open" framework. It's the exact structure that shows up when a free tool becomes the entry point to a metered product. If your abstraction layer also owns your billing meter, you don't just depend on the code. You depend on the invoice.
What does a framework actually buy you, and what does it cost?
Frameworks aren't worthless. They bundle integrations, give you a faster starting point, and standardize patterns across a team. That's real value for some teams and some problems.
The cost is what Anthropic's engineers flagged directly: frameworks "create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug."1 The 12-Factor Agents author names the failure mode precisely: teams get to a 70-80% quality ceiling with a framework, then hit a wall.2 Past that point, improving the agent means reverse-engineering the framework's internals, often more work than not having used it in the first place.
| Debuggability | Initial setup speed | Customization | Vendor lock-in risk | Hits 70-80% quality ceiling | |
|---|---|---|---|---|---|
| Framework-based (e.g. LangChain)Fast prototypes, standardized team patterns | Low | High | Low | High | Yes |
| RecommendedFramework-free (direct API + vector store)Production agents needing full control | High | Low | High | Low | No |
Anthropic's recommendation is plain: start with direct LLM API calls, and only adopt a framework once you deeply understand the code it would abstract away.1 That's the opposite of how most teams onboard a framework today. They usually adopt it before they understand what it's hiding.
What's the four-step pattern behind framework-free RAG?
Strip away the orchestration layer and RAG is four steps: chunk your documents, embed them, search against a vector store, and ask the model using the retrieved context. A working teardown shows this can be built with five plain Python libraries — chromadb, openai, pypdf2, python-docx, and sentence-transformers — no LangChain, no LlamaIndex.3
The author's framing is direct: "All code is production-ready and debuggable — no black boxes."3 The same writeup lists the recurring pain points of framework-based RAG: debugging across abstraction layers, documentation overhead spent learning framework-specific concepts instead of RAG fundamentals, limited customization, and breaking changes on version upgrades.3 None of those problems disappear because the framework is popular. They compound.
Workflows vs. agents: a decision that has nothing to do with frameworks
Here's where a lot of the debate gets muddled. Anthropic draws a hard line between two different things: workflows, where LLMs and tools run through predefined code paths, and agents, where the LLM dynamically directs its own process.1 That distinction is about autonomy, not tooling.
Whether you use a framework and how much autonomy you give the model are separate decisions. You can build a fully autonomous agent in plain Python, and you can build a rigid, predictable workflow inside LangGraph. Conflating "agentic" with "framework-based" is the mistake that pushes teams toward heavier tooling than the problem needs.
The notebook as the new IDE for evaluation
Ask practitioners where they actually build these pipelines and a split shows up. Coding agents and IDEs like Cursor and Claude Code get used to write the production pipeline code. Jupyter and Colab notebooks remain the preferred surface for the messier, more important work: comparing document parsers, running retrieval and model evals, and picking apart win/loss patterns.6
The reasoning from that community is simple: notebooks make each step directly inspectable.6 You can see the chunks, see the embeddings, see the retrieved context, and see the final prompt, all in cells you can rerun individually. A framework's abstraction hides exactly the layer a notebook exposes.
Engineering principles worth stealing anyway
Even teams that end up adopting a framework can borrow the discipline behind 12-Factor Agents. The methodology treats agent-building as software engineering, not framework configuration. The core ideas that carry over regardless of tooling choice:
- Own your prompts. Don't let a framework template hide what you're actually sending the model.
- Own your context window. Know exactly what tokens go in, not what a library decides to inject.
- Own your control flow. Branching logic should live in code you can read, not a framework's internal graph.
- Treat the 70-80% ceiling as a warning sign. If a framework gets you most of the way and stalls, that's the point to reconsider, not push harder.2
- Keep tools and functions plain. Structured output and tool calls work fine as local Python functions and schemas, no orchestration required.
None of this requires abandoning frameworks forever. It requires understanding the pipeline well enough that adopting one is a choice, not a dependency.
When should you actually reach for a framework?
The practical rule: prototype framework-free, adopt a framework only once you hit real production concerns — persistence across sessions, observability across a team, human-in-the-loop approval steps, multi-agent handoffs at scale. Those are legitimate reasons to pay for infrastructure. Locking in early because a framework made the first demo faster is not.
This is the same math that shows up across the software-ownership beat: renting infrastructure before you understand what it does costs you leverage later, whether that's cloud LLM APIs versus owning your models or the orchestration layer behind your internal agents. If you're weighing whether to build that agent stack in-house at all, tools like Remy exist specifically to help teams stand up owned, framework-free agent infrastructure without renting the abstraction layer along with it.
Own the four steps. Own the control flow. Decide on the framework last, not first.
It means building a retrieval-augmented generation pipeline with plain code — direct calls to an LLM API and a vector database — instead of routing the logic through an orchestration library like LangChain or LlamaIndex. The core steps are chunking documents, embedding them, searching a vector store, and asking the model with retrieved context.3
The core library is open source, but the commercial layer built on top, LangSmith, charges per seat and per usage unit. Pricing runs from $0 on the Developer tier to $39 per seat per month on Plus, plus pay-as-you-go compute and storage units billed separately.5
Anthropic defines workflows as systems where LLMs and tools are orchestrated through predefined code paths, and agents as systems where the LLM dynamically directs its own process and tool use.1 This distinction is about autonomy, not about whether you're using a framework.
Once you hit real production requirements a framework is built for — persistence across sessions, team-wide observability, human-in-the-loop approvals, or complex multi-agent orchestration. Prototyping framework-free first means you understand what you're abstracting away before you pay to abstract it.
- 1Building effective agentsAnthropic
- 212-Factor Agents - Principles for building reliable LLM applicationsHumanLayer (GitHub)
- 3Building RAG Applications Without LangChain or LlamaIndexFutureSmart AI (Hashnode)
- 4LangChain raises $125M to build the platform for agent engineeringLangChain
- 5LangSmith Plans and PricingLangChain
- 6Where are you building your RAG systems? AI IDEs? Colab / Jupyter Notebooks? Both?Reddit r/Rag
