Blog
bitter lesson softwareAaron Levie AIAI context layer

How Should Software Companies Build for the Bitter Lesson?

Box CEO Aaron Levie explains how to decide what to build when AI models keep improving: treat scaffolding as disposable and invest in governance and APIs.

Portrait of Luis Chavez-Mattos
Edited by Luis Chavez-Mattos, Director, Product··RSS

For Executives, Product Management and Engineering

In side profile, a man in a backwards cap lit by cool screen glow leans in as a small scaffolding frame is taken apart in the dark.

How should a software company decide what to build when AI models keep improving?

Build only what stays useful even if AI models become far smarter, and treat everything else as temporary. Box CEO Aaron Levie describes his company's rule this way: ride the curve of model intelligence as far as possible, and build only things that would still matter if models were a thousand times better. That means data governance, security, fast infrastructure, and clean interfaces for agents. Anything that merely compensates for what models can't do today is scaffolding, and you should expect to throw it away.

TL;DR

  • The bitter lesson is the idea that general methods powered by more computing power eventually beat clever hand-built workarounds. Software companies now have to plan around it.
  • Levie says Box builds almost nothing that better models would make obsolete, other than temporary scaffolding that exists because models can't yet do the job.
  • The durable layer is governance, security, and access control: deciding what data an agent can see and keeping that data curated rather than dumped in one big pile.
  • Companies should go API-first, because in many cases no person will ever look at the product's interface. Agents will call it directly.
  • Throwing work away is normal. Levie says it happens "all the time," including a capability that a new model made useless before Box could ship it.
  • For workers and managers, the useful move is to put effort into organized data and clear permissions rather than elaborate workarounds for today's model weaknesses.

What is the bitter lesson, and why does it matter to software?

The "bitter lesson" comes from an essay by AI researcher Rich Sutton. His observation, drawn from decades of AI research, was that approaches built on general methods and more computing power keep beating approaches built on hand-crafted human knowledge. The lesson is bitter because researchers kept investing in clever tricks that the next wave of raw capability made unnecessary.

Software companies face a version of this now. If you spend a year building a feature that helps a model do something it can't do well today, the next model may do it without help. Levie says he is "probably in the upper limit" of being "bitter lesson pilled," meaning he leans hard on this idea when deciding what Box builds.

His example is concrete. An engineer at Box built a prototype that used the latest models of the time to design a full application inside Box. Levie says the agent did in about 30 seconds what customers spend two days on: thinking, whiteboarding, and designing the architecture. As models improved, the team concluded that models might soon be better at designing business workflows than the people who normally do it.

What is scaffolding, and why should you expect to throw it away?

Scaffolding is the extra code, prompts, rules, and structure that surround a model to make up for something it can't yet do reliably. It is useful and often necessary. It is also temporary by design.

Levie is direct about the cost. Asked how often Box throws things away, he answered "All the time." He described unreleased capabilities that a new model made "totally useless" before Box could get them out the door.

This changes how a company should plan. A few habits follow:

  • Keep scaffolding cheap to build and cheap to delete.
  • Don't tie a product's identity or pricing to a workaround.
  • Re-test old assumptions whenever a major model arrives. The thing you built because the model couldn't do it may now be unnecessary.

What stays valuable when models get much smarter?

Levie's thought experiment is to assume superintelligence, a model smarter than every person at every topic, and ask what an organization would still need to deploy it safely. His answer is a short list:

What stays valuableWhy it survives better models
Data governanceSomeone must decide what each agent may access, no matter how smart it is
Strong securityMore capable agents raise the stakes of a mistake
High-speed, high-throughput infrastructureAgents work at volumes people never did
Easy-to-use schemas and toolsAn agent needs a clear way to create and update objects in your systems

None of these depend on a model being weak. A smarter model doesn't remove the need to say which contracts it may read or which customer records are off limits. If anything, it adds to that need.

What is the "context layer," and why does it need governance?

Levie describes a future with many more agents than people. He attributes the "10 or 100 times more agents than people" idea to Nvidia CEO Jensen Huang. With that many agents working inside a company, something has to hold the ground truth and set the guardrails. Which CRM data can an agent use? Which ERP data? Which files?

People often assume you can throw everything into a database and let the agent sort it out. Levie argues against that. In his words, you can't drop it into "some gigantic blob of information." The data has to be curated, with:

  • access controls, so agents see only what they should
  • metadata, so the agent knows where to route a query and how to find the right document
  • sandboxing and separation between what different systems can reach

His broader picture has two layers. On top are the agents. Beneath them are modernized systems of record, the places where a company's official data lives, which become what he calls systems of intelligence. They control access, enforce governance, store the information, and likely keep a user interface, because people still need to put data in and interact with it.

Why does going API-first matter for agents?

An API is a way for one piece of software to talk to another without a person clicking through screens. Levie compares the shift to an old internal memo: 15 years ago, a company might have declared it was going "completely API first," like Amazon's approach with AWS, where anything built for internal use gets exposed externally.

His point is that most software companies now need to make that same move. In his words, there will be "a very long list of scenarios where nobody will ever see your interface." So the question becomes whether your product creates real value when the only user is an agent.

He also says openness is central to Box's approach. Information should flow to wherever work happens, whether that is Salesforce, SAP, Oracle, ChatGPT, or Claude. Box describes itself as "religiously" partner-oriented.

How does Box decide when to build something itself?

Box's rule has two parts. First, anything that is durable infrastructure (governance, security, access, interoperability) is core. Second, where Box has a particular understanding of its domain, which is unstructured data and enterprise content, it will build higher-level tools.

The example is Box Architect, announced at Box's BoxWorks event. Levie describes it as an autonomous system for designing, building, running, and eventually optimizing agents, workflows, and apps built on Box. A company could ask for a contract management, invoice processing, or document review solution, and a frontier model would design it using Box's building blocks.

Notice what that design does. Box does not compete with the model on intelligence. It uses whatever the leading model is at the time and supplies the parts the model can't supply on its own: the governed content and the building blocks. When models improve, Architect gets better without Box rewriting it. Levie says Box is also fine if customers use Box as a "headless" service (no interface, just the underlying capabilities) with other technology.

Levie also mentions similar patterns in customer support, where tools help a support operations person design the agents that talk to customers. Architect applies the same idea to any business process, such as customer onboarding, loan processing, or client intake.

What does this mean for you?

If you work at a company that buys software: ask vendors what happens to their product when models get better. A vendor whose value is a workaround is at risk. A vendor whose value is governed data, permissions, and reliable APIs is more likely to hold up. Ask whether their tools work when an agent, not a person, is the user.

If you manage a team or build internal tools: put your effort into the durable parts.

  • Know where your key documents and records live, and who can access them.
  • Clean up permissions before you connect any agent to them.
  • Write down your processes as they should work, not only as they work today. Levie's team concluded that you shouldn't just automate today's work exactly as it looks. A model might propose a better process.

If you are budgeting: expect uncertainty. Levie notes that you can't predict in advance how much you will spend on tokens, the units AI providers use to bill usage. Plan for ranges and review them regularly.

If you are an individual contributor: don't over-invest in elaborate prompt tricks for a weakness a model has this month. Skills that keep their value are knowing your data, your process, and what good output looks like.

Can I build my own tools without renting more software?

Yes, and the bitter lesson makes it more practical. If models keep getting better at designing and building workflows, a team can describe a small internal tool (a request form, an approval tracker, a document intake dashboard) and have it built, rather than adding another subscription. Remy is built for this: you describe the app in plain English, and it writes a spec, builds a full-stack app with a database and sign-in, and deploys it. Because the spec is the source of truth and the code is derived from it, you can change the tool as your process changes. The code is standard TypeScript in a git repository you own.

Frequently Asked Questions

5 questions

What does "bitter lesson pilled" mean?

It means taking the bitter lesson seriously when making decisions. Levie uses it to describe a strategy of building only things that stay useful as models improve, and avoiding investments that a smarter model would make obsolete.

Does a smarter AI model remove the need for data governance?

Levie argues the opposite. More capable agents working across more systems make access controls, metadata, and security more important, not less. Someone still has to decide what an agent may see and do.

What is Box Architect?

Per Levie, it is a system for designing, building, running, and eventually optimizing agents, workflows, and apps on Box. A customer describes a process such as invoice processing, and a frontier model designs the solution using Box's building blocks.

Should every software company go API-first?

Levie says most software companies need to make that journey. His reasoning is that in many cases no human will see the interface, so the product has to deliver value to agents that call it directly.

How often should a company expect to discard AI features?

Often. Levie says Box throws things away "all the time," and describes a capability that a new model made useless before release. Build scaffolding to be disposable and re-test it whenever a major model arrives.

How we source, write, and correct our articles. Editorial standards

Related articles

Straight to your inbox.
New pieces as they publish. No noise.