What Are AI Software Factories? The 5 Types Compared
Dark, fleet, PR, release and self-improving AI software factories compared, with real examples from Spotify and Airbnb and a guide to which fits your work.
For Engineering, Executives and Managers

What is an AI software factory?
An AI software factory is a setup that lets a coding agent work through a whole batch of tickets with little supervision, instead of you guiding it through one ticket at a time. You plan the work with the agent up front. The agent maps dependencies, builds pieces in parallel, runs its build loops, and usually deploys to a staging environment. You then review the results in batches. It is the next level of autonomy for coding agents, but it does not remove you. You still own the planning and the validation.
TL;DR
- A software factory gives a coding agent a batch of issues instead of one ticket, so your time goes into planning before and validation after.
- There are five types: dark, fleet, PR, release and self-improving. They differ mainly in how much a human reviews and how well automated tests can prove the work is right.
- The dark factory (no human review, any kind of work) is risky because your intent will not always be communicated perfectly, no matter how good the model is.
- The fleet factory works because it targets tasks with clear goals and clear tests, such as migrations and maintenance. Spotify's Honk has merged more than 2.5 million automated maintenance PRs.
- The PR factory is the most reliable but keeps you as the bottleneck. The release factory sits in the middle and is the balance the source's creator recommends.
- The right choice depends on one question: can a passing test prove this work is correct?
How is a software factory different from using a coding agent normally?
For the past couple of years, most people have worked with coding agents at the individual ticket level. You take a GitHub issue or a Jira ticket and guide the agent through planning, building and validating. You are in the loop the whole way.
A factory changes the unit of work from one ticket to a group of them. The agent handles the middle stretch on its own: working out which tickets depend on which, building in parallel, and iterating until its own checks pass. The result usually lands in staging, where you can look at everything together.
The reasoning is about bottlenecks. Writing code is no longer the slow part. Planning up front and validating afterward are. If the agent can run the middle autonomously and in batch, you can scale output without scaling your own hours.
How do the five types compare?
The clearest way to compare them is a two-axis chart. The vertical axis is how much human review there is, from "review every pull request" at the bottom to "review nothing" at the top. The horizontal axis is whether the agent's tests can prove the work is right. On the left, correctness depends on judgment. On the right, a passing test is enough.
Healthy factories sit along the diagonal. Above it is the danger zone: little review on work that tests cannot verify. Below it is wasted efficiency: you are reviewing more than you need to.
| Type | Human review | Best for | Main risk |
|---|---|---|---|
| Dark factory | None | Prototypes and proofs of concept | Bad intent ships straight to production |
| Fleet factory | None (tests act as the check) | Migrations, refactors, maintenance | Only works for a narrow set of tasks |
| PR factory | Every pull request, one issue at a time | Maximum reliability, getting started | You stay the bottleneck |
| Release factory | Heavy planning up front, batch review in staging | Most general product work | Needs well-scoped releases |
| Self-improving factory | Not detailed in the source | Still emerging | Little to compare yet |
What is a dark factory, and why is it risky?
The dark factory is the dream version. You send in a loosely structured request, and the system plans, builds, reviews and merges straight to production with no human in the loop.
The problem is not the model. It is communication. You will not describe what you want perfectly every time, and when your intent is off, a bad implementation goes live. A smarter coding agent does not fix that, because the gap is between what you said and what you meant.
The creator behind the source material ran a public dark factory experiment earlier this year, with separate review agents and automatic merging. The pipeline worked reasonably well, but small problems kept surfacing because he did not know exactly what he wanted up front. The agents did their job. The result still was not what he wanted.
Dark factories do have a place. For a proof of concept or prototype where you can validate the idea quickly, they are the fastest way to get something built.
What is a fleet factory?
A fleet factory is the dark factory that works. The pipeline is the same, but the target is different. Instead of managing a whole application, it focuses on work with a very clear goal and a clear way to measure success. If the risk of the dark factory is miscommunicated intent, pick tasks where that is nearly impossible: migrations, refactors, and routine maintenance.
The name comes from Spotify, which built its own version called Honk. Spotify reports that it has merged more than 2.5 million automated maintenance PRs, and it uses Honk for complex migrations too. Spotify is not running its platform through a dark factory. It found the slice of work where describing the goal and checking the result are both easy, and automated that.
Other examples cited in the source:
- Airbnb migrated 3,500 test files in six weeks, as documented in its own blog post.
- Bun was reportedly rewritten from Zig to Rust using a software factory.
All of these share one trait: a passing test is trustworthy evidence the work is done.
Why do studies disagree about AI coding agents?
You will see confident claims that agents can be trusted with more autonomy, and equally confident studies saying they cannot. The chart explains much of the gap.
METR, an AI evaluation group, found earlier this year that about one in two agent-written PRs that passed tests in open source projects would not be merged by the maintainers. The likely reasons are that the contributors did not understand the project's direction or did not communicate their intent well. That is the dark factory problem: work that needs judgment, with nobody checking it.
Spotify and Airbnb sit on the other side of the chart, where tests really can prove the work. Both camps are right about their own kind of work.
What is a PR factory?
A PR factory is the opposite of a fleet factory. You review every pull request, one issue at a time. You can still run many issues through the build loop in parallel, but when it is time to review, you look at each one individually.
Because a human checks each change, you can trust this setup with work that needs judgment calls. If your intent was slightly off, you catch it before anything merges. Stripe's Minions is cited as an example of this pattern.
The cost is that you remain the bottleneck. What you automate is the planning, building and iterating of separate agent sessions, which is close to how many people already work with coding agents, just with less manual orchestration. Still, it is the most reliable type, and it is the easiest way to start. You can test the core build loop before expanding to something larger.
What is a release factory?
The release factory sits in the middle of the chart, and the source's creator recommends it as the best balance of autonomy and control. It is what he is building himself.
It scales further than a PR factory because you review in batches, not one change at a time. It is not limited to fleet-style tasks, and it avoids the dark factory's unreliability because humans stay deeply involved. You do heavy planning up front, the agent builds the batch, and you validate in staging before anything reaches production.
A common worry is that this means a return to waterfall development. The creator says it does not, because the releases are small in scope. The transcript I worked from cuts off before the full workflow is laid out, so I will not guess at the details beyond that.
What is a self-improving factory?
The fifth type is the least settled. The name suggests a factory that learns from its own runs and adjusts its process over time. The material behind this article does not explain it in enough detail to compare it fairly, so treat it as something to watch, not a setup to copy.
Which type fits which kind of work?
Ask one question first: can a passing test prove this work is correct?
- Yes, clearly. Migrations, refactors, dependency updates, maintenance. Use a fleet factory. Reviewing every PR here is wasted effort.
- Partly. Normal product features, where tests catch bugs but not wrong decisions. Use a release factory, with batch review in staging.
- No, and mistakes are expensive. Sensitive logic, unfamiliar codebases, or your first attempt at any of this. Use a PR factory.
- You are exploring. A throwaway prototype you can validate by looking at it. A dark factory is fine.
What does this mean for you?
If you manage or work alongside engineers, the useful shift is in what gets asked of people. Time moves away from writing code and toward writing clear plans and checking results. Weak tickets and vague requirements cost more when an agent builds a whole batch from them.
Practical steps:
- Sort your backlog by testability. Identify the migrations and maintenance chores where "done" is easy to define. Those are your first candidates for automation.
- Start at the PR factory level if you are new to this, then move up to batch review once the build loop proves reliable.
- Invest in planning. The clearer your intent, the fewer surprises in review.
- Be wary of "ship straight to production" pitches for work that depends on judgment.
If you are not an engineer, the same logic applies to any delegated work: the more checkable the outcome, the less you need to review.
Could you build your own tracker for a software factory?
Batch work creates its own admin: which tickets are in this release, which staging link goes with it, who has signed off. Many teams end up juggling that in a spreadsheet. With Remy, you can describe a small internal app in plain English, such as a release tracker with batches, staging links, reviewer sign-off status and a notes field, and Remy builds and deploys it with a login and a database. It runs as a normal web app your team can open from a link, and you own the code. It is a practical fit if you want a review queue shaped to your process instead of another subscription.
Frequently Asked Questions
Is an AI software factory the same as a fully autonomous AI developer?
No. That is the dark factory, which is only one of five types and the riskiest. Most factories keep humans heavily involved in planning and validation, and generally sign off before anything reaches production.
What is the Spotify Honk example?
Honk is Spotify's own fleet factory. Spotify reports it has merged more than 2.5 million automated maintenance PRs, and uses it to orchestrate complex migrations. It works because those tasks have clear goals and clear tests.
Which factory type should a beginner start with?
The PR factory. You review every pull request, so mistakes are caught early, and you can test the core build loop before moving to batch review in a release factory.
Why did METR find that many test-passing agent PRs would not be merged?
METR found that about one in two test-passing agent PRs in open source projects would not be merged by maintainers. A likely explanation is that the contributors misunderstood the project's direction or did not communicate their intent well, which is a judgment problem that tests alone cannot catch.
How we source, write, and correct our articles. Editorial standards
Related articles

How Do Teams Verify AI-Written Code at Speed? Sonar Survey

What Was the Hugging Face Incident? Christiano on Reward Hacking

Can a Coding Agent Run a Robot? Stanford's VIA Experiment
