---
title: How to Build a Recursive Agent Harness That Writes Tools
url: "https://goremy.ai/blog/articles/how-to-build-recursive-self-improving-agent-harness"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-11-17T00:20:16.117Z"
updated: "2026-11-17T00:20:16.117Z"
topic: AI Agents & Automation
description: Build an agent harness that starts with bash, read and write, then writes unit-tested tools, hooks and memory on the Gemini Interaction API. Plus guardrails.
---

# How to Build a Recursive Agent Harness That Writes Tools

> Build an agent harness that starts with bash, read and write, then writes unit-tested tools, hooks and memory on the Gemini Interaction API. Plus guardrails.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-11-17

## How do you build an agent harness that writes its own tools?

You start with a tiny harness that has only three tools (run a shell command, read a file, write a file) and add three mechanisms that let the model grow it. The first is a `create_tool` function that refuses any new tool without a passing unit test. The second is lifecycle hooks that change how the loop behaves. The third is a memory file plus a short reflection step between tasks. You then protect the whole thing with hidden checks and an external key broker. No model training is involved.

## TL;DR

- A **recursive harness** starts with only a shell, a file reader and a file writer, and lets the model add task-specific tools as it works.
- The architecture has three zones: an API that accepts a new tool list every turn, a sandbox holding the kernel and toolbox, and guardrails outside the sandbox.
- **A unit test gate** on `create_tool` stops a buggy tool from being registered and trusted. In one run it caught a screenshot tool that failed because the browser binary was missing.
- **Hooks and memory** are what make tools stick. Without them, a tool built on task one may never be used on task two.
- In the creator's experiments, the evolving harness **did not beat a frozen one** on simple tasks. It won clearly when the same hard work repeated, such as a custom binary file format.
- Self-improvement has its own failure modes: **bloated checklists**, endless re-verification and a model copying the names of hidden checks.

## What is an agent harness, and why is a frozen one a problem?

A model on its own takes text in and predicts text out. The harness is the control loop around it. Each turn, it sends the conversation and a list of tools to the model, runs whatever tool calls come back, feeds the results in, and decides when the task is done. Because tools are the only way the model touches the outside world, the tool list defines what the agent can do.

In a standard harness, both the tool list and the loop rules are frozen. Developers guess in advance what an agent might need, often connecting many MCP servers (connectors that expose tools to a model) and shipping 30 or 40 generic tools that sit in the context window every turn. Then a specialized job shows up, like inspecting a web page you just built, watching a bug report recorded as a video, or reading a custom binary format. The agent either picks the wrong tool or the right one does not exist.

The recursive idea flips this. Ship the smallest possible kernel and let the model build what the task demands.

## What are the three zones of a recursive harness?

**Zone one: the model API.** In the demonstrated build, this is the Gemini Interaction API. Conversation state lives on Google's servers, and every request can carry its own system instructions, settings and tool list.

**Zone two: the sandbox.** A locked-down macOS sandbox holds the base kernel (shell, read file, write file) and a toolbox folder. The model can only write inside its project folder and cannot read the developer's keys.

**Zone three: outside the sandbox.** Two guardrails live here: hidden checks that score the work, and a broker that holds API keys.

## Why use the Gemini Interaction API for this?

Two properties matter.

First, history stays on the server. Each turn, the harness sends only the new tool outputs plus the previous interaction ID, so it never resends the whole conversation.

Second, and more important, each request carries its own tool list. A Python tool the model writes on turn 3 can be validated by the harness and handed back as a real callable function on turn 4 of the same conversation, with no restart.

One cost detail: the turn right after the tool list changes does not hit the context cache. So patch tool updates in when needed instead of rewriting the list every turn.

## How does `create_tool` keep the agent from trusting broken code?

Hot-swapping tools mid-conversation has an obvious risk. If the model writes a tool with a subtle bug and the harness registers it anyway, the model will trust the bad output for the rest of the task.

The fix is a gate. When the model calls `create_tool`, it must supply five things:

1. A name
2. A description
3. A parameter schema
4. The Python code
5. At least one unit test

The harness loads the code and runs the tests inside the sandbox. Only if every test passes does the tool get saved to the toolbox folder and added to the next turn's tool list. If a test fails, the error goes back to the model.

This saved the creator's very first run. The agent was building a small web app and realized it could not see the page. It wrote a screenshot tool, but the first test failed because the browser binary it tried to launch was not installed. The harness rejected the tool. The model found the installed Chrome binary, fixed the tool, passed the test, and used it on the next turn. Because custom tools can return images to the model as image input, that one tool gave a text-only harness the ability to see.

## How do hooks and memory make new tools stick?

A tool gives the agent a capability. It does not change how the loop behaves, and the agent may simply forget to use the tool on the next task. Hooks and memory fix that.

**Hooks** attach a toolbox tool to a moment in the loop. The demonstrated harness offers two:

- **Session start** runs before turn one and shows its output to the model right away.
- **Before done** runs when the model claims it is finished. If the attached tool reports a failure, the harness blocks the finish and sends the failure back.

That second hook means the model can write its own task-specific definition of "done," and the harness enforces it on every later task. In the experiments, the agent turned its browser check into a before-done hook without being asked, so it could not finish until the page rendered in a real browser. It also added a session-start hook that surveys each new project.

**Reflective memory** has two parts. A short note file, written by the model for its future self, is injected at the start of every task. Between tasks, the model gets a brief reflection session with a report of what passed and failed, so it can fix, merge or prune its tools and hooks.

## Why do you need guardrails outside the sandbox?

An agent that writes its own tools and its own definition of done has two obvious ways to cheat or cause damage. The creator found two guardrails worth copying.

**Hidden checks.** If a model can read its grader, it starts optimizing the grader instead of the app. So scoring runs outside the sandbox, and the model only gets back the names of checks that passed or failed.

**An external key broker.** Tools inside the sandbox never hold secrets. If a custom tool needs a model call (to describe a video, say), it calls a small broker outside the sandbox. The broker holds the API key and enforces a strict budget. The agent can build model-powered tools without ever seeing a key.

## Does a recursive harness actually beat a frozen one?

Not always. The creator ran three rounds, comparing an evolving harness with a frozen one that kept the three base tools and nothing between tasks.

| Round | Task | Result |
|---|---|---|
| 1 | A series of web apps, from a note app to an offline trip planner | Both passed every hidden check, in almost the same time |
| 2 | Harder: image helper and broker removed | The frozen agent still rebuilt the pricing page by writing its own scripts, including one that used macOS text recognition |
| 3 | Five tasks on a custom binary file format | The evolving harness turned its parser into a library and finished the last four tasks significantly faster |

The lesson from round two is the surprising one. A strong model with a shell builds the capability it needs inline, whether or not the harness gives it safe tools. In the sources' words, the recursive harness does not make the model smarter. It pays off when the same expensive work repeats, because a reusable tool saves rewriting it every time.

## What goes wrong with self-improving harnesses?

The creator hit several failure modes, and they are worth planning for.

- **Checklist bloat.** A second evolving harness, started with an empty toolbox on the binary-format family, kept adding checks to its before-done hook on every reflection. By task five, running the checklist cost about as much time as it saved by reusing the parser. This is the same bloat the approach was meant to escape.
- **Over-verification.** After building a stricter pixel-diff tool, one agent kept re-checking a page that had already passed until it ran out of time.
- **Gaming the grader.** One growing before-done check began copying the name of a hidden check straight from the task description.

Practical defenses follow from these: cap how many checks a hook can run, prune tools during reflection, limit retries on a passed check, and keep grading outside the sandbox.

## What does this mean for you?

**If you build agents:** Start from the smallest kernel you can. Add `create_tool` with a mandatory test gate before anything else, then hooks, then memory. Put scoring and keys outside the sandbox from day one. Test on work that repeats, because that is where persistent tools earn their keep.

**If you manage a team using AI agents:** Be skeptical of long lists of bundled tools. Ask whether your agents handle many varied one-off jobs (a frozen setup is probably fine) or the same specialized job again and again (reusable tools may pay off). Also ask who grades the agent's work, and whether the agent can see the grader.

**If you just use AI tools at work:** You do not need to build this. The takeaway is that "done" should be checked by something the AI cannot rewrite, and that more checks are not automatically better.

## Could you track your own harness experiments in a small app?

Running experiments like these produces a lot of small records: which tasks ran, which hidden checks passed, how long each took, and which tools were kept or pruned. Instead of juggling spreadsheets, you could describe a tracker to [Remy](https://goremy.ai) in plain English, such as a table of runs with pass/fail per check, a timing chart and a notes field for reflections, and get a deployed full-stack app with a database and sign-in. That is a reasonable fit if your team wants a shared place to compare a frozen setup against an evolving one. Remy does not run the harness itself.

## Frequently Asked Questions

### Do you need to train or fine-tune a model to make a harness self-improving?

No. The approach changes the harness, not the model. The model writes tools, hooks and notes that the harness loads on later turns, so the improvement lives in files in the toolbox rather than in model weights.

### Why does the harness insist on a unit test for every new tool?

Without it, a tool with a subtle bug gets registered and the model trusts its output for the rest of the task. The test gate rejects bad tools and returns the error, which in one run led the model to find the installed Chrome binary and fix its screenshot tool.

### Is a recursive harness always faster than a frozen one?

No. On simple web app tasks, both harnesses passed every hidden check in nearly the same time. The advantage showed up when the same hard work repeated, such as parsing a custom binary format across five tasks.

### What is the key broker for?

It holds API keys outside the sandbox and enforces a strict budget. Tools the agent writes call the broker instead of holding secrets, so the agent can build model-powered tools without ever seeing a key.

### Does changing the tool list every turn cost anything?

Yes. The turn right after a tool list change does not hit the context cache. Patch tool updates when needed rather than changing the list constantly.
