---
title: "How Do PandaCode Council Reviewers Check AI Agent Work?"
url: "https://goremy.ai/blog/articles/pandacode-council-multi-agent-review"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-11-18T18:40:12.637Z"
updated: "2026-11-18T18:40:12.637Z"
topic: AI Agents & Automation
description: "How PandaCode's Council has fresh reviewer agents score AI work, counts the lowest score, and what it costs in rounds, tokens and time."
---

# How Do PandaCode Council Reviewers Check AI Agent Work?

> How PandaCode's Council has fresh reviewer agents score AI work, counts the lowest score, and what it costs in rounds, tokens and time.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-11-18

## How does a council of reviewer agents check an AI agent's work?

A council is a group of fresh AI reviewers that score finished work before the lead agent is allowed to say it is done. Each reviewer gets one "lens," meaning the single thing it cares about, such as security, accuracy or how the screen looks. The lowest score counts, not the average. If the work falls below your target, the agent has to fix the problems and call a new round of new reviewers. PandaCode, an agent tool, added this as a Council toggle.

## TL;DR

- **An agent grading its own work is unreliable.** In one test, seven agents each scored their own video scene 9 out of 10, while separate agents with no context scored the same scenes between 5.5 and 8.
- **The lowest score wins, not the average**, so one weak area (like a security hole) can't hide behind strong ones.
- **Reviewers never see your target score or earlier scores**, and every round brings in new reviewers, so they judge the work as a new user would.
- **A reviewer can only score against the goal you write.** "Top notch" is not checkable. Specific, named states and behaviors are.
- **Councils are expensive and rarely hit perfect.** One user's three days of councils ran about 250 rounds with 837 reviewers, roughly 180 hours of agent time.
- **You can copy the idea without the tool** by opening a fresh chat, giving it one lens, and hiding your expectations.

## Why can't an AI agent judge its own work?

The person who makes a thing is the worst person to judge it. They know what they meant to build, and they see that intention every time they look at the result. A reviewer has no access to the intention. A reviewer only sees what is there.

This is why writers have editors and papers have peer review. Agents have the same problem, and arguably a worse one. Ask an agent to build a settings page and it will tell you everything is done and everything works. It isn't exactly lying. It is grading its own homework, and it has every reason to like the answer.

The creator of PandaCode put a number on it. He had seven agents build seven scenes of a video and asked each to score its own scene. All seven gave themselves a 9. Separate agents with no context scored the same scenes between 5.5 and 8. That gap is the reason the feature exists.

Before Council existed, he closed the gap with a prompt, typing something like "use a council of agents to score your work out of 10 and keep going until you reach a nine." Going back through his logs, he found he had typed some version of it in 61 tasks. Repeated that often, a paragraph becomes a button.

## How do you set up Council in PandaCode?

In the prompt box, next to the other toggles, there is a small scale icon. Clicking it opens a panel with one switch, three settings, and an option to start new threads with Council on.

| Setting | Default | What it does |
|---|---|---|
| Target score | 9 | The score the work must reach. Adjustable from 7 up to 10. |
| Max rounds | 10 | How many review rounds are allowed before it stops and asks you. |
| Reviewers | Auto | How many reviewers per round. On auto, the lead agent decides how many the work needs. |

Once it is on, PandaCode starts a team for the task if you don't have one. The lead agent plans and hands work to workers as usual. The change is at the end. The lead can no longer just declare it finished. It has to call a council, pick reviewers, and give each a lens.

## What rules make the scores trustworthy?

Four rules do the work, and the app enforces them rather than relying on a prompt.

1. **The lowest score counts.** If two reviewers give 9.5 and the security reviewer gives 6, your score is 6. An average lets a strong part hide a weak part.
2. **Reviewers never see the target or old scores.** If a reviewer knows you want a 9, you will get a 9. If it knows last round was a 7, it hands you a polite 8.
3. **Every round uses fresh reviewers.** A reviewer who looked before is now checking whether you did what it said. A fresh one sees the work the way a new user would.
4. **Lenses are locked after round one.** The lead can't swap a strict reviewer for a friendlier one halfway through.

If the lead tries to finish below the target, PandaCode refuses and tells it to fix the blockers and call a new round.

## What does a real Council run look like?

The creator asked PandaCode to build a skill manager for itself, with the instruction that the UX be "top notch." After the workers finished, the first council returned a lowest score of 5. The findings were not nitpicks:

- Switching projects did nothing for tasks running in a work tree.
- Some skills showed as installed in about 14 tools, whether or not those tools were on his machine.
- The scope switcher was misaligned in every screenshot.

The score moved from 5 to 6 to 7, then sat between 6 and 7 for about 18 rounds. By round 22 it hit 8.5, and a reviewer found that a failed uninstall could leave a skill half deleted with no undo. That was fixed, and round 23 returned a 9. The run ended at the 25-round limit with a 9 against a 9.5 target. PandaCode then handed the decision back to him. When the lead is stuck, you can keep fixing, lower the target, or press accept and finish.

## How do you write prompts and lenses that produce useful scores?

Reviewers score against the goal you give them, not against your taste. "Top notch" is not something anyone can check. Write what you would check yourself.

**Be specific about the outcome.** Instead of "top notch," the creator would now write: the scope switcher lines up with the search field in every state, install, uninstall and undo each take one click, and nothing moves when a list loads.

**Name the states.** Say empty, loading, error, and a list of 200 items. A reviewer only screenshots the states you name.

**Say how reviewers should look.** If a visual review matters, write that one reviewer must run the app and judge screenshots, not the code.

**Split the eye from the hands.** Use one lens for how it looks and another for how it behaves when you click fast.

A template you can adapt:

```
Goal: [one sentence describing what done looks like]

Must be true:
- [specific, checkable condition]
- [specific, checkable condition]

States to check: empty, loading, error, [large case, e.g. 200 items]

Reviewer lenses:
1. Visual: run the app, judge screenshots in every state listed.
2. Behavior: click fast, undo, switch screens, try to break it.
3. Safety: what could lose data or expose something private?
```

## Does it work on things that aren't a user interface?

Yes, and this was the surprise. A lens can be anything. Examples from the creator's own use:

- **Security:** on a phone app that controls PandaCode on a computer, a security lens found that a phone with view-only access could read the output of every terminal on the machine.
- **Real data:** when he redesigned how tool calls look in chat, reviewers ran the design against his actual history and found about 4,200 old records now read as junk.
- **Releases:** one lens checked release notes accuracy and another checked build integrity. The build lens caught that the tagged commit never passed CI.
- **Scripts and thumbnails:** a factual accuracy lens caught a voiceover saying $203 where the evidence said $23.04. A click-appeal lens noticed a thumbnail showed a different product than the video covered.
- **Honesty:** two lenses the lead picked itself, report honesty and diagnosis quality, caught an agent claiming a recording failed because a Mac was overloaded. The reviewer read the crash reports and gave that diagnosis a 3 out of 10.

## What does Council cost, and where does it fall short?

The cost is real. In three days, the creator's councils ran about 250 rounds with 837 reviewers, roughly 180 hours of agent time. The panel warns that it uses more of your plan. It is not for a typo fix.

Other limits he reported:

- **Perfect scores are rare.** None of his councils with a 9.5 target reached it. They usually started between 5 and 7 and ended between 8 and 9.
- **Late rounds move slowly.** The skill manager went from 5 to 7 in three rounds, then needed 20 more to reach 9. Most of those moved the score very little, though round 22 found the data-loss bug.
- **Reviewers act on your machine.** One filled his disk by compiling the app to check it.
- **Same model, same opinion.** Three reviewers on one model is one opinion three times. PandaCode's Panda Router, when on, tries to give each reviewer a different model.

His settings: target 9, rounds at 5 or 10 for most work, and 25 only when losing someone's data is possible.

## What does this mean for you?

You don't need PandaCode to use the idea. Anyone who asks an AI to draft a report, a spreadsheet formula, a policy summary or a slide deck can apply it.

1. **Don't ask the same chat if it did a good job.** Open a new conversation, ideally in a different model.
2. **Give the reviewer one lens.** "Check only the numbers against these sources" beats "review this."
3. **Hide your expectations.** Don't say what score you want or what the last review said.
4. **Run several lenses and keep the worst result.** Accuracy, tone and risk can each get their own fresh chat.
5. **Write checkable goals.** "Every figure matches the source table" beats "make it good."

A score of 9 is not the truth. It means a few strangers, each caring about a different thing, looked hard and found little to complain about. Taste stays with you: you choose the goal, set the bar, and decide when to stop. The useful shift is the question you ask. Not "did the agent say it's done?" but "who else looked at it?"

## Can you build your own review tracker instead of using another subscription?

If you review AI output regularly, a spreadsheet of lenses, scores and decisions gets messy fast. With [Remy](https://goremy.ai), you can describe a small internal tool in plain English: a form where someone submits an AI-produced draft, assigns lenses like accuracy or risk, records each reviewer's score, flags the lowest one automatically, and logs whether the team fixed, lowered the bar or accepted. Remy builds it as a full-stack app with sign-in and a database, and deploys it to a live URL, so your team shares one record instead of scattered files.

## Frequently Asked Questions

### Why use the lowest score instead of the average?
An average lets strong areas cover for a weak one. A great design score means little if a security reviewer finds that private data can leak. Taking the lowest score makes the worst problem the number that matters.

### Why do reviewers need to be fresh each round?
A reviewer who saw an earlier version starts checking whether its own feedback was addressed. A new reviewer sees the work as a new user would, which catches different problems. In the skill manager run, a late round found a half-deleted uninstall bug that earlier rounds missed.

### How many rounds should I allow?
The creator keeps the target at 9 and rounds at 5 or 10 for most work. He only pays for 25 rounds when losing someone's data is on the table, because later rounds often move the score very little.

### Does Council guarantee good work?
No. It means several independent reviewers, each focused on one concern, couldn't find much wrong. The creator's runs usually ended between 8 and 9, and none reached a 9.5 target. You still choose the goal and decide when to stop.

### Can I do this with ordinary chatbots?
Yes, by hand. Start a new chat for each reviewer, give each one lens, don't share your target or earlier scores, and use different models if you can. Then act on the lowest score.
