The Taste-Tester Protocol: How to Evaluate AI-Generated Code When Benchmarks Lie

To evaluate AI-generated code, stop trusting the benchmark on the model's spec sheet. Assign a domain expert, not a generalist reviewer, to check every AI-authored pull request against a short correctness checklist before merge, and track how often that review changes the outcome. Call it the Taste-Tester Protocol. It catches what published scores can't.

The 99.9% Problem

When OpenAI announced GPT-6 Astra, the headline number was hard to ignore: a 99.9% score on ARC-AGI-3, a benchmark designed to be hard for machines and easy for humans.¹ ARC Prize's own Greg Kamradt called it effective human parity.¹

Then came the footnote. ARC Prize, the organization that runs the benchmark, reported that the same model scores about 66% using a standard API harness. The near-100% number only shows up with a custom, continuous-conversation harness that can cost tens of thousands of dollars to run once.² That's a 34-point gap between the number in the press release and the number an engineering team would actually see calling the model through a normal API.

This isn't a story about one vendor cutting corners. It's a story about what happens when a benchmark becomes a marketing surface instead of a measurement. That's why a growing number of engineering teams have stopped asking "what did this model score" and started asking a narrower question: does this specific output work, according to someone who actually knows the domain.

How Do You Evaluate AI-Generated Code? The Short Answer

The Taste-Tester Protocol comes down to four steps:

1. Assign a domain expert, not a generic reviewer. The person checking a billing-logic PR should understand billing, not just syntax. Benchmarks score general code tasks; your production code lives in a specific domain with specific failure modes.
2. Run a correctness checklist against real requirements, not test coverage. Passing unit tests isn't the same as matching what the business actually needs. Check the PR against the ticket, not just the diff.
3. Score confidence and effort, not just pass/fail. Log how much the reviewer had to fix, not whether they eventually approved it. That gap is your real signal.
4. Feed the corrections back into how you prompt and scaffold the model. Recurring mistakes point at a systemic gap in how the tool is being used, not a one-off bug.

The rest of this piece explains why that protocol exists and what it's built to catch.

Why Benchmarks Lie

Three structural problems explain the gap between benchmark scores and shipped code quality.

1. Contamination. In February 2026, OpenAI stopped reporting SWE-bench Verified scores. The company audited a sample of the hardest problems in the dataset and found that at least 59.4% had flawed test cases that reject correct submissions. Worse, every frontier model tested, including OpenAI's own, could reproduce the verbatim "gold patch" fix for problems it should never have seen, evidence of training-data contamination across the industry.³
2. Saturation and task mismatch. A 2025 academic study built a benchmark called RealClassEval by pulling class-level coding tasks straight from live GitHub repositories, then compared model performance on those against the synthetic ClassEval benchmark. Models scored 84 to 89% correctness on the synthetic tasks. On the real-world equivalent, correctness dropped to 24 to 35%, a roughly 60-point collapse.⁴ The most common failure modes, AttributeError, TypeError, and AssertionError, accounted for 84% of all failures, the kind of thing that only shows up when code has to run against a live, messy system rather than a clean test harness.⁴
3. Goodhart's Law. When a measure becomes the target, it stops measuring what it was meant to measure. Benchmarks reward narrow, self-contained, algorithmically-scored tasks. Real software work involves accumulated context, shifting requirements, and discovery that no static test suite captures.⁵ Once a benchmark becomes the thing vendors optimize marketing around, the score and the underlying capability start to drift apart.

The Productivity Paradox

Even setting benchmark gaming aside, the productivity case for AI coding tools is shakier than most teams assume.

METR ran a randomized controlled trial with experienced open-source developers using early-2025 AI coding tools on real tasks in their own repositories. The developers who used AI took 19% longer to finish than those who didn't.⁶ The stranger finding: even after completing the study, those same developers believed AI had made them about 20% faster.⁶ The perception of speed and the measured outcome pointed in opposite directions.

Google's 2024 DORA report, drawing on tens of thousands of surveyed practitioners, found a similar split at the team level. A 25% increase in AI adoption correlated with a 7.2% decrease in software delivery stability and a 1.5% decrease in throughput, even as the same adoption correlated with a 3.4% increase in code quality and a 7.5% increase in documentation quality. Individual contributors felt more productive. Systems got less stable.

The Hidden Cost: Churn, Duplication, and Verification Debt

GitClear's analysis of 211 million lines of code between 2020 and 2024 found that copy-pasted code overtook refactored code for the first time in 2024, and duplicated code blocks rose roughly eightfold compared to prior years.⁷ Code churn, meaning code revised within two weeks of being written, rose from 3.1% to 5.7% over the same period.⁷ That's the signature of code optimized for fast output rather than long-term maintainability, exactly what you'd expect if AI-generated suggestions were getting merged with less scrutiny than hand-written code used to get.

Developers themselves seem aware of the gap even when they don't act on it. Sonar's 2026 State of Code developer survey of over 1,100 developers found that 96% don't fully trust AI-generated code to be functionally correct, yet only 48% say they always check it before committing.⁸ Ninety-five percent spend at least some effort reviewing or correcting AI output, and 59% call that effort moderate or substantial.⁸ The trust gap is real. For most teams, the review discipline to close it isn't.

The Taste-Tester Protocol in Practice

The fix isn't more benchmarks. It's a review structure built around who does the checking and what they're checking for.

The core idea: pair a subject-matter expert, someone who understands the domain the code serves, with the AI model on every generated PR, and give that person a lightweight scoring rubric rather than a vague "looks good to me" approval. A payments engineer reviews payment logic. A person who understands your inventory system reviews the inventory service. This sounds obvious, but most AI-assisted review pipelines route PRs to whoever's free, not whoever knows the domain.

As one code-review guide for AI output puts it, "AI does not write bad code, it writes plausible code. The lines compile, the linter passes, the variable names look reasonable, and on a fast PR review it slides through."⁹ That's precisely why a generalist reviewer, skimming for syntax and style, is the wrong gate. The bugs that matter are semantic, not syntactic.

Roman Zhukov of Red Hat and Damian Brady of GitHub have made a similar point about the limits of automation here: faster doesn't mean better, and AI amplifies a developer's existing practice rather than replacing it. It is, in their words, "never a replacement for a human in the loop or somebody who knows what they're doing, a proper security review and things like that."¹⁰ Teams building their own AI-native development workflows are learning this the same way: speed without ownership just moves the risk downstream.

What Should You Check in an AI-Generated PR?

When a domain expert sits down with an AI-generated PR, the checklist should go beyond "does it pass tests." A few checks that matter most:

• Hallucinated APIs and imports. Confirm every library call, method signature, and endpoint actually exists in the version you're running, not a plausible-sounding version the model half-remembers.
• Silent scope creep. Check whether the model touched files, functions, or config outside what the ticket asked for, even if the extra changes look harmless.
• Duplicated logic. Look for a new function that reimplements something that already exists elsewhere in the codebase, a pattern GitClear's data shows is becoming more common, not less.⁷
• Security assumptions. Verify input validation, auth checks, and secrets handling explicitly. Don't assume the model applied your team's security conventions just because the code compiles.
• Domain correctness against the actual requirement. Read the ticket, not just the diff, and confirm the logic matches what the business actually needs, not just what a plausible interpretation of the prompt produced.

This kind of structured review is a companion, not a substitute, to the security-specific governance work covered in Securing the Code Your AI Wrote, which goes deeper on the vulnerability side of this same problem.

Why This Matters Beyond the IDE: Shadow AI and Governance

The Taste-Tester Protocol isn't just an engineering-team discipline. It matters just as much, arguably more, outside the sanctioned dev pipeline.

More than 80% of employees report using AI tools at work that their organization hasn't approved, often with little visibility into how those tools are used or what data they touch.¹¹ That includes people outside engineering building their own internal scripts, dashboards, and automations with frontier models, code that ships into daily operations with no PR review, no CI pipeline, and no domain expert checking it against anything.

If a 99.9% benchmark score can hide a 66% real-world result inside a well-resourced engineering org with formal review, the gap is almost certainly wider in a spreadsheet macro or internal tool built by someone in finance or ops with no code review process at all. The same logic that says "pair a domain expert with the model" for a production PR applies, maybe with more urgency, to the software your own employees are quietly building for themselves.

Owning the Outcome

The pattern underneath all of this is familiar to anyone who has evaluated a piece of SaaS on the vendor's own case study. You don't take the vendor's number at face value. You run your own trial, with your own data, against your own workflow, and you let a person who understands the domain make the call.

AI-generated code deserves the same discipline. A benchmark score, even a genuinely earned one, describes what a model can do under specific, controlled conditions. It says nothing about what happens when that same model is pointed at your codebase, your edge cases, and your customers. Treating AI-generated code as an asset you own, rather than an output you passively accept, means building the verification step in from the start: a real reviewer, real domain knowledge, and a real checklist, on every PR, regardless of what the leaderboard says. Tools like Remy that build internal software around this kind of expert-in-the-loop review are betting that the gap between benchmark and reality isn't closing on its own, and that the winning move is designing for it now.