We Analyzed 10,000 Lines of AI-Generated Code. Here's How to Measure the Sloppiness.

You measure AI code quality the way you measure any code's quality: churn rate, duplication rate, refactor ratio, complexity, issue density per pull request, and security-flaw rate, tracked over time and benchmarked against a human-written baseline. The problem is almost nobody does this. Teams measure how fast AI ships code. Almost none measure what that code costs them six weeks later.

The Productivity Story Everyone Tells (and the Quality Story No One Measures)

Every AI coding vendor has a velocity chart: more PRs, more lines shipped, faster review cycles. What's missing from almost every internal dashboard is the other half of the story, what happens to that code after it merges. Does it get reverted? Rewritten? Does it sit quietly until it turns into a vulnerability six months later?

That gap isn't a hunch. It shows up, consistently, across independent research efforts that never coordinated with each other and still landed on the same conclusion.

What Does the Research Say About AI Code Quality?

• A large academic study tracked 302,600 verified AI-authored commits across 6,299 GitHub repositories and found 484,366 distinct code issues introduced by AI assistants.¹ Code smells made up 89.3% of those issues, and every AI assistant studied had more than 15% of its commits introduce at least one problem.¹ Worse, 22.7% of those AI-introduced issues were still sitting in the codebase at the latest revision. Nobody caught them. They became permanent residents.¹
• GitClear's analysis of 211 million changed lines of code from 2020 to 2024, pulled from repos at Google, Microsoft, Meta, and enterprise C-corps, found the same erosion from a different angle.² Refactored code fell from 25% of changed lines in 2021 to under 10% in 2024, while copy-pasted code rose from 8.3% to 12.3%.² 2024 was the first year on record where copy-paste code outnumbered refactored code.² Code churn (code reverted or heavily altered within two weeks of being written) roughly doubled, from a pre-AI baseline near 3.3% to 5.7% in 2024, with a projected 7.1% in 2025.²³
• Google's 2024 DORA State of DevOps report adds nuance rather than contradiction: a 25% increase in AI adoption tracks with a 3.4% bump in code quality and a 7.5% bump in documentation quality, but also a 1.5% drop in delivery throughput and a 7.2% drop in delivery stability. Thirty-nine percent of practitioners still report little to no trust in AI-generated code.³
• CodeRabbit's review of 470 real-world pull requests, 320 AI-co-authored and 150 human-only, found AI PRs carried 1.7x more issues per PR (10.83 vs. 6.45), with logic and correctness errors 75% more common and security issues up to 2.74x higher.⁴
• Veracode's testing of over 100 LLMs across 80 coding tasks found 45% of AI-generated samples introduced an OWASP Top 10 vulnerability, with Java failing 72% of the time.⁵
• Stanford's Software Engineering Productivity Research Group, analyzing commit data from over 100,000 engineers at 600+ companies, found AI adoption increased PR volume by 14% but cut a code-quality score by 9%, tripled quality variance (3.6x), and increased rework by 2.6x.⁶

Different methodologies. Different datasets. Same pattern: more code, shipped faster, worse over time.

What Metrics Actually Measure AI Code 'Sloppiness'?

"Sloppiness" sounds like a vibe. It isn't. It's a set of measurable signals that show up in every dataset above:

• Code churn rate. The percentage of lines reverted or substantially rewritten within a short window, typically two weeks, after being committed. Rising churn means code isn't landing right the first time.²
• Duplication / clone rate. The share of changed lines that are copy-pasted rather than reused through abstraction. GitClear's 12.3% figure is the benchmark to beat.²
• Refactor (moved-lines) ratio. The inverse of clone rate: how much changed code represents genuine restructuring versus new sprawl. A shrinking ratio signals technical debt accumulating unaddressed.²
• Cyclomatic complexity. A standard measure of how many independent paths run through a function. AI models tend to generate verbose, branchy code that scores worse here even when it "works."
• Issue density per PR. Static-analysis findings (bugs, code smells, vulnerabilities) divided by lines changed. CodeRabbit's 10.83-vs-6.45 comparison is this metric in practice.⁴
• Security-flaw rate. The share of code samples or PRs that introduce a known vulnerability class, benchmarked against OWASP Top 10 categories.⁵

None of this is exotic. Static-analysis tooling has tracked these metrics for a decade. What's new is treating them as a gate on AI output specifically, not a generic quality dashboard nobody checks.

Our Experiment: Applying the Sloppiness Metrics to 10,000 Lines of AI-Generated Code

We wanted to see how this toolkit holds up as a working process, not just a research abstraction. So we ran roughly 10,000 lines of AI-generated commits across several feature branches through a static-analysis pipeline built on the same principles as SonarQube's quality gates: deterministic go/no-go thresholds on duplication, complexity, code smells, and security hotspots.⁷

The mechanics are simple enough for any team to replicate:

1. Baseline first. Run the same static-analysis scan against a recent human-written codebase from the same repo. Without a baseline, "45% security failure rate" or "12% duplication" is a number with no meaning.
2. Tag the origin. Label commits by whether they were AI-assisted, human-only, or mixed. Most git platforms support commit trailers or PR labels for this; if yours doesn't, build it.
3. Run the six metrics per commit window. Churn, duplication, refactor ratio, complexity, issue density, and security-flaw rate, computed weekly, not just at merge time, since churn only reveals itself over the following two weeks.
4. Composite into a single sloppiness score. Weight each metric, normalize against your baseline, and track the trend line. A single PR's score means little. The trend across a quarter is the signal.
5. Compare cohorts. AI-tagged commits against human-only commits, on the same repo, same time window. This is the step most teams skip, and it's the one that actually proves whether AI is helping or quietly taxing you.

What we saw lined up with the published research: higher raw volume, lower refactor ratio, and meaningfully higher issue counts per PR on AI-assisted branches versus human-only ones from the same repo. None of it required exotic tooling. It required tagging commits by origin and actually watching the trend line instead of the merge count. For the full mechanical walkthrough, see how to test AI-generated code before it ships.

Why Does AI-Generated Code Get Sloppy?

The pattern isn't mysterious once you look at how these tools work. AI coding assistants are pattern-matchers operating inside a context window. They don't hold your system's business logic in their head the way an engineer who's spent two years in the codebase does. Stanford's rework number makes the case: a 2.6x increase in bug-fixing work after AI adoption isn't a coincidence. It's the cost of code that looked right at generation time and didn't account for how the rest of the system actually behaved.⁶

CodeRabbit's finding that logic and correctness errors ran 75% more common in AI PRs points the same direction.⁴ The model produces plausible code, not necessarily correct code, and plausible is exactly what surface-level pattern matching is built to generate. Add a limited context window and duplication climbs for an obvious reason: the model can't see the helper function three files away, so it writes a new one.

The Hidden Cost: From Sloppiness to Technical Debt to Security Risk

This doesn't stay contained to a code review. Sloppy, unmeasured AI output is exactly how shadow AI tools spread inside companies, often outside any security review at all. IBM's 2025 Cost of a Data Breach Report found breaches involving shadow AI cost $670,000 more on average than standard incidents ($4.63 million versus $3.96 million) and accounted for 20% of all breaches studied.⁸ Ninety-seven percent of those incidents happened with no proper access controls in place, and only 17% of companies had technical controls capable of blocking unauthorized uploads to AI tools at all.⁸

That's the real cost of sloppiness: not a slightly messier codebase, but a governance blind spot that compounds until it shows up as a breach line item. If your team generates code faster than it reviews it, read our governance playbook for AI-generated code before that gap becomes your incident report.

How Do You Build a Sloppiness Gate for AI Code?

None of the research above is an argument against AI coding tools. It's an argument for measuring what they actually cost. A workable sloppiness gate looks like this:

• Set numeric thresholds, not vibes. Define a maximum acceptable duplication rate, complexity score, and issue density per PR, and block merges that exceed them, the way SonarQube-style quality gates already do.⁷
• Track churn on a two-week lag, not at merge time. A PR that looks clean on day one and gets reverted on day twelve is still a failure, and most CI pipelines never look back far enough to catch it.
• Run security scans on every AI-assisted PR, not a sample. With OWASP Top 10 failure rates as high as 45% in tested models, sampling isn't rigorous enough.⁵
• Require human review for anything touching business logic. Pattern-matching tools are weakest exactly where your system's judgment calls live.
• Report the composite score to engineering leadership monthly. Velocity metrics already get a dashboard. Give sloppiness one too, or it will never compete for attention.

The Bottom Line: Speed Without Measurement Is Just Debt on a Delay

The productivity story around AI coding tools is real. So is the quality story, and until now most teams have only told one of them. Every dataset in this piece, academic, vendor, and platform research alike, points the same direction: more code, produced faster, with measurably worse durability. That's not a reason to stop using these tools. It's a reason to stop treating "it shipped" as the finish line.

Software you can't measure is software you don't control, no matter whose name is on the commit. That's the same principle behind owning your software stack instead of renting it blind: the asset is only worth what you can verify about it. Sloppiness metrics are how you verify AI-generated code before it becomes the technical debt nobody remembers approving.