---
title: "How to Detect AI Slop: 5 Open-Source Tools to Build Your Own Detector"
url: "https://goremy.ai/blog/articles/how-to-detect-ai-slop-5-open-source-tools-to-build-it"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-09-12T10:00:07.155Z"
updated: "2026-09-12T10:00:07.155Z"
topic: AI Safety & Security
description: "AI content is now roughly half the web. Here's how to build a self-hosted AI slop detector with Binoculars, Fast-DetectGPT, ZipPy, SynthID, and bot filters."
---

# How to Detect AI Slop: 5 Open-Source Tools to Build Your Own Detector

> AI content is now roughly half the web. Here's how to build a self-hosted AI slop detector with Binoculars, Fast-DetectGPT, ZipPy, SynthID, and bot filters.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-09-12

## How do you detect AI slop with open-source tools?

You detect AI slop by combining a statistical text classifier, a watermark check, and a bot-traffic filter into one pipeline you run yourself, not by paying a SaaS vendor per word scanned. Five open-source tools do the job: Binoculars, Fast-DetectGPT, ZipPy, SynthID, and a stack of anti-bot tools like BotD and Anubis.

## TL;DR

- AI-generated articles briefly **overtook human-written ones** on the open web in November 2024.
- Commercial AI detectors market roughly 99% accuracy, but independent testing shows they **often fall short of those claims**.
- Open-source tools like **Binoculars** offer zero-shot detection with no training data and no per-word invoice.
- Effective detection means **layering traffic filters, compression checks, and statistical models** rather than trusting a single tool.
- SynthID only works on text your own models produced, so it is **a provenance tool** and not a general slop detector.
- Detection is an arms race, so send low-confidence cases to **human review** instead of auto-rejecting them.

## Why does detection suddenly matter?

AI slop is content made with generative AI that reads as low-effort, low-quality, or built purely to farm engagement rather than inform anyone. [1] It stopped being a meme problem sometime in the last two years. Graphite analyzed 65,000 English-language URLs from Common Crawl going back to 2020 and found AI-generated articles briefly overtook human-written ones in November 2024. By May 2025, AI content made up roughly 52% of newly published web articles, up from about 2.2% in January 2020. [2] [3]

**AI-Generated Share of New Web Articles**

| Date | AI-generated content share |
| --- | --- |
| Jan 2020 | 2.2% |
| Nov 2024 | 50% |
| May 2025 | 51.7% |

_Based on Graphite's analysis of 65,000 English-language URLs from Common Crawl; AI-generated articles briefly overtook human-written ones in November 2024._

The submission side of the internet feels it too. Clarkesworld, the science fiction magazine, saw story submissions more than double at the peak of AI-generated spam, much of it driven by "get rich quick" schemes circulating on YouTube and TikTok. [4] The editor has kept adapting his screening process ever since, and the magazine still received 14,805 submissions in 2025, averaging over 1,200 a month. [5] If a niche literary magazine needs a standing filter, so does any platform that accepts user-generated content, reviews, forum posts, or agent-submitted data. Open-source maintainers face the same flood, as the [impact of AI-generated code on open-source quality](https://goremy.ai/blog/articles/what-is-the-impact-of-ai-generated-code-on-open-source-quality) shows.

**Clarkesworld Submission Volume, 2025**

- 14,805: Total submissions received
- 1,233: Average submissions per month

## Why doesn't the SaaS detection market fit this problem?

Commercial detectors like GPTZero, Originality.ai, and Copyleaks were built for one job: checking whether a single student's essay or a single freelancer's article was written by a human. That's a single-document, low-volume use case, and the pricing reflects it. Originality.ai's Enterprise plan runs $136.58 a month for high-volume checking, structured around per-word or per-credit consumption. [6] Scale that to a platform screening thousands of submissions, comments, or agent outputs a day, and the bill stops looking like a rounding error.

Accuracy is the other problem. These vendors market roughly 99% accuracy, but independent testing tells a messier story: GPTZero lands around 80-91% in practice, Originality.ai 85-92%, and Copyleaks anywhere from 74-94% depending on the sample, with false positive rates as high as 5.7-7% despite claims of under 1%. [6] GPTZero's own head-to-head benchmark reports 99.3% accuracy and a 0.24% false positive rate against Copyleaks, which just underscores that even vendor-run tests disagree with each other. [7] There's no single source of truth here, paid or free. That's an argument for owning the detection logic instead of outsourcing your judgment to a black box you can't inspect.

**Independent Accuracy of Commercial AI Detectors (Midpoint of Reported Range)**

| Detector | Independent accuracy |
| --- | --- |
| Copyleaks | 84% |
| GPTZero | 85.5% |
| Originality.ai | 88.5% |

_Midpoints of independently tested accuracy ranges (Copyleaks 74-94%, GPTZero 80-91%, Originality.ai 85-92%), versus ~99% marketed accuracy across all three._

## Why should you build your own detector instead of buying one?

A self-hosted slop detector costs engineering time instead of a subscription, and it can be tuned to your own content patterns rather than a generic academic-essay benchmark. That's the same logic driving the broader open-source infrastructure shift right now, where teams are pulling entire categories of rented tooling in-house because the tools have gotten good enough and the ops burden has dropped.[Build vs. Buy in 2026: The Open-Source Infrastructure Boom](https://goremy.ai/blog/articles/build-vs-buy-2026-open-source-data-infra-vs-enterprise-saas) You don't need to match a commercial vendor's marketing claims. You need something that catches the specific slop hitting your specific pipeline, and that you can retrain or retune without a vendor's roadmap in the way.

Here are the five tools that make that possible.

**Comparing the Five Open-Source Detection Tools**

| Option | Compute Cost | Setup Effort | Analyzes Text Content | Scope |
| --- | --- | --- | --- | --- |
| Binoculars (recommended) | high | medium | Yes | General text |
| Fast-DetectGPT | medium | medium | Yes | General text |
| ZipPy | low | low | Yes | General text |
| SynthID | low | medium | Yes | Own watermarked text |

_Ratings are relative across these options, not absolute scores._

### 1. Binoculars: zero-shot detection with no training data

Binoculars is a zero-shot, domain-agnostic method that needs no training data at all. [8] It works by running text through two closely related pretrained LLMs and contrasting their perplexity scores; human writing and machine writing produce different signatures under that comparison. The published numbers are striking: over 90% detection of ChatGPT and other LLM-generated text at a false positive rate of just 0.01%. [8] [9] That false positive rate matters more than the headline accuracy number. A detector that flags 1 in 10,000 real users instead of 1 in 20 is the difference between a useful filter and a support-ticket generator. Binoculars is the closest thing on this list to a flagship general-purpose text detector, and it's the one worth standing up first.

**Binoculars Detection Performance**

- 90%: LLM-generated text detected
- 0.01%: False positive rate

### 2. Fast-DetectGPT: efficient curvature-based detection

Fast-DetectGPT, published at ICLR 2024, is a faster, cheaper descendant of the original DetectGPT approach. [10] Instead of expensive perturbation sampling, it estimates conditional-probability curvature directly, which cuts compute cost substantially while staying zero-shot, no fine-tuning per model needed. If Binoculars is your accuracy baseline, Fast-DetectGPT is what you reach for when you need to run detection at volume without burning a GPU budget on every submission.

### 3. ZipPy: compression-ratio detection for lightweight filtering

ZipPy, built by security firm Thinkst, skips large language models entirely. It classifies text as AI- or human-generated using LZMA, zlib, or Brotli compression ratios as a stand-in for perplexity. [11] Compression algorithms compress predictable, low-entropy text more efficiently, and AI-generated text tends to be more compressible than human writing. It's not the most accurate tool here, but it's the lightest: pip-installable, packaged with browser extensions, and cheap enough to run on every inbound request instead of a sample. Good as a first-pass filter ahead of something heavier like Binoculars.

### 4. SynthID: watermark-based detection for text and images

Google DeepMind open-sourced SynthID Text in October 2024, letting any team watermark and later detect text generated by their own models. [12] Detection works by comparing the expected probability scores for words during generation against the words that actually appear. [13] The catch: it only works on text your own models produced with the watermark embedded, so it's a provenance tool, not a general slop detector. For images and video, the equivalent is C2PA, an open standard backed by Adobe, Google, and Microsoft that embeds cryptographic Content Credentials proving or disproving AI origin. Open-source projects like origin-lens implement C2PA verification for free. [14] Neither replaces statistical detection, but both add a second, independent signal when you control the generation side.

### 5. Open-source bot and traffic filters

AI slop isn't only generated text sitting still on a page. A growing share of it arrives as automated agent traffic: scraper floods, fake account signups, bulk submission bots. BotD does client-side bot fingerprinting in the browser, Anubis makes scrapers solve a proof-of-work challenge before they get through, and Fail2Ban bans IPs based on log patterns. [15] None of these read the content of a submission. They filter the traffic pattern around it, which catches slop that a text classifier alone would miss entirely. This matters more as more of that traffic is agent-driven rather than human, the same shift covered in [Build vs. Buy in the Agent Era: Orchestrating Your Own Office of Clones](https://goremy.ai/blog/articles/build-vs-buy-ai-agents-how-to-build-internal-workflows).

## How do you stack these into one pipeline?

No single tool here is a complete answer. The pattern that works is layering:

1. **Traffic filter first.** Run BotD or Anubis at the edge to strip out obvious bot and scraper traffic before it ever reaches your content pipeline.
2. **Fast pre-screen.** Run ZipPy or a similar compression-based check on everything that gets through, since it's cheap enough to apply universally.
3. **Accurate second pass.** Send anything ZipPy flags, or a sampled portion of everything, through Binoculars or Fast-DetectGPT for a stronger statistical read.
4. **Provenance check where relevant.** If you control the generation side, check SynthID watermarks on text and C2PA credentials on images to confirm or rule out origin.
5. **Human review on the edge cases.** Anything the pipeline flags with low confidence goes to a person, not an auto-reject.

This is roughly the same architecture teams already use for internal agent orchestration and AI gateways: layered checks with a human backstop rather than one model deciding everything.[Self-Hosted AI Gateway vs. SaaS: The Real Build vs. Buy Math](https://goremy.ai/blog/articles/self-hosted-ai-gateway-vs-saas-the-real-cost-math) The tools here are small enough that most of this can run on infrastructure you already have.

## Where does this approach still fall short?

Be honest about the limits before you ship this. Independent testing of even the best-funded commercial detectors shows real-world accuracy landing well below marketing claims, 80% to 94% rather than the advertised 99%, with false positive rates that can climb into the single digits under adversarial conditions. [6] Nothing here changes that math for the open-source tools either. Detection is an arms race: as detectors improve, so do the paraphrasing tools built specifically to evade them, and a determined bad actor with access to Binoculars' own paper can design around it. Watermark-based tools like SynthID only work on content generated with the watermark on, which means they're useless against slop from a competitor's model or an older unwatermarked version. And false positives carry real cost: wrongly flagging a human writer's work erodes trust fast.

If you're running a single-document, low-volume check, say, verifying one contributor's article before publication, a paid tool with a support team and an appeals process might still be the right call. But for platforms filtering thousands of submissions, comments, or agent-generated records a day, the economics flip. A stacked, self-hosted pipeline built from Binoculars, Fast-DetectGPT, ZipPy, SynthID, and open bot filters costs engineering hours, not a per-word invoice, and you can retune it the moment your own slop patterns change. That's the tradeoff worth making.

## Frequently Asked Questions

### What is AI slop, exactly?

AI slop is content made with generative AI that comes across as low-effort or low-quality, or produced mainly to farm engagement and ad revenue rather than to inform readers. [1] It covers generated articles, images, and increasingly automated agent or bot submissions.

### Can open-source tools really replace paid AI detectors like GPTZero or Originality.ai?

For high-volume, internal pipeline filtering, yes, tools like Binoculars and Fast-DetectGPT publish accuracy comparable to or better than what independent testing finds for commercial detectors, and they cost engineering time rather than a per-word subscription. [6] [8] For single-document, high-stakes checks with an appeals process, a paid tool with support may still make sense.

### How accurate is Binoculars at detecting AI-generated text?

Binoculars detects over 90% of text generated by ChatGPT and other LLMs, at a false positive rate of just 0.01%, without needing any training data. [8] [9]

### Do watermarks like SynthID work on any AI-generated text?

No. SynthID only detects watermarks embedded during generation by models that used it, so it works for content your own models produced but says nothing about text from unwatermarked models elsewhere. [12] [13]

### Is AI slop only a text problem?

No. A large share of AI slop now arrives as automated bot and agent traffic, fake submissions, scraper floods, bulk signups, which is why open-source anti-bot tools like BotD, Anubis, and Fail2Ban matter as much as text classifiers. [15]

## Sources

1. [What is AI slop? A technologist explains this new and largely unwelcome form of online content](https://theconversation.com/what-is-ai-slop-a-technologist-explains-this-new-and-largely-unwelcome-form-of-online-content-256554) (The Conversation)
2. [More Articles Are Now Created by AI Than Humans](https://graphite.io/five-percent/research/more-articles-are-now-created-by-ai-than-humans) (Graphite (Five Percent Research))
3. [Exclusive: AI writing hasn't overwhelmed the web yet](https://www.axios.com/2025/10/14/ai-generated-writing-humans) (Axios)
4. [Editor's Desk: The Future of Dealing with AI Submissions](https://clarkesworldmagazine.com/clarke_08_25/) (Clarkesworld Magazine (Neil Clarke))
5. [2025 Clarkesworld Submission Stats](https://neil-clarke.com/2025-clarkesworld-submission-stats/) (Neil Clarke (personal site))
6. [GPTZero vs Originality AI vs Copyleaks: Detection Compared](https://humanizethisai.com/blog/gptzero-vs-originality-vs-copyleaks) (HumanizeThisAI)
7. [GPTZero vs Copyleaks vs Originality: Most Accurate AI Detector?](https://gptzero.me/news/gptzero-vs-copyleaks-vs-originality/) (GPTZero)
8. [[ICML 2024] Binoculars: Zero-Shot Detection of LLM-Generated Text](https://github.com/ahans30/Binoculars) (GitHub (ahans30))
9. [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://arxiv.org/html/2401.12070v3) (arXiv)
10. [baoguangsheng/fast-detect-gpt: Code base for ICLR 2024](https://github.com/baoguangsheng/fast-detect-gpt) (GitHub)
11. [thinkst/zippy: Detect AI-generated text [relatively] quickly via compression ratios](https://github.com/thinkst/zippy) (GitHub (Thinkst))
12. [google-deepmind/synthid-text](https://github.com/google-deepmind/synthid-text) (GitHub (Google DeepMind))
13. [Google DeepMind is making its AI text watermark open source](https://www.technologyreview.com/2024/10/23/1106105/google-deepmind-is-making-its-ai-text-watermark-open-source/) (MIT Technology Review)
14. [GitHub - aloth/origin-lens: Combat fake news with C2PA content credentials](https://github.com/aloth/origin-lens) (GitHub)
15. [Top 5 Open-Source Anti-Bot Tools in 2025](https://dev.to/lialiago/top-5-open-source-anti-bot-tools-for-web-applications-21ag) (DEV Community)