Underdog Woof TTS Review: How Accurate Is It, Really?

How accurate is Underdog Woof compared with its claims?

Underdog Woof is accurate on simple sentences but not as accurate as its model card suggests on hard ones. The card reports a 1.42% word error rate on a "hard cases" test. A reviewer who ran the model on a CPU found slips on phone numbers, names, fractions, big numbers and long sentences. The two results can both be true. The card's number comes from the authors' own test set, and the reviewer's sentences were packed with the trickiest content a text-to-speech model can face.

Underdog Woof is a small English text-to-speech (TTS) model. It has two voices, Juno and Mira. The whole download is about 40.6 MB, and it needs no GPU and no API key.

What is Underdog Woof, and what does it claim?

Text-to-speech software turns written text into spoken audio. Most good-sounding TTS today runs on large models that need a GPU or a paid cloud service. Underdog Woof goes the other way. According to its model card, it runs on any edge device with only NumPy (a common Python library for number crunching), so there is no GPU, no big server and no API key.

The card makes a few specific claims:

• A download of around 40.6 MB.
• Two voices, Juno and Mira.
• It reads phone numbers, dates, prices and quotes the way people say them.
• A 1.42% word error rate on the Gradio TTS hard cases test.
• A chart showing that no model they tested is both smaller and more accurate.

The reviewer ran it on an Ubuntu virtual machine with no GPU in use and very low CPU load. He put a Gradio interface (a simple browser front end) on top to test it. Juno's first test sentence came back in under five seconds.

What does a 1.42% word error rate actually mean?

Word error rate (WER) is the share of words a system gets wrong. For TTS, the usual method is to have the model speak a sentence, transcribe the audio with a speech recognizer, and compare the transcript to the original text. A 1.42% WER means roughly 1 or 2 wrong words per 100.

That sounds excellent, but three caveats apply:

1. It is the authors' own test. The reviewer said this directly: the model was run on its makers' hard test set, so treat it as their claim.
2. It is an average. A model can score well overall and still fail repeatedly on one category, such as long digit strings.
3. Small samples feel different. A single 30-word sentence with one slip is not proof the benchmark is wrong. But if a model slips on most dense sentences you throw at it, the average probably does not reflect your content.

What did the hands-on tests find?

The reviewer ran three dense test sentences. Each packs in the content types the model card says it handles.

His summary was that simple sentences sound pretty good, but once you add complex names, titles and especially numbers, it "starts cracking." He also thought Juno sounded better than Mira.

One note on the second test: the transcript of the audio is not clean enough to say exactly which words were wrong beyond the fraction. The takeaway is the pattern, not a word-by-word score.

Why do numbers and names trip up small TTS models?

Written text is ambiguous. "3/4" could be "three quarters," "three over four" or a date. "1,200" is "twelve hundred" or "one thousand two hundred." "415-555-0199" should be read digit by digit, with no word for the dashes. "VIII" after "Henry" is "the eighth." A TTS system has to decide all of this before it makes any sound. This step is often called text normalization.

Large models learn many of these rules from huge amounts of data. A 40 MB model has little room to store them, so it relies on shortcuts, and shortcuts break on unusual patterns. That is a reasonable explanation for the reviewer's results, though the sources do not confirm the model's internal design, so treat it as general background rather than a diagnosis.

Names are a separate problem. Uncommon spellings like "Siobhan" depend on pronunciation knowledge that is hard to compress.

How does Underdog Woof compare with larger models and Kitten Nano?

The model card includes a chart. The horizontal axis is download size and the vertical axis is how many words the model gets wrong on hard sentences. The best spot is the top left: small and accurate. The card places Woof there.

According to the reviewer's reading of that chart:

• Models as accurate as Woof, which he named as Fish and "U-Drama 3" (the spoken name is unclear in the source), are tens of gigabytes. In his own testing, he found Fish or U-Drama 3 "way better."
• Kitten Nano, closer to Woof in size, makes far more mistakes on the card's chart. The reviewer's own remark on how Kitten Nano performed in his tests was ambiguous, so there is no clear hands-on verdict on that comparison.

So the size claim is believable: it is a very small model. The "nothing is both smaller and more accurate" claim is a statement about the models the authors chose to test, on a benchmark they chose. The reviewer's closing view was that not all of it holds up at the moment.

Can you use Underdog Woof for commercial work?

Check the license first. Per the reviewer, the model is free for non-commercial use with credit, but the commercial license is not Apache 2, a widely used permissive license. He called this a huge blocker. If you plan to put the voice in a product, a paid service or client work, read the license text on the model page before building anything.

What does this mean for you?

If you are a hobbyist or developer who wants offline narration, notifications or a quick prototype, this is worth trying. Install cost is small and nothing leaves your machine. Stay with simple sentences.

If you work in a business setting (finance, support, logistics, healthcare), be careful. Spoken invoice numbers, dollar amounts, phone numbers and dates are exactly where this model slipped. A wrong digit read aloud is a real error, not a cosmetic one.

If you are evaluating any TTS tool, run your own test rather than trusting a benchmark. Here is a quick method:

1. Collect 20 to 30 real sentences from your own content, including the ugliest ones.
2. Include names, prices, dates, phone numbers, abbreviations and your longest sentences.
3. Generate audio in each voice and listen with the text in front of you.
4. Mark each sentence pass or fail, and note the type of content that failed.
5. Compare two or three models on the same sentences.

You will learn more from that than from any single percentage.

Could you build a voice-check tool for your own team?

If your team needs to compare voices or review generated audio, a small internal tool beats a shared spreadsheet. With Remy, you describe the app in plain English, for example a page where people paste a script, pick a voice from the available audio models, play the result, and log pass or fail against categories like numbers, names and dates. Remy builds and deploys it as a full-stack app with a database and sign-in, so reviewers can share results. It would not fix a model's pronunciation, but it would keep your test sentences and scores in one place.