Blog
Haru-4BHaru-4B localfull duplex speech-to-speech model

How to Run Haru-4B Locally: VRAM Needs, Setup and Results

Haru-4B is an open-weight voice model that runs in about 12 GB of VRAM. Here is the setup, the persona and voice prompts, and how it did in live tests.

Portrait of Luis Chavez-Mattos
Edited by Luis Chavez-Mattos, Director, Product··RSS

For Engineering and IT

Overhead flat-lay of a wired headset with microphone at the centre, arranged with a graphics card, laptop, recorder, memory module and blank cards in a grid.

Can you run Haru-4B on your own GPU?

Yes, if you have an NVIDIA GPU with about 12 GB of memory. That is the figure on the model card, and a reviewer who ran it locally saw roughly 12 GB of VRAM in use (on an H100, which has far more). Setup is three commands and a browser tab. Whether it works well is a different question. In live trials the model repeated itself and gave replies unrelated to what was said, and only English and Japanese held up. The developer calls it a research base model, and the results fit that label.

TL;DR

  • Haru-4B is a full-duplex speech-to-speech model: it listens and talks at the same time, like a phone call, instead of waiting for you to finish.
  • It needs about 12 GB of VRAM on an NVIDIA GPU, and it is released under the Apache 2.0 license, so the weights are free to use and modify.
  • You control it with a text card for the character and a short voice clip you supply. It ships with no voice clips.
  • It lists nine languages, but a hands-on tester found only English and Japanese worked.
  • In a live test with a custom persona, the model repeated phrases and drifted off topic, even on the developer's own demo input.
  • The architecture is the interesting part. The conversation quality is not there yet.
  • Treat it as something to experiment with, not something to put in front of customers.

What is Haru-4B?

Haru-4B comes from Higurashi, a Japanese company. It is built on the Qwen3.5-4B text backbone (32 layers, hidden size 2560) and released as open weights under Apache 2.0.

The key term is full duplex. Most voice assistants are half duplex: you speak, it detects you stopped, then it answers. A full-duplex model processes your voice while it generates its own, so in theory it can interject, laugh, say "uh-huh" or get interrupted mid-sentence, the way people do.

"Speech-to-speech" means audio goes in and audio comes out from one model, with no separate transcription step and no separate text-to-speech step.

Higurashi describes it as a pretrained base model for research. Its own limitations section says it can hold a natural conversation, but it is not a reliable source of facts and does not follow instructions the way an assistant does.

How much VRAM does Haru-4B need?

VRAM is the memory on your graphics card. The model card says to expect about 12 GB on an NVIDIA GPU. The reviewer's test machine used an H100, a data-center card with much more memory, and the model used around 12 GB of it. That matches the model card.

Practical reading of that:

  • A card with 12 GB or more is the stated target. Anything smaller is unverified.
  • The weights are stored in bfloat16 and shipped as a single file, with the language model, audio projections, audio head and audio codec all included.
  • Higurashi also publishes an MLX implementation with a native macOS app, and a plugin for serving through vLLM-Omni. Those exist in the repository, but the test described here used the NVIDIA path.

How do you set up Haru-4B locally?

The commands below come from the model card:

git clone https://github.com/higurashi-ai/Haru && cd Haru
pip install -e ./haru
python -m haru.server --hf-repo higurashi-ai/Haru-4B

Then open http://localhost:8998 in your browser. You pick a character, give the model a voice, and talk.

A few details matter:

  1. Use wired audio if you can. The developer recommends a wired headset and microphone for low latency. Bluetooth adds 100 ms or more.
  2. Supply your own voice clip. Haru ships without voice clips. The clip should be one speaker, between 3 seconds and 3 minutes, with clean speech. Only use voices you have the right to use, since the model can imitate a reference voice.
  3. Write a persona card. The system prompt is a character card with a fixed layout. The repository's README shows how to build one.

For developers, there is also a Python interface. You load the model, prepare a reference voice from a WAV file, and create a session with your character card. Then you step through it one frame at a time, sending 3,200 audio samples (at 24 kHz) in and getting 3,200 out.

What does a persona card look like?

In the test, the reviewer wrote a persona in plain language: Mia, a 34-year-old home baker who runs a weekend bakery stall, cheerful, talkative, laughs a lot, on a phone call with a close friend named Sam. Then they recorded their own voice clip, pressed record, and asked about the weekend: "I tried baking sourdough for the first time and it came out like a brick. What's the worst thing you have ever burned in the oven?"

That is a fair test. It is casual small talk, the kind of thing a conversational model should handle.

How did Haru-4B perform in live tests?

Poorly, in the one hands-on trial available. Here is what happened.

On the developer's own demo. The model card's example plays a spoken passage and expects a proper response. The model replied roughly with "I'm Haru, 22 years old," repeating the phrase. That answer had nothing to do with the question, and the voice sounded similar to the reference.

On the custom persona. The model responded quickly (speed was not the problem), but the output repeated phrases and reused fragments of earlier answers. It did not engage with the baking question in a coherent way.

On the technical blog's demo. A conversation with a "financial adviser" persona sounded less than natural to the reviewer, though better than the live local results.

Some fairness is needed here. This is one reviewer, one machine and a handful of prompts. The developer says it is a base model that people are expected to fine-tune, and it has published training pipelines and a GitHub repository. But the reviewer's fair question stands: if fine-tuning is what makes it work, why is there no fine-tuned model to demo?

Which languages does Haru-4B support?

The model card lists nine: English, Spanish, Portuguese, German, French, Chinese, Japanese, Russian and Korean. In testing, the reviewer found it did not do anything useful beyond English and Japanese. If you need another language, assume it is not ready until you test it yourself.

How does the full-duplex design work?

This is where the project is most interesting, and the reviewer agreed.

Audio is cut into frames of about 133 milliseconds (7.5 frames per second). In each frame, the model takes three things and adds them together into one position of the language model:

  • your latest slice of voice
  • its own previous slice of voice
  • its own previous word

From that, the model predicts the next word and the next slice of its own voice in the same step. Listening and speaking never wait for each other.

Two design choices stand out. The audio is a continuous 64-dimensional representation per frame, not the chunky discrete audio tokens many speech models use. And a small head (a four-layer MLP using a method called MeanFlow) produces it in a single sampling step, with no separate audio transformer.

Higurashi also contrasts this with the older "cascaded" pipeline: speech recognition, then a language model, then text-to-speech, each waiting on the last. It illustrates that pipeline as a delay of about 2 to 3 seconds, against about 0.32 seconds for Haru. Those are the company's illustrative figures, not independent measurements. Cascading also strips out tone and emotion because everything becomes plain text first. A single model can, in principle, hear excitement in your voice.

What does this mean for you?

If you are a curious non-developer: you can try it, but expect friction (Python, a command line, an NVIDIA GPU) and a rough conversation. Do not judge voice AI by this model. Commercial voice assistants are a different category.

If you are a developer or researcher: this is a legitimate open-weight starting point for full-duplex work. The Apache 2.0 license, training pipelines and clean architecture are real assets. Budget time for fine-tuning and for evaluating it on your own data.

If you are evaluating voice AI for work: do not put a base research model in front of customers. Before trusting any model, write ten realistic conversations, run them, and score each reply as relevant or not. A reviewer's repeated "I'm Haru" is exactly the kind of failure a short test catches early.

Before you start, check:

  • You have an NVIDIA GPU with about 12 GB of memory.
  • You have a clean, single-speaker voice clip you are allowed to use.
  • You have a wired headset.
  • You are testing in English or Japanese.

Could you build your own tool to track voice-model trials?

If you test several local models, the results usually end up in scattered notes. A small shared app fixes that: a form to log each trial (model, persona card, voice clip used, language, whether the reply was relevant, whether it repeated), plus a table and chart to compare models over time. With Remy you can describe that tracker in plain English and get a deployed full-stack app with a database and sign-in, so a team can score trials together instead of passing around a spreadsheet. Remy would not run Haru itself, but it can hold the evidence you use to decide.

Frequently Asked Questions

5 questions

Is Haru-4B free to use commercially?

The weights are released under the Apache License 2.0, which generally permits commercial use. Voice cloning is a separate matter: the developer says to only use voices you have the right to use. Read the full license text before shipping anything.

Does Haru-4B work on a Mac?

The repository includes an MLX implementation with a native macOS app. The hands-on test described here used an NVIDIA GPU, so this article cannot speak to how the Mac version performs.

Why did Haru-4B repeat itself in testing?

The cause is not confirmed. Higurashi describes the model as a pretrained base model that does not follow instructions like an assistant. The reviewer suspected it needs fine-tuning, but a fine-tuned version was not available to test.

Can I use any audio clip as the voice?

The clip should be a single speaker with clean speech, between 3 seconds and 3 minutes. Haru ships without voice clips, so you provide your own.

Is the full-duplex idea worth following?

Yes. A single model that listens and speaks at once, without separate transcription and text-to-speech steps, could make voice AI feel more natural. This release shows the design on paper, but it has not shown it working reliably in practice yet.

How we source, write, and correct our articles. Editorial standards

Related articles

Straight to your inbox.
New pieces as they publish. No noise.