How to Run Hy-Memory-30B-A3B Locally with vLLM for Mem0 and MemOS

How do you run Hy-Memory-30B-A3B locally and use it with Mem0?

Serve the model with vLLM using --served-model-name hy-memory, then call it through vLLM's OpenAI-compatible endpoint. Point your memory framework's LLM setting at that endpoint and use hy-memory as the model name. The model is a ~3B-active-parameter memory writer from Tencent Hunyuan. It replaces the general-purpose LLM that Mem0, MemOS or Hy-Memory normally use to decide what to store. For fact extraction, pass reasoning_effort: "no_think" through chat_template_kwargs.

What does a memory writer model actually do?

An agent memory system has two halves. The read path retrieves stored memories and feeds them to the answering model. The write path takes new conversation or task content and decides what becomes a memory. Hy-Memory-30B-A3B targets the write path only.

According to the model card, the writer decides three things: what information to keep, how precisely to represent it, and how new evidence should update the existing memory bank. That last part matters. A writer that only appends facts produces a bank full of duplicates and contradictions. A good one revises or replaces entries when a user's situation changes.

Swapping the writer is a contained change. Tencent's evaluation holds the harness version, prompts, retrieval configuration, answer model and judge fixed, and varies only the writer. Your retrieval and answering stack can stay as it is.

How was Hy-Memory-30B-A3B trained?

Training has two stages. The first is Memory On-Policy Distillation (OPD), where a teacher gives dense supervision on chains of memory state that the student itself wrote. The second is Attributed Memory Reinforcement Learning. It examines the resulting memory banks, assigns credit at the chunk level, and carries the selected bank forward across windows.

Both stages train inside the same deployed harness configurations. That is why the model is pitched as a drop-in for Mem0, MemOS and Hy-Memory, and why EverMemOS, which is held out from training, serves as a transfer test.

How do you serve it with vLLM?

The model card says it was tested with vLLM v0.31.0. The one flag it calls out is the served model name. A minimal launch looks like this:

vllm serve tencent/Hy-Memory-30B-A3B --served-model-name hy-memory

Add whatever parallelism, context length or memory options your hardware needs. The card does not list them, and it gives no VRAM requirements, so size your GPUs yourself. By default vLLM listens on port 8000, which matches the base URL in the card's example: http://127.0.0.1:8000/v1.

The served name matters because every client call and every framework config must use the same string. If you launch without --served-model-name, vLLM exposes the model under its path, and requests using hy-memory will fail.

How do you call it in no-think mode?

Use the standard OpenAI Python client against your local server. The key is the extra_body parameter, which passes a chat template argument to the model.

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="hy-memory",
    messages=[
        {
            "role": "system",
            "content": (
                "Extract explicit user facts and preferences from the conversation. "
                "Write each memory as a concise, self-contained statement. "
                "Do not infer or invent information. "
                'Return only a JSON object with a "memories" list of strings.'
            ),
        },
        {
            "role": "user",
            "content": (
                "My name is Alex. I live in Seattle and work as a software engineer. "
                "I mainly use Python and prefer concise answers with code examples. "
                "I'm vegetarian, so please keep that in mind when suggesting meals."
            ),
        },
    ],
    temperature=0.0,
    max_tokens=512,
    extra_body={
        "chat_template_kwargs": {
            "reasoning_effort": "no_think",
        },
    },
)

A few details are worth copying. The API key is a placeholder because vLLM does not require one by default. Temperature is 0.0, so repeated runs on the same input stay consistent. The system prompt asks for self-contained statements and forbids inference, and it requires a JSON object with a memories list. Parse the message content with json.loads and write each string to your store.

No-think mode skips extended reasoning and returns the answer directly. For turning a short user message into a handful of facts, that cuts latency and token use. This is a reasonable inference from how the setting works. The card does not publish latency numbers.

How do you plug it into Mem0, MemOS or EverMemOS?

The sources do not include framework-specific configuration, so treat this as the general pattern. Each framework has a setting for the LLM that does extraction and memory updates. Point that setting at an OpenAI-compatible provider with your vLLM base URL and the model name hy-memory. Leave retrieval, embedding and the answering model alone.

Two practical notes:

• Keep the framework's own extraction prompts. The model was trained inside the deployed harness configurations, so changing the prompts moves you away from the setup it was tuned for.
• Check whether your framework version lets you pass chat_template_kwargs per request. If it does not, you may need to set no-think behavior on the server side or through a thin proxy. Verify this against the framework's documentation.

Is Hy-Memory-30B-A3B better than larger models at memory writing?

On Tencent's benchmark matrix, it comes out on top by average. Each harness score is the mean of six metrics (LoCoMo Judge, LongMemEval, PersonaMem-v2, Travel SPS, Shopping PS and Code Memory Bench), each run on a fixed 20% subsample.

Read these numbers with some care:

• The margins over Claude Opus 5 are small. On MemOS the gap is 0.01 points, and on the Hy-Memory harness it is 0.02.
• The larger gains are against comparable-size open models. Against Qwen3-30B-A3B, the average gap is over 11 points.
• It does not win every metric. On the Mem0 harness, Claude Opus 5 scores 69.47 on LoCoMo against 59.54 for Hy-Memory, while Hy-Memory leads on LongMemEval (73.00 vs 66.00) and Travel (54.51 vs 42.98).
• Shopping PS scores sit around 14 to 24 for every writer, so that metric separates models poorly.
• The EverMemOS result is the most interesting one, since that harness was held out from training. The model scored 56.89 there, ahead of DeepSeek V4.1 Flash at 55.47.

The evaluation comes from the model's developer, and every benchmark in the main four-harness matrix is evaluated on a fixed 20% subsample. Run a test on your own conversation data before trusting it.

What should you check before putting it in production?

Start with your own hardware. A 30B-total model with 3B active parameters is cheap per token, but the full weights still need to fit in memory. Then test extraction quality on real transcripts, including edge cases like corrections ("actually, I moved to Portland") to see whether the bank updates instead of duplicating. Finally, validate that the JSON output parses reliably under your prompts, and add a retry or fallback for malformed responses.