How to Run Raon-1 mini Locally: NVFP4, RTX 5090 and Context Needs

What hardware do you need to run Raon-1 mini locally?

You need one NVIDIA RTX 5090 (32 GB VRAM) and the NVFP4 build. According to the model card, NVFP4 is the quantization that enables inference on a single RTX 5090. Raon-1 mini is a 28.8B-parameter Mixture-of-Experts model with 4.3B active per token, released by KRAFTON AI under Apache 2.0. An MXFP8 build also exists, but the card ties the single-GPU claim to NVFP4 only. The model is a preview release, so specs and results may change.

Why does NVFP4 matter for a 32 GB card?

Weight memory is the first constraint. All 28.8B parameters must be resident, even though only 4.3B are active per token. In BF16 that is roughly 57 GB, far past any consumer GPU. At 8-bit it is roughly 29 GB before overhead, which leaves almost nothing on a 32 GB card for the KV cache, activations and runtime buffers. At 4-bit it drops to roughly 15 GB before scale factors and overhead.

Those are back-of-envelope figures from the parameter count, not numbers from the card. The card makes one concrete claim: NVFP4 enables inference on a single RTX 5090 with 32 GB. That is consistent with the arithmetic, since a 4-bit build leaves real headroom for context.

NVFP4 is also a Blackwell-oriented format, and the RTX 5090 is a Blackwell card. The card does not say how well NVFP4 runs on older GPUs, so check your inference stack before assuming it does.

What is special about the Raon-1 mini architecture?

Raon-1 mini is a hybrid. Of its 25 layers, 18 use linear attention and 7 use full attention, a 3:1 ratio. The model card names these components:

• Raon Delta Attention (RDA). It decouples the erase and write gates of Kimi Delta Attention. KRAFTON says this improves long-context performance over KDA.
• Gated MLA. The full-attention layers use multi-head latent attention with a 576 query rank and a 512 KV LoRA rank.
• Raon-mHC. A cheaper take on DeepSeek's manifold-constrained hyper-connections, with 4 residual streams. The card says it speeds up training.
• NoPE. The model uses no positional encoding.
• Sparse MoE. 128 routed experts, 6 active plus 1 shared, with an expert intermediate size of 1,088.

Other listed specs: hidden size 2,560, 40 attention heads, vocabulary of 259,840 tokens, untied LM head, and SiTU-GLU activation. The first layer uses a dense FFN with an intermediate size of 12,032.

The practical point is the layout. The 18 RDA layers hold a fixed-size state. They do not grow with sequence length.

How much memory does the KV cache need at 262K context?

The card says only the 7 MLA layers keep a KV cache, about 8 KiB per token in BF16. Multiply that by the native window:

• 8 KiB × 262,144 tokens = 2,097,152 KiB, or about 2 GiB.

That is a calculation from the card's figure, not a measured result. Real usage will add the fixed RDA state, activations and framework overhead. Still, a cache of that size is small next to what a conventional full-attention model of similar scale would need at the same length. That is the reason the architecture matters for local use. On a 32 GB card, a 4-bit weight footprint plus a roughly 2 GiB cache leaves room to spare, at least on paper.

If your runtime stores the cache in a lower precision than BF16, the figure should shrink further. The card does not say whether that is supported.

What is DFlash2 and how does it speed up inference?

The release includes a DFlash2 draft model for speculative decoding. In speculative decoding, a small draft model proposes several tokens and the main model verifies them in one pass. When the draft guesses well, you get more tokens per forward pass of the large model. Output quality stays the same, because the large model accepts or rejects each proposal.

The card does not publish speedup numbers or say which serving frameworks support DFlash2. The draft model also takes extra VRAM. On a 32 GB card that trade-off is worth measuring before you assume it is a free win.

Does Raon-1 mini work with agent harnesses?

The card lists tool calling for agentic workflows and names three harnesses: Hermes Agent, Pi and OpenCode. KRAFTON positions the model as strong in Korean conversation and agentic tasks. It also has built-in thinking, so expect reasoning tokens before answers.

Training covered Korean, English, Japanese, Chinese, Spanish, French and Thai. The card describes Korean performance as strong and English as solid.

How does Raon-1 mini compare on benchmarks?

The card compares the model with Qwen3.6-35B-A3B and Gemma 4 26B-A4B (MoE), plus Qwen3.8-27B and Muse Glimmer 30B (dense). The visible results are mixed.

It leads the MoE group on GPQA Diamond, AIME 2026 and AA-Omniscience, and trails clearly on MMLU Pro. The card also claims leading results on instruction-following and search benchmarks, but that part of the table was cut off in the material available here. These are vendor-reported numbers, so run your own evals on your own tasks.

Is Raon-1 mini worth running locally?

It makes sense if you want an Apache 2.0 model with long context on a single prosumer GPU, or if Korean-language quality matters. The cheap KV cache is the main hardware advantage. It is a weaker fit if broad factual knowledge is your priority, given the MMLU Pro gap. The preview label also means you should expect revisions.