How to Run Ripple-65B Locally: VRAM and Setup Requirements

What do you need to run Ripple-65B locally?

You need a GPU with roughly 96 GB of memory if you want comfortable headroom. One creator who self-hosted the quantized model on an RTX Pro 6000 Blackwell (96 GB) saw about 75 GB of VRAM in use. The provided inference code did not run as shipped because it targeted a different GPU, so it had to be rewritten for Nvidia's Blackwell card. A small server then exposed the model to an agent. Expect several hours of setup.

What is Ripple-65B?

Ripple is a model from Recursive Intelligence, a small team that trained it from scratch. It is a dense transformer, not a mixture of experts. That is the opposite of the approach most large labs use now, where only a slice of the network activates for each token.

The design is thin and deep: 80 layers with a narrow width. The team describes a Gemma-style layout for stable training, more attention heads to help reasoning, and more sliding-window layers with a smaller window to hold down cost. It also uses extras such as gated attention and value residual. Pretraining ran on 12 trillion tokens, mostly English text and code. Post-training covers mid-training, a long-context stage, supervised fine-tuning, a long-context fine-tuning pass, and reinforcement learning across mixed domains. The release is a preview with light post-training, aimed mainly at agentic coding and reasoning. It is licensed Apache 2 and handles fairly long context.

How much VRAM does Ripple-65B need?

The only hard number from the self-hosting test is this: the quantized model sat at around 75 GB of VRAM on a 96 GB card. The source does not say which quantization format was used, and no official memory requirements were available to check against.

Some general arithmetic helps frame it. Weights alone for a 65B dense model take about 2 bytes per parameter at 16-bit, so roughly 130 GB. At 8-bit that drops to about 65 GB, and at 4-bit to about 33 GB. The observed 75 GB would be consistent with something near 8-bit weights plus KV cache and runtime overhead, but that is an inference, not something the source states. Your own figure will shift with the quantization format, context length, batch size and inference engine.

Because the model is dense, you cannot offload inactive experts the way you might with a mixture-of-experts model. All 65B parameters participate in every token.

Which GPUs can run it?

The tested setup was a single RTX Pro 6000 Blackwell with 96 GB. That fit the quantized model with room left over for context and overhead.

The creator also said the model can fit on a DGX Spark or a similar device. DGX Spark is a compact Nvidia system with 128 GB of unified memory, which is a general-knowledge spec rather than something from the test. That is more total memory than the 75 GB observed, so the claim is plausible. Unified memory is slower than dedicated GPU memory on a workstation card, though, so expect lower token throughput. No speed numbers were reported for either machine.

A 24 GB or 48 GB consumer or prosumer card would not hold a 65B model at the precision used in the test. Heavier quantization or multi-GPU splitting would be the route there, and neither was tested.

How long does setup take?

The tester reported two stages:

1. Base environment: about 2 to 3 hours to move to a new VM, download the model and configure everything.
2. Full pipeline: about 3 to 4 hours in total, which included the inference server, the agent connection and rebuilding the Docker-based test application.

Most of the pain was not the download. It was getting the inference path to work on different hardware.

Why did the inference code need rewriting?

The inference code released with the model was built for a different GPU than the one the tester owned. It did not run on the RTX Pro 6000 Blackwell as provided. The fix was to adjust it for that Nvidia card.

The source does not detail which parts changed. Typical causes of this kind of breakage include kernels or attention implementations tuned for one GPU generation, and hardcoded device or precision assumptions. Treat that as general context, not a description of what was edited here.

Practical advice if you hit this:

• Check which GPU the reference code targets before you provision hardware.
• Budget extra time for kernel and dtype adjustments on newer architectures.
• Test with a short prompt before wiring in an agent, so you can separate model-loading problems from agent problems.

How do you connect an agent to a self-hosted Ripple?

After fixing inference, the tester wrote a small server to serve the model, then configured the Hermes agent to talk to it. The agent received a goal with no hand-holding: find and fix a single planted bug in a full-stack retirement calculator running in Docker.

The bug was subtle. A withdrawal rate was stored as 4 but used as 0.04, which made every retirement number come out 100 times too small. The app ran without crashing and the logic read as correct, so the model had to trace the data flow. According to the test, it found the issue on its own and the corrected app produced sensible figures for the US, UK and Poland.

Earlier, the same task through hosted providers failed with a max token error. The creator tried several providers and got the same result, which is what pushed the move to self-hosting. That detail suggests that for long agentic runs, controlling your own token limits can matter.

How good is Ripple-65B for coding?

Based on the benchmark charts shown, Ripple led on a terminal-based agentic coding benchmark and a harder software engineering benchmark, ahead of models many times its size. On a deep research style benchmark it sat near the top group, with a Meta model clearly ahead. It trailed on math, safety and automation, where Meta and Reflection models led. No scores were available to quote here.

Hands-on tests in the same session included a physics and graphics simulation generated without external libraries, which came out with a recognizable structure and convincing dripping effects but weak meat textures. A complex Databricks query over a six-table schema, with traps like double-counted revenue across joins, came back as a staged, correct-looking answer. These are single runs, not a systematic evaluation. The tester noted that production verification is still needed.

Is self-hosting Ripple-65B worth it?

It depends on your hardware and goals.

It makes sense if you already own a 96 GB class GPU or a DGX Spark, want an Apache 2 model for agentic coding, and are comfortable debugging inference code. Local hosting also avoids the provider token limits that tripped the first run.

It probably does not make sense if you only have a small GPU, need a polished release, or want a turnkey setup. This is a preview, the reference code was not portable out of the box, and the setup took half a day.