Can You Run Qwen 3.8 27B Locally With Ollama? VRAM Needs

Can you run Qwen 3.8 27B locally with Ollama?

Yes. Qwen 3.8 27B is an open-weight dense model under the Apache 2.0 license, and it runs through Ollama on a single workstation GPU. In one hands-on run on Ubuntu with a 48 GB GPU, VRAM use sat at just over 26 GB while the model worked. You can then send prompts to it from the command line with Simon Willison's LLM tool. It handles long, slow reasoning jobs, including a puzzle about reading neural network weights, though it is not fast.

What is Qwen 3.8 27B?

Qwen 3.8 is described by its makers as the most capable generation in the Qwen open-model family so far. It builds on the Qwen3.5 architecture. The 27B version is a dense model (not mixture-of-experts) with a vision encoder, so it takes images and video as well as text.

Key specs from the model card:

The card also lists a reasoning_effort setting for tuning reasoning depth, and a preserve_thinking option that keeps reasoning from earlier messages.

On the vendor's own benchmark table, Qwen3.8-27B beats its predecessor Qwen3.6-27B on coding tasks. Terminal Bench 2.1 goes from 63.4 to 73.0. SWE-bench Pro goes from 53.5 to 61.7. These are the vendor's numbers, so treat them as a starting point and test on your own work.

How much VRAM does Qwen 3.8 27B need?

The only hands-on figure in the sources is about 26 GB of VRAM in use, observed on an Ubuntu machine with a 48 GB GPU while the model was running the weights puzzle.

The source did not say which quantization Ollama served. As a rough sanity check, 27 billion parameters at 16-bit precision would need about 54 GB for the weights alone. A figure near 26 GB therefore points to a quantized build. Context length also matters, since long prompts grow the KV cache. The puzzle prompt is a big wall of numbers, so it likely pushed memory up.

Practical reading of that number:

• 48 GB GPU: comfortable, with room for long prompts.
• 24 GB GPU: likely too small for the observed footprint without a more aggressive quantization or shorter context.
• Unified-memory machines: possible in principle, but expect slower generation. Verify your own memory use rather than trusting a single data point.

How do you set it up with Ollama and the LLM tool?

The setup has three parts: Ollama to serve the model, the LLM CLI to send prompts, and a plugin to connect them. The commands below follow the standard tooling. Check the Ollama library for the exact model tag, since the sources don't give one.

# 1. Pull the model (replace the tag with the one listed in Ollama's library)
ollama pull <qwen-3.8-27b-tag>

# 2. Install Simon Willison's LLM tool and the Ollama plugin
pip install llm
llm install llm-ollama

# 3. Confirm the model shows up, then prompt it
llm models
llm -m <qwen-3.8-27b-tag> "Your prompt here"

For a big input like a dump of weights, pipe a file in instead of pasting it:

cat prompt.txt | llm -m <qwen-3.8-27b-tag>

While it runs, nvidia-smi in a second terminal shows the VRAM figure.

What was the weights puzzle?

The puzzle comes from an MIT paper, "Sparks of weight space programming." The paper's claim is that a strong model (GPT-6 Astra in the paper) can read the raw weights of another network, understand what it does, and edit a small part of them on purpose.

Two results from the paper are worth knowing:

• Reading weights from a game-playing network and naming the game it was trained on.
• Editing a small Qwen model so its math accuracy rose from 7.9% to 48.1%, without training.

The paper also reports a fix to a network that kept failing to get a car over a ramp, by changing 3.5% of its weights in one shot. The limits are just as informative. The approach could not tell which of two models was the sycophantic one from weights alone. It also failed to improve a small Gemma model at math.

How did Qwen 3.8 27B do on the puzzle?

In a run shown by creator Fahad Mirza, the model got the paper's puzzle (a giant block of raw numbers from a game-playing network) with no tools and no code. The question was simply which game the network was trained on. The visible reasoning worked out how the numbers were laid out and went through them row by row. Partway through, it noticed its own counting was off, wrote "Wait," and restarted that step. The sources don't say which answer it finally gave.

Simon Willison ran the same kind of test with the same CLI tool and the paper's prompt. His results:

Runs took between 46 minutes and over an hour each. Willison also noted that the paper blends two separate ideas: what a model can do by only reading numbers, and what it can do once it has code tools. The local test covers only the first.

Is it practical to read model weights with a local LLM?

Not yet, for most work. Three limits stand out.

1. Time. Nearly an hour per answer rules out iteration. Long visible reasoning is the cost of the approach.
2. Accuracy. Three of four games is promising, but one miss on a small test is not a benchmark. A human with a notepad could probably do the same job more accurately, just slowly.
3. Scope. Reading one small network's weights is a long way from auditing a production model for a backdoor or patching overthinking by editing a few numbers. Those are ideas the paper's direction suggests, not shown results.

Still, the experiment is a useful stress test for a local model. It checks long-context handling, patient step-by-step reasoning and self-correction on a task with a known answer. If you want to evaluate Qwen 3.8 27B on your own hardware, a prompt with a verifiable answer and a long input tells you more than a short chat does.