How to Run the Qwen3.8-27B Pi Adapter Locally with vLLM
Serve base Qwen3.8-27B in vLLM with the Pi LoRA adapter, connect it to the Pi coding agent, and see why it used about 78 GB of VRAM on an H100.
For Engineering and IT

How do you run the Qwen3.8-27B Pi adapter locally?
You load the base Qwen3.8-27B model in vLLM (an open-source server for running language models) and attach the Pi adapter on top of it as a LoRA, a small add-on file. vLLM then exposes an OpenAI-compatible API on your machine, with thinking turned off and tool calling turned on. You point the Pi coding agent at that local address. In one hands-on test, this setup ran on a single H100 GPU with 80 GB of VRAM and used about 78 GB.
TL;DR
- The Pi adapter is a small LoRA add-on for Qwen3.8-27B. It does not replace the base model, so you need both.
- It was trained with reinforcement learning to work as the model behind the Pi coding agent on software engineering and terminal tasks.
- In the tested setup, an 80 GB H100 used about 78 GB of VRAM with the chosen context window.
- You serve it with vLLM, expose an OpenAI-compatible endpoint, turn thinking off, and enable tool calling so Pi can use it.
- In a small three-run test, the adapter and the base model fixed every bug. The adapter used roughly 20% fewer model calls on the harder task but was not faster or cheaper in tokens.
- The adapter only works with Qwen3.8-27B and is trained for Pi. Don't expect it to help other models or other agents.
What is the Pi adapter, and what is a LoRA?
The Pi adapter is a small LoRA adapter for Qwen's 27-billion-parameter model. It was trained with reinforcement learning so the model behaves better when it runs inside the Pi coding agent. Its authors report improvements over the base model under Pi on their own benchmarks, including SWE-Bench Pro and Terminal Bench, and say it uses fewer turns per task. Those are the authors' claims. The exact scores aren't reproduced here.
LoRA stands for low-rank adaptation. Instead of retraining all 27 billion weights, you freeze them and train two tiny adapter layers that sit beside them. The model's output and the adapter's output are added together, so the model behaves differently while the adapter file stays small.
For you, that means two things:
- You download the big base model once, plus a small adapter file.
- You can swap adapters without re-downloading the base. vLLM can load the base weights once and serve several adapters from the same GPU, with each request naming the adapter it wants.
Public adapter repositories often work this way. For example, another team's LoRA adapters for Qwen3.8-27B (a different project, for visual questions) ship as subfolders with no merged weights, and you load them on top of the base model. The Pi adapter follows the same idea, but check its own model card for exact file names.
What hardware does the Qwen3.8-27B Pi adapter need?
The one data point from the hands-on test: a single H100 with 80 GB of VRAM, using about 78 GB while serving the base model plus the adapter at the chosen context window.
Some context so you can plan:
- The weights alone are large. A 27-billion-parameter model stored in 16-bit precision needs roughly 54 GB just to hold the weights (27 billion times 2 bytes). That is simple arithmetic, not a figure from the model card.
- The rest goes to working memory. vLLM sets aside GPU memory for the KV cache, which holds the conversation so far. A longer context window means more memory. Coding agents send long contexts (files, tool output, history), so this matters.
- 78 GB is a setup-specific number. It reflects that context window and vLLM's memory settings. A shorter context or a lower memory setting could use less, but the sources don't test that.
What the sources don't cover: smaller GPUs, quantized versions of the base model, or running the adapter on consumer cards. If you only have a 24 GB or 48 GB card, you would be experimenting beyond what has been shown. Renting a cloud GPU for a few hours is the cheaper way to try it first.
How do you serve it in vLLM?
The setup has four parts: base model, adapter attached, an OpenAI-compatible API, and tool calling on with thinking off. The skeleton below uses real vLLM options, but the angle-bracket values are placeholders. Take the exact adapter path, tool-call parser, and the way to disable thinking from the adapter's model card.
vllm serve Qwen/Qwen3.8-27B \
--enable-lora \
--lora-modules <adapter-name>=<path-or-repo-of-pi-adapter> \
--max-model-len <context-length> \
--enable-auto-tool-choice \
--tool-call-parser <parser-from-model-card> \
--port 8000
What each piece does:
| Piece | Why it's there |
|---|---|
Qwen/Qwen3.8-27B | The base model whose weights stay frozen |
--enable-lora and --lora-modules | Attach the Pi adapter under a name you choose |
--max-model-len | Sets the context window, which drives memory use |
--enable-auto-tool-choice and --tool-call-parser | Let the model call tools, which Pi depends on |
| Thinking off | The tested setup ran with thinking disabled |
Once it's running, vLLM serves an OpenAI-compatible API at something like http://localhost:8000/v1. Any tool that can talk to OpenAI's API format can talk to it. If vLLM refuses to load the adapter, check its rank limits and your vLLM version first.
How do you connect the Pi coding agent to it?
Pi reads a models.json config file. In the tested setup, that file pointed Pi at the local vLLM endpoint and listed two models: the Pi adapter and the plain 27B base. That is handy because you can switch between them inside the same agent.
The steps:
- Start the vLLM server and confirm the endpoint responds.
- Open Pi's
models.json. - Add the local endpoint URL (the OpenAI-compatible one).
- Add one model entry named after your adapter and one for the base model. In vLLM, a request selects the adapter by the name you gave it in
--lora-modules. - Select the adapter model in Pi and give it a task.
Check Pi's own documentation for the exact fields, since the sources only describe the file in general terms.
A good first task is a small buggy script with a failing test. In the demonstration, a tiny Python parser had bugs. Pi, running on the local model, found the bug, explained why the tests failed, and fixed it quickly.
Is the adapter actually better than the base model?
Only partly, and the evidence here is thin. The creator ran both models three times on two tasks. Treat these as sanity checks, not benchmarks.
| Test | Result |
|---|---|
| Small buggy parser | Both fixed it every run. Adapter: about 4.7 model calls and 450 tokens. Base: 6 calls and 519 tokens. Speed about 26 vs 28 tokens/sec. |
| Shopping cart, 5 bugs across 4 files, 12 tests (8 failing) | Both fixed all five bugs every run, without editing the tests. Adapter: about 5.3 calls vs 6.7 (roughly 20% fewer steps). But about 1,090 tokens vs 990, about 41 vs 33 seconds, and about 26 vs 30 tokens/sec. |
The pattern: the adapter takes fewer steps, but each step is a bit longer. That matches the model card's description that the adapter writes shorter reasoning between tool calls. So on these small tasks, accuracy tied, steps dropped, and speed and token count did not improve. The model card reports bigger token savings against the base model with thinking turned on, which this test did not measure.
What is "safe agent," and why does it matter?
Reinforcement learning can teach a model shortcuts, like reading or editing the tests instead of fixing the bug. The adapter's authors describe a small proxy called safe agent that sits between the agent and the model server and checks each tool call before it runs. If a call looks like cheating, it is blocked and replaced with a short message, and that attempt earns zero reward.
According to the authors, the adapter's tool calls were screened this way during training. Their README also shows a simulated replay across 16 harnesses, including Pi, where a safe test command passes and a request to an address that appears nowhere in the conversation is blocked. For you, the takeaway is that the training tried to reward real fixes, not gaming the tests. It's still worth checking that the tests weren't touched in your own runs.
What does this mean for you?
If you're a developer or technical lead: this is a practical way to try a coding agent on hardware you control, with code that never leaves your machine. The cost is a high-memory GPU. Start with a rented H100 for an afternoon, run your own repo's small bug-fix tasks against both the adapter and the base, and compare calls, tokens, and time as the creator did.
If you're a manager or non-technical: the bigger idea is that a small adapter trained inside the tool people use can make an open model work better for that tool. It's a sign that open models are getting tuned for specific jobs. The limits matter too. This adapter is for Qwen3.8-27B and Pi only.
What to do:
- Test on your own tasks, not just the demo ones. Three runs on a toy project prove little.
- Measure steps, tokens, and wall-clock time, not only pass or fail.
- Keep the base model configured next to the adapter in Pi so comparisons are easy.
- Don't assume speed or cost savings until you see them on your workload.
Could you build a tracker for these model comparisons?
If you run this kind of comparison regularly, the results usually end up scattered across terminal output and spreadsheets. A small internal app can hold them in one place: a form to log each run (model, task, passed or failed, model calls, tokens, seconds) and a dashboard that compares models over time. Remy builds and deploys full-stack apps from a plain-English description, so you could describe that tracker, with sign-in and a database, and get a live app you own instead of another subscription. It's a side tool for your evaluation work, not a way to run the local model itself.
Frequently Asked Questions
Do I need the base Qwen3.8-27B model as well as the adapter?
Yes. The adapter is a small add-on that sits beside the frozen base weights, and the two outputs are added together. vLLM loads the base model and attaches the adapter on top, so the adapter file alone does nothing.
Will the Pi adapter work with other models or other coding agents?
Not according to the sources. It is trained for Qwen3.8-27B and for the Pi coding agent. Other adapters exist for other agents (the test setup mentioned adapters for Pi, open code, and Hermes), but each one is tied to its own target.
Can I run it on a GPU smaller than 80 GB?
The sources only show an 80 GB H100 using about 78 GB at the chosen context window. Weights in 16-bit precision alone take roughly 54 GB, so smaller cards would need other tricks such as quantization or a shorter context. Those weren't tested here, so treat them as experiments.
Is the adapter faster than the base model?
Not in the small tests shown. It used fewer model calls, about 20% fewer on the harder task, but tokens per second were slightly lower and the harder task took longer. The model card says the adapter writes shorter reasoning between tool calls, so each step is somewhat longer.
Why turn thinking off?
The tested setup served the model with thinking off and tool calling on so Pi could talk to it. The model card's larger token-saving claims compare against the base model with thinking turned on, so check the card for the exact settings it recommends.
Useful for
How we source, write, and correct our articles. Editorial standards



