---
title: How to Run TwIL LM3 Pro Locally With llama.cpp
url: "https://goremy.ai/blog/articles/twil-lm3-pro-run-locally-llama-cpp"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-11-11T07:00:15.513Z"
updated: "2026-11-11T07:00:18.459Z"
topic: Local AI
description: "Run the 3.66B TwIL LM3 Pro logic model locally with llama.cpp: VRAM needs, quantized builds, a Hermes agent setup, and the non-commercial license."
---

# How to Run TwIL LM3 Pro Locally With llama.cpp

> Run the 3.66B TwIL LM3 Pro logic model locally with llama.cpp: VRAM needs, quantized builds, a Hermes agent setup, and the non-commercial license.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-11-11

## How do you run TwIL LM3 Pro locally?

Download a GGUF build of the model, start it with llama.cpp's `llama-server`, and point a client or agent at the local address. TwIL LM3 Pro has 3.66 billion parameters and is built from IBM's Granite 4.2 3B. Its model card says a Q4_K_M build is 2.09 GiB and fits on a CPU or 4 GB of VRAM. A creator who served it with llama.cpp and the Hermes agent showed about 9 GB of VRAM in use. The license is non-commercial.

## TL;DR

- **TwIL LM3 Pro is a narrow specialist.** It is a 3.66B model from webAI, fine-tuned from IBM's Granite 4.2 3B for formal logic, such as checking whether statements contradict each other.
- **VRAM depends on the build and settings.** The card lists a 2.09 GiB Q4_K_M file for CPU or 4 GB of VRAM, and a 6.82 GiB bf16 checkpoint. A demo showed about 9 GB in use while serving.
- **llama.cpp handles the serving.** `llama-server` exposes a local endpoint that agents like Hermes can use.
- **It thinks out loud, and at length.** It writes a `<think>` block before the answer, averaging 1,902 tokens on the card's logic tests, so budget for long outputs.
- **Use it for logic checks, not general work.** It did well on contradiction-spotting in specs and weaker on harder backward-chaining puzzles. The card says it is not a general assistant.
- **The license blocks commercial use.** It is the webAI Non-Commercial License ver. 1.0. Revenue-generating use needs a separate agreement with webAI.

## What is TwIL LM3 Pro, and what is it for?

TwIL LM3 Pro is a small "reasoning model" from webAI. A reasoning model works through a problem step by step before it answers. This one was trained for formal logic: translating statements into first-order logic, labeling whether one statement follows from another, parsing meaning into structure, and writing and critiquing Lean statements (Lean is a proof language).

The base is `ibm-granite/granite-4.2-3b`. The card describes the training as LoRA supervised fine-tuning, checkpoint fusion, weight interpolation, and reinforcement learning. The video's summary of the last step is useful: the model was rewarded for answers a checker could prove right, not for sounding convincing.

The practical use is a job most teams dislike: reading a requirements document, policy or rulebook and finding clauses that cannot both be true.

## How much VRAM does TwIL LM3 Pro need?

VRAM is the memory on your graphics card. A model has to fit there (or in system RAM, more slowly) to run at a usable speed. The numbers below come from the model card, except the last row.

| Build | Size | What the source says |
|---|---|---|
| Q4_K_M GGUF | 2.09 GiB | Runs on CPU or 4 GB of VRAM |
| bf16 checkpoint | 6.82 GiB | The full-precision-style weights |
| Other GGUF builds (Q4_0, Q5_K_M, Q6_K, Q8_0, F16) | Not listed in the material I have | Available per the card |
| Demo setup with llama.cpp | About 9 GB VRAM in use | Shown in a creator's video; quantization not stated |

Why would a 2 GB file use 9 GB? The file size covers only the weights. While running, the server also holds a context cache, which grows with the context window you set. Larger quantizations also take more room. The video did not say which build or settings it used, so treat 9 GB as a comfortable ceiling to plan around, not a requirement. If you have a 12 GB or 16 GB card, you will have room to spare. On a smaller card, start with Q4_K_M and a modest context size.

The model's context window is 131,072 tokens, inherited from Granite. The card is clear, though, that every score was measured inside an 8,192-token window. Longer contexts are untested by the authors.

## How do you set it up with llama.cpp?

llama.cpp is an open-source program that runs language models on ordinary hardware. Its `llama-server` tool turns a model file into a local web service.

1. **Install llama.cpp.** Use a prebuilt release or build it from source, following the project's instructions for your GPU.
2. **Download a GGUF file.** GGUF is the file format llama.cpp reads. Check the TwIL-LM3-Pro page on Hugging Face (`webAI-Official/TwIL-LM3-Pro`) for the GGUF builds the card lists. Start with Q4_K_M if memory is tight, or a larger build if you have room.
3. **Start the server.** The filename below is a placeholder, so use the one you downloaded.

```
llama-server -m TwIL-LM3-Pro-Q4_K_M.gguf -c 8192 -ngl 99 --port 8080
```

`-c 8192` sets the context size to match what the model was evaluated on. `-ngl 99` asks llama.cpp to put all layers on the GPU. Drop it, or lower it, if you are running on CPU or a small card.

4. **Check it works.** `llama-server` offers an OpenAI-compatible API, so any tool that can talk to an OpenAI-style endpoint can talk to it at `http://localhost:8080/v1`.
5. **Match the evaluation settings when testing.** The card's scores used greedy decoding (temperature 0), 2,048 new tokens with one retry at 4,096. Set your output limit well above 2,000 tokens, or answers will get cut off mid-reasoning.

## How do you connect it to a Hermes agent?

In the video, the model was served with llama.cpp and attached to the Hermes agent. The creator then asked it to read local files, work through the reasoning, and report back. The agent used tools to read the document, and the model did the logic.

I don't have Hermes' configuration details from the sources, so check its documentation for the exact setting. The general approach is the same for most agent tools: add a custom model provider, enter your local server address, and give it any model name your server accepts. You can also skip the agent entirely and paste a document into a chat window pointed at the same server.

A prompt template for the use case that works best:

```
You are reviewing a requirements document for logical contradictions.
Below are numbered clauses. Find every pair of clauses that cannot
both be true. For each pair, give the clause numbers and a one-sentence
reason. Do not flag clauses that are merely related or that overlap
without conflicting. If you are unsure, say "cannot tell" and explain.

[paste clauses here]
```

The "do not flag related clauses" line matters. In the demo, the model avoided decoys, and this instruction asks for the same behavior.

## How well does it actually work?

The video's tests are small, informal checks, not benchmarks, but they show the pattern.

| Test | Result |
|---|---|
| Grocery delivery spec, three planted contradictions | 3 of 3, plus one arguable extra flag |
| Electronics return policy, 10 customer cases | 8 of 10, including "cannot tell" answers |
| 12-clause festival ticketing spec, three hidden contradictions | 3 of 3, no decoys flagged |
| Invented chemistry rules, 8 questions | 4 of 8 |
| Invented masked ball rules, 8 questions | 5 of 8 |

The misses share a cause. The model handled forward chains well. It was shakier when it had to reason backward from a negative fact, and it sometimes answered "cannot tell" too early. In one case it reversed a rule that only works one way.

The model card's numbers agree. On in-domain formal logic, it posts the highest macro gate score (0.5539) in the card's tables, ahead of Qwen3-8B (0.5336). The card itself says that gap is smaller than the sampling noise, so read it as parity. On held-out general benchmarks it trails Qwen3-8B and gpt-oss-120b. It is a specialist that holds its own on home turf.

The card also lists costs. It reasons at length (1,902 tokens on average on the logic tests), and 24.2% of generations hit the length cap. There is no safety or preference tuning beyond what Granite 4.2 carries.

## Can you use TwIL LM3 Pro commercially?

No, not under the standard license. The model is released under the webAI Non-Commercial License ver. 1.0. The video states that research and personal projects are fine, and that anyone generating revenue with it needs a separate agreement with webAI.

What counts as commercial for your situation, such as internal use at a company, is a question for your legal or compliance team, not a blog post. If your work involves a business use, ask before building on it. The video's creator also noted that closed models can do similar logic checks, so if licensing is a blocker, you have alternatives.

## When does running it locally make sense?

Local makes sense when the documents are sensitive, such as draft contracts, internal policies or unreleased specs, and you don't want to send them to an outside service. It also fits personal projects, experiments, and cases where you want to run many checks without per-request fees.

It makes less sense if you need a general assistant, coding help or writing help. The card and the video both say to keep it to formal logic. It is also a poor fit if your use is commercial and you can't get a license.

## Could you build a tool around a local logic model?

If you wanted a shared place for people to paste a spec and get a contradiction report back, you could build that as a small internal app. [Remy](https://goremy.ai) builds full-stack apps from a plain-English description, with a database and sign-in included, and it can reach local models over a secure tunnel. You could describe a "spec review" form that stores each document, the flagged clause pairs and who reviewed them. Whether TwIL LM3 Pro's license allows that use at your company is a separate question to settle first.

## Frequently Asked Questions

### Can TwIL LM3 Pro run without a GPU?

Yes, in principle. The model card says the Q4_K_M GGUF, at 2.09 GiB, runs on CPU or 4 GB of VRAM. Expect it to be slower on CPU, and slower still because the model writes long reasoning before answering.

### Is TwIL LM3 Pro good for coding or general chat?

No. It is tuned for formal logic and the card says it is not a general assistant. The video also advises against using it for generic, productivity or coding tasks.

### What is the difference between TwIL LM3 Pro and its base model?

The base is IBM's Granite 4.2 3B. The card reports an in-domain macro gate score of 0.5539, the highest in its tables, while on held-out general benchmarks it trails Qwen3-8B and gpt-oss-120b.

### Why does it produce a long `<think>` block?

The default chat template makes the model write its reasoning before the answer. That is how it works through clauses and catches its own mistakes, but it means slower responses and a need for a generous output limit.