How Good Is TwIL LM3 Pro at Logic and Contradictions?

How good is TwIL LM3 Pro at formal logic and spotting contradictions?

In one creator's hands-on tests, TwIL LM3 Pro was very good at finding contradictions in written specs. It scored 3 out of 3 on two separate documents, with no false alarms on decoy clauses. It was solid on rule application (8 out of 10 on a return policy). It was weaker on invented-rule puzzles (4 out of 8 and 5 out of 8), where it failed mostly when the reasoning had to run backward. On broader benchmarks, larger general models beat it.

What is TwIL LM3 Pro?

It is a small language model (3.66 billion parameters) built from IBM's Granite 4.2 and post-trained to do one job: formal logic. You give it statements, and it reasons step by step about whether they agree with each other or cannot all be true.

The training approach matters for understanding its behavior. It used LoRA supervised fine-tuning (a lightweight way to adjust a model), checkpoint merging, and reinforcement learning against a programmatic verifier. In plain terms, the model was rewarded for answers a checker could prove correct, not for answers that sounded convincing.

In the demonstration, it ran locally through llama.cpp (a popular program for running models on your own machine) and used about 9 GB of VRAM (graphics card memory). It was connected to an agent tool called Hermes, which let it read files before reasoning about them.

Can it find contradictions in a requirements document?

This is the most practical test, because contradictory requirements are a real and expensive problem. One page says an order can be changed until delivery. Another page says it locks at checkout. Nobody notices until the app is half built.

Test 1: grocery delivery app spec. The document had three planted contradictions, plus a few quieter clauses meant to tempt a false alarm. The model found all three, cited the correct clause numbers (for example, 2.1 against 9.1), and gave clear reasons. It also raised one extra flag, 3.4 against 5.2, which was not a planted bug. The creator judged it defensible, since a locked order should not accept substitution changes. That makes it a 3 out of 3 with one arguable bonus, not a false positive.

Test 2: festival ticketing spec. This was harder: 12 clauses, three hidden contradictions, and decoy clauses that look related but do not actually conflict. The model found all three with the right clause pairs and flagged none of the decoys. Another 3 out of 3.

Decoys matter because a tool that cries wolf is worse than none. A reviewer who has to check 20 false flags will stop reading the output.

How does it handle rules with "cannot tell" answers?

Test 3: electronics return policy. The policy had 10 clauses and 10 customer cases. Some cases have a clear answer. Others cannot be decided from the policy, and the right answer is "cannot tell." That is the part many models get wrong, because they feel pressure to give a definite answer.

The model scored 8 out of 10 with correct clause citations. It also correctly said "cannot tell" where the rules did not settle the case. Its reasoning was visible and instructive. It first thought the cases were missing, then read the file, laid out all 10 clauses, and worked through every case twice. It caught and fixed its own slip on case five.

This is the kind of task most workplaces actually have: here is a policy, here is a situation, what does the policy say?

Where does it fail?

The two invented-rule puzzles are where the limits show. The creator made up the rules so that memorized knowledge could not help. Each puzzle had eight questions that needed multi-step deduction.

The pattern is consistent. Forward chains (if A then B, if B then C, so A leads to C) were handled well. Backward chains (B is false, so what must be false about A?) were shaky. Reversing a one-way rule is a classic logic error, and humans make it too. Here, though, the failure is specific and predictable, which helps you decide when to trust the output.

The creator also said he stressed the model on purpose and that some of the puzzle questions are debatable. Treat these as signals, not a rigorous evaluation.

How does it compare with larger models on benchmarks?

The creator showed benchmark charts comparing it with Qwen3 8B, a 3.54 billion parameter LFM model, and the 120 billion parameter GPT-OSS. There were two tracks.

• Track A, in-domain formal logic: TwIL LM3 Pro led on the overall gate score and the strict test, and had the best average across six lanes, beating models more than twice its size.
• Track B, held-out benchmarks: the picture flipped. GPT-OSS and Qwen3 8B pulled ahead, and TwIL LM3 Pro landed in the middle of the pack.

The takeaway is the usual one for specialists. A small model trained for a narrow job can win at that job and still lose on tests it was not built for. The creator's advice was to keep it on formal logic and not try it on general productivity or coding work.

His own caveat is worth repeating: many closed models, and newer open models such as GLM, can do these kinds of tasks quickly too. The appeal here is a small, open, local option, not exclusive capability.

Is TwIL LM3 Pro worth using?

It depends on what you need.

It fits when:

• You want to check whether a conclusion follows from a set of rules or policies.
• You want a first pass for contradictions in specs or policy documents.
• Privacy matters and you want the check to run on your own hardware.
• You are doing personal or research projects.

It does not fit when:

• You need a general assistant, writing help, or coding help.
• You need the reasoning to run reliably backward from negative facts.
• You want to use it in a revenue-generating product without a separate agreement. The license is non-commercial.

Whatever you use it for, treat its output as a reviewer's shortlist. It scored well on contradiction spotting, but the 8 out of 10 and 5 out of 8 results mean a person should confirm anything that matters.

What does this mean for you?

If you write or review documents full of rules (product requirements, HR policies, refund terms, compliance procedures), this is a useful signal about where small models are headed. You do not need a giant model to catch a clause that says the opposite of another clause.

What you can do:

1. Try the task with whatever AI you already have. Paste two or three sections of a real spec and ask it to list pairs of clauses that cannot both be true, citing clause numbers.
2. Ask for "cannot tell" explicitly. Tell the model it may say the rules do not decide the case. That reduces forced guesses.
3. Include decoys when you test. Add a few clauses that look related but do not conflict, and see whether the tool flags them.
4. Keep a human on backward reasoning. If a question depends on "this was not true, so what follows?", double-check the answer.
5. Check the license before using any open model at work.

Could you build your own contradiction-review tool?

The model is only half the job. Teams also need somewhere to paste a spec, see flagged clause pairs, assign each to an owner, and mark it resolved. With Remy, you can describe that tracker in plain English and have it built and deployed as a full-stack app, instead of managing flags in a spreadsheet. Remy apps can call nearly 400 AI models, and you can also register a custom or self-hosted model or reach a local one over a secure tunnel. That would let a small logic model sit behind the review form if its license suits your use.