How to Fine-Tune LFM3-1.7B With TRL on a Single GPU
Learn how to fine-tune Liquid AI's LFM3-1.7B with TRL using SFT, DPO, GRPO and continued pre-training, and which recipe fits JSON, RAG, tools or new languages.
How do you fine-tune LFM3-1.7B on a single GPU?
Use Hugging Face TRL. Liquid AI's model card lists four notebook recipes for LFM3-1.7B: supervised fine-tuning (SFT) for extracting structured JSON, direct preference optimization (DPO) for answering from retrieved documents or declining, GRPO for calling the right tools in a 108-tool phone app, and continued pre-training (CPT) for learning Turkish from raw text. Pick the recipe that matches your problem, swap in your own data, and run it on one GPU.
TL;DR
- LFM3-1.7B is a 1.7B-parameter text model with a 128,000-token vocabulary, built for local agents, tool calling and structured output.
- Liquid AI says the model is made for quick single-GPU fine-tuning through TRL, with a Colab notebook for each of four methods.
- Use SFT when you can show the model the right output, DPO when you can rank good and bad answers, GRPO when you can score outcomes with a reward, and CPT when the model lacks language or domain knowledge.
- The model card recommends retrieval or search for knowledge-heavy tasks and says the model is not meant for agentic coding or complex multi-step reasoning.
- Fine-tuned models can run locally, since the 4-bit version needs about 1 GB of memory according to the card.
What is LFM3-1.7B?
LFM3-1.7B is a post-trained, text-only model from Liquid AI, released with open weights under the LFM Open License v1.0. It has 24 layers in a "tri-hybrid" layout: 13 gated short convolutions, 5 sliding-window attention layers and 6 global attention layers. The vocabulary holds 128,000 tokens.
It supports 16 languages, including English, Arabic, Chinese, Japanese, Korean, Spanish, Hindi and Russian. Liquid AI post-trained it for agent harnesses such as Hermes Agent, Pi and OpenClaw.
The card is blunt about scope. The model targets local agents, tool calling, structured outputs and domain-specific tuning. It is not recommended for agentic coding or complex multi-step reasoning.
Which fine-tuning method should you use?
The card pairs each method with a concrete task. Treat those tasks as templates for your own.
| Method | Card's use case | Use it when |
|---|---|---|
| SFT | Extract structured JSON from messy documents | You have input and ideal-output pairs |
| DPO | Answer from retrieved documents, or decline | You have preferred and rejected responses |
| GRPO | Call the right tools in a 108-tool phone app | You can compute a reward for a result |
| CPT | Learn a new language (Turkish) from raw text | You have lots of unlabeled text |
The order also works as a rough escalation path. Start with SFT. Move to preference or reward-based methods when imitation stops being enough.
How does SFT work for JSON extraction?
SFT trains the model on examples of the exact behavior you want. In the card's recipe, the input is a messy document and the target is clean, structured JSON.
This is the cheapest way to get reliable formatting out of a small model. Your dataset is a list of chat-style records: a user message with the raw text, and an assistant message with the JSON. TRL's SFTTrainer handles the rest.
Two practical notes from the model card apply here:
- Format training data with the checkpoint's ChatML chat template via
tokenizer.apply_chat_template(). Mismatched formatting is a common reason fine-tunes underperform. - Reasoning is on by default and the template opens an assistant turn with a
<think>tag. Decide up front whether your task needs thinking traces, sinceenable_thinking=Falseis described as sometimes inconsistent.
Pick SFT for classification, extraction, rewriting into a fixed schema, or any task where a labeled example is easy to produce.
How does DPO teach a model to answer from documents or decline?
DPO trains on pairs: a preferred response and a rejected one for the same prompt. The card's example is a RAG setting. Given retrieved documents, the model should answer when the documents support an answer and decline when they do not.
SFT struggles with this because "decline" is a judgment, not a format. With DPO you show both behaviors. A good pair might have a grounded answer as the chosen response and a confident guess as the rejected one, for a question the context cannot answer. The reverse pairs teach it not to refuse when the answer is present.
This fits the card's guidance. Small models should lean on retrieval for facts, so the useful skill is staying inside the retrieved context. Use DPO for refusal behavior, tone, safety boundaries and grounding.
How does GRPO train tool calling across 108 tools?
GRPO is a reinforcement learning method. The model samples several completions per prompt, a reward function scores them, and the model shifts toward the higher-scoring ones. You do not need a gold answer for every step, only a way to check the outcome.
The card's recipe applies it to a phone app with 108 tools. The reward can check whether the model picked the right tool and filled its arguments correctly. That is verifiable, which is where GRPO works best.
Know the output format before you write rewards. LFM3 emits tool calls as XML:
<tool_call>
<function=get_weather>
<parameter=city>
Paris
</parameter>
</function>
</tool_call>
Your reward function will parse that structure. Also note that function.arguments must be a Python mapping, so decode JSON strings with json.loads() before applying the template.
With a catalog this large, tool definitions can crowd the prompt. LFM3 supports deferred tools: mark a tool with defer_loading=True and let a search tool return names as tool_reference parts. Your harness runs the search. Decide whether you train with the full tool list or the deferred setup, because your deployment should match your training.
When should you use continued pre-training?
CPT continues next-token training on raw text, with no instruction pairs. The card's example teaches the model Turkish. Turkish is not among the 16 listed languages, so this fills a real gap.
Use CPT when the model lacks the underlying language or vocabulary, such as a new language, a dialect, or dense domain text like internal technical documentation. SFT cannot reliably add that kind of knowledge, since it mostly shapes behavior.
CPT often needs a follow-up. A model trained on raw text may lose some chat and tool behavior, so plan a short SFT pass afterward to restore instruction following. That sequencing is a common practice, not something the card prescribes.
Does it fit on one GPU, and how do you run the result?
Liquid AI describes the model as suited to quick single-GPU fine-tuning. The sources do not give VRAM figures or training times, so check each notebook's setup for the hardware it assumes before you plan a run.
For deployment, the card lists several checkpoint formats:
- BF16 for vLLM, SGLang and Transformers
- FP8 and NVFP4 for SGLang
- GGUF for llama.cpp, Ollama and LM Studio
- MLX 4-bit for Apple Silicon
- ONNX for ONNX Runtime
Recommended sampling for chat, tool use and agents is temperature=0.1, top_k=50, repetition_penalty=1.1. Use those when you evaluate your tuned model, or you may misjudge its quality.
Is fine-tuning LFM3-1.7B worth it?
It is worth it when the task is narrow and checkable: extraction, grounded answering, tool selection, or language coverage. A 1.7B model that runs in about 1 GB at 4-bit fits on any phone or laptop, according to Liquid AI's model card, and tuning narrows the gap to bigger models on that one job.
It is a poor fit for open-ended reasoning or coding agents, where the card itself advises against it. For knowledge-heavy questions, add retrieval before you add training.
Frequently Asked Questions
Do I need all four methods?
No. Most projects need one or two. SFT alone covers many formatting and extraction tasks. Add DPO or GRPO only when you can build preference pairs or a reward function.
Which trainer library does the model card use?
TRL, Hugging Face's training library. Each recipe is a separate Colab notebook linked from the card, and Liquid AI also maintains a TRL fine-tuning page in its docs.
Can I fine-tune on a language the model does not support?
The CPT recipe does this with Turkish using raw text. Expect to follow it with instruction tuning so the model keeps its chat and tool-calling behavior.
Should I train with thinking enabled?
It depends on the task. Reasoning is on by default, and the card warns that turning it off can be inconsistent. Keep your training data's format consistent with how you will prompt the model in production.
How we source, write, and correct our articles. Editorial standards
