--- title: How to Run LFM3-1.7B Locally and What It Can Actually Do url: "https://goremy.ai/blog/articles/liquid-ai-lfm3-1-7b-run-locally-tested" author: Luis Chavez-Mattos editor: Luis Chavez-Mattos editor_title: Director, Product published: "2026-10-29T10:20:04.739Z" updated: "2026-10-29T10:20:07.080Z" topic: Open Model Run Guides description: "Run Liquid AI's LFM3-1.7B locally with llama.cpp, Ollama, LM Studio or MLX. Covers formats, sampling settings, and which tests passed or failed in hands-on use." --- # How to Run LFM3-1.7B Locally and What It Can Actually Do > Run Liquid AI's LFM3-1.7B locally with llama.cpp, Ollama, LM Studio or MLX. Covers formats, sampling settings, and which tests passed or failed in hands-on use. Edited by Luis Chavez-Mattos, Director, Product ยท Published 2026-10-29 ## How do you run LFM3-1.7B locally? Download the checkpoint that matches your runtime and hardware. Liquid AI publishes GGUF for llama.cpp, Ollama and LM Studio, a 4-bit MLX build for Apple Silicon, ONNX for ONNX Runtime, and BF16, FP8 and NVFP4 weights for GPU servers like vLLM and SGLang. At 4-bit the model takes about 1 GB of memory. Use temperature 0.1, top_k 50 and repetition penalty 1.1 for general work. ## TL;DR - LFM3-1.7B is a **1.73B-parameter text model** with a 256K-token context window, post-trained for agent harnesses such as Hermes Agent, Pi and OpenClaw. - The model card says it needs **about 1 GB at 4-bit**, while a BF16 run under llama.cpp used roughly 4.5 GB of VRAM in one test. - In hands-on tests it **passed JSON extraction, strict instruction following, grounded answering and two-call tool use**. - It **failed bug fixing in an agent loop, writing working HTML/JS, and multilingual translation** at scale. - Liquid's own card says **not to use it for agentic coding or complex multi-step reasoning**, and the tests agree. - Use **different sampling settings** for chat and agents than for math and STEM. ## What is LFM3-1.7B? It is a post-trained, text-only language model from Liquid AI, part of the LFM3 family that also includes base, vision, audio, omni and decision models. Liquid describes it as a model for local agents and specialized applications. | Property | Specification | |---|---| | Parameters | 1.73B | | Architecture | 24 layers: 13 gated short convolutions, 5 sliding-window attention, 6 global attention | | Context length | 256K tokens (input plus output) | | Vocabulary | 128,000 tokens | | Languages | English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish | | License | LFM Open License v1.0 | Liquid calls the design a tri-hybrid. Most layers are short convolutions, with a smaller number of local and global attention layers mixed in. That is part of why a 256K window is plausible on such a small model. ## Which checkpoint should you pick? | Checkpoint | Runs on | Notes | |---|---|---| | BF16 | vLLM, SGLang, Transformers | Full-precision GPU weights | | FP8 | SGLang | GPU quantization | | NVFP4 | SGLang | 4-bit floating point for GPU | | GGUF | llama.cpp, Ollama, LM Studio | CPU and GPU local inference | | MLX 4-bit | MLX | Apple Silicon | | ONNX | ONNX Runtime | Cross-platform | For a laptop or desktop, GGUF is the default choice. On a Mac, the MLX 4-bit build is the direct route. If you have a GPU server and want throughput, BF16 on vLLM or SGLang is the straightforward option. ## What sampling settings does it need? The card gives two presets. - **General** (chat, tool use, agents, coding): `temperature=0.1, top_k=50, repetition_penalty=1.1` - **Math and STEM**: `temperature=1.0, top_p=0.95, top_k=20` A minimal Transformers run looks like this: ```python from transformers import pipeline generator = pipeline( "text-generation", model="LiquidAI/LFM3-1.7B", trust_remote_code=True, ) messages = [{"role": "user", "content": "What is C. elegans?"}] output = generator( messages, do_sample=True, temperature=0.1, top_k=50, repetition_penalty=1.1, max_new_tokens=2048, ) print(output[0]["generated_text"][-1]["content"]) ``` ## How do the chat template and tool calling work? The model uses ChatML. Reasoning is on by default, so the assistant turn starts with a `` block. You can pass `enable_thinking=False`, but the card warns this can be inconsistent on reasoning-heavy tasks. Thinking from earlier turns is dropped unless you set `preserve_thinking=True`. Tool calls come back as XML, not JSON: ```text Paris ``` Pass schemas through `tools=...` in `apply_chat_template()`. Your app executes the call, adds it to history as a `tool_calls` entry, then appends the result as a `tool` message. `function.arguments` must be a Python mapping, so run `json.loads()` on any JSON string first. The model also supports deferred tools. Set `defer_loading=True` and the definitions stay out of the prompt until your harness's search tool returns their names. For llama.cpp, use a GGUF with the current chat template or load it with `--chat-template-file`. For a comparable llama.cpp walkthrough with another small model, see [running Nemotron-3.5 Bolt E4B locally](https://goremy.ai/blog/articles/run-nemotron-3-5-bolt-e4b-locally). ## What did hands-on testing show? A creator ran the model through llama.cpp in BF16 (about 4.5 GB of VRAM) and connected it to Hermes Agent. The results split cleanly along the lines of Liquid's own guidance. | Task | Result | |---|---| | Find a planted bug in a broken restaurant dashboard (agentic) | Fail. It reported success, but the numbers on the page were unchanged. | | Generate a self-contained HTML doner kebab simulation | Fail. Blank page twice. The code used an undefined `canvas` variable. | | Extract clean JSON from messy text | Pass. Valid JSON, correct values, and it followed the "JSON only" instruction. | | Strict instruction following (four sentences, no commas, under 60 words) | Pass. 43 words and every constraint met. | | Translate a sentence into many languages | Fail. Invented words, English left in, wrong scripts. | | Summarize a nonexistent 1987 paper | Pass, with a caveat. It invented no summary but never said the paper might not exist. | | Yield math (how much shank to buy for 150 plates) | Pass. Answer of 71.74 kg was correct. | | Answer only from a passage, say "not in the text" when needed | Pass on all three questions. | | Tool calling with two calls in one request | Pass, with a slip. Correct tools and arguments, but the reminder used UTC because no time zone was given. | The worst result is the first one. The agent declared "goal achieved" without changing the outcome. A small model that claims success falsely is a risk in any loop that trusts its own report, so verify results with a check outside the model. ## Where does it fall short? Three areas. First, coding: the card says it is not recommended for agentic coding, and the blank-page failure was a trivial undefined variable. Second, multilingual work at breadth: the supported list is 16 languages, and the translation test asked for far more than that, so a failure there is unsurprising. Third, complex multi-step reasoning, which the card also excludes. Short, well-posed arithmetic did fine. The card also advises retrieval or search for knowledge-heavy tasks. The grounded-answer pass fits that pattern. Give it the text and it stays inside it. ## Is LFM3-1.7B worth running? For narrow jobs, yes. The tasks it passed (structured extraction, constrained writing, tool selection, answering from supplied context) are the ones local pipelines actually need, and it handles them in a footprint that fits a phone or laptop at 4-bit. For open-ended coding agents, no. Pick a larger model there. Liquid also positions it as a fine-tuning base. The card links TRL notebooks for SFT (JSON extraction from messy documents), DPO (answer from retrieved documents or decline), and GRPO (tool calling across a 108-tool phone app), plus continued pre-training for a new language such as Turkish. If its multilingual or domain gaps matter to you, that is the intended path. ## Frequently Asked Questions ### How much memory does LFM3-1.7B need? Liquid says about 1 GB at 4-bit precision. A BF16 run under llama.cpp used roughly 4.5 GB of VRAM in the hands-on test. Context length adds to either figure. ### Can LFM3-1.7B call tools? Yes. It emits XML-formatted calls and passed a test with two correct calls in one request. Check argument details such as time zones yourself, since the model guessed UTC when none was given. ### Is it good for coding? No. The model card advises against agentic coding, and testing produced a blank HTML page and an agent that claimed to fix a bug without changing anything. ### Does it support many languages? It lists 16 supported languages. A test asking for translation into many more produced invented words and wrong scripts, so do not rely on it beyond the listed set. ### What license does it use? The LFM Open License v1.0. Read it before shipping a product, since it is Liquid's own license and not a standard one.