---
title: "LFM3-1.7B-Base: Specs, Memory Needs and Fine-Tuning Guide"
url: "https://goremy.ai/blog/articles/lfm3-1-7b-base-specs-local-fine-tuning"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-11-02T22:00:08.416Z"
updated: "2026-11-02T22:00:08.416Z"
topic: Open Model Run Guides
description: "Liquid AI's LFM3-1.7B-Base is a 1.73B model with 256K context that takes about 1 GB at 4-bit. Specs, inference code, thinking toggle and fine-tuning with TRL."
---

# LFM3-1.7B-Base: Specs, Memory Needs and Fine-Tuning Guide

> Liquid AI's LFM3-1.7B-Base is a 1.73B model with 256K context that takes about 1 GB at 4-bit. Specs, inference code, thinking toggle and fine-tuning with TRL.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-11-02

## What is LFM3-1.7B-Base?

LFM3-1.7B-Base is the pre-trained, text-only checkpoint behind Liquid AI's LFM3-1.7B. It has 1.73B parameters, a 24-layer hybrid of convolution and attention blocks, and a 256K-token context window. Liquid AI says it needs about 1 GB of memory at 4-bit precision. It is meant as a starting point for continued pre-training and custom post-training, not for chat out of the box.

## TL;DR

- LFM3-1.7B-Base has **1.73B parameters** and was pre-trained on 22T tokens, with the last 3.5T being curated mid-training data.
- The **tri-hybrid design** uses 24 layers: 13 gated short convolutions, 5 sliding-window attention layers and 6 global attention layers.
- The context window is **256K tokens** (input plus output), and Liquid AI says the model takes about 1 GB at 4-bit.
- For chat, tool calling and agents, Liquid AI points you to **LFM3-1.7B instead**. The base model is for heavy customization.
- Fine-tuning runs through **TRL**, with Liquid AI-provided notebooks for SFT (structured JSON extraction) and continued pre-training (learning Turkish).
- The license is the **LFM Open License v1.0**, so read it before shipping anything commercial.

## What are the core specs?

The model card lists these properties.

| Property | Specification |
|---|---|
| Checkpoint | Pre-trained, text-only base model |
| Parameters | 1.73B |
| Layers | 24 (13 gated short convolution, 5 sliding-window attention, 6 global attention) |
| Context length | 256K tokens (input + output) |
| Vocabulary | 128,000 tokens |
| Weights | BF16 |
| License | LFM Open License v1.0 |

Supported languages are English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian and Polish.

## What does the tri-hybrid architecture mean?

Most of the network is not attention. Thirteen of the 24 layers are gated short convolutions, which are cheap and local. Eleven are attention layers, split between five sliding-window layers (attention over a limited span) and six global layers (attention over the whole sequence).

The practical appeal is clear even without benchmarks. Global attention is the expensive part of long-context inference, and this design keeps only six such layers. That is the likely reason a 256K window is plausible on small hardware, though the model card does not publish memory figures at long context. Test your own sequence lengths before assuming the 1 GB number holds.

## How was it trained?

Liquid AI describes two stages totaling 22T tokens.

1. **Main pre-training.** Compared with LFM2, the mix adds more scientific papers, repository-level code, math and broader knowledge sources. The recipe graphic describes about 15T tokens of web and synthetic data, then 3T tokens of code, math and science at 4K context.
2. **Mid-training.** The last 3.5T tokens are higher-quality curated data with more agentic traces and tool calls. This stage also stretches context: 3T tokens at 32K, 500B at 128K and 50B at 256K.

Post-training from this checkpoint (SFT, mixed RL, multi-domain on-policy distillation and agentic RL) produces LFM3-1.7B.

## What hardware does LFM3-1.7B-Base need?

The card says the model takes about 1 GB at 4-bit, so it fits on any phone or laptop. That figure refers to quantized inference. The released weights are BF16.

A rough sanity check: 1.73B parameters at 2 bytes each is around 3.5 GB for the weights alone in BF16. Add activations and cache on top. At 4-bit the weights shrink to roughly a quarter of that, which lines up with the card's claim. Those are back-of-envelope numbers, not published requirements.

Fine-tuning needs more than inference. Full fine-tuning holds gradients and optimizer state in addition to weights, so plan on a GPU with comfortable headroom. The model card gives no training memory numbers, so measure on your setup. The Colab notebooks are a low-friction way to find out what fits.

## How do you run it with transformers?

The base model uses the same architecture and modeling code as LFM3-1.7B. You need `trust_remote_code=True`.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "LiquidAI/LFM3-1.7B-Base"
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype="bfloat16",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

prompt = "The nematode C. elegans is"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(model.device)
output = model.generate(input_ids, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

Note the prompt style. This is plain text continuation, which is what a base model does well. Greedy decoding (`do_sample=False`) is used in the example.

## How does the thinking toggle work?

Mid-training included chat data with and without thinking traces, formatted with the same chat template as LFM3-1.7B. So the base model has seen the format, even though it is not tuned as an assistant.

When you use `apply_chat_template`, reasoning is on by default and responses begin with a thinking trace. Pass `enable_thinking=False` and the template prefills an empty thinking trace, so the model answers directly. If you plan to fine-tune your own assistant, decide early whether your training data should include thinking traces, and keep the template consistent between training and inference.

## How do you fine-tune LFM3-1.7B-Base?

Liquid AI recommends the base model for substantial customization: new domains, new languages, or behaviors that diverge sharply from LFM3-1.7B. The route is continued pre-training or full post-training through TRL.

Two notebooks are provided:

| Training | Use case |
|---|---|
| SFT | Extract structured JSON from messy documents |
| CPT | Learn a new language (Turkish) from raw text |

Both notebooks start from LFM3-1.7B. To start from the base model, change one line:

```python
model_id = "LiquidAI/LFM3-1.7B-Base"
```

A reasonable workflow is to run the SFT notebook as-is first to confirm your environment works, then swap in the base model and your own data. Use CPT when the gap is knowledge or language (raw text in a domain or tongue the model handles poorly). Use SFT when the gap is behavior or output format.

## Should you use the base model or LFM3-1.7B?

Use LFM3-1.7B if you want chat, tool calling or agents now. Liquid AI says the base model is only recommended for tasks needing heavy fine-tuning, such as language-specific or domain-specific assistants, training on proprietary data, or experiments with new post-training methods.

The tradeoff is control against effort. The base model gives you a clean slate and avoids inheriting the instruct model's post-training habits. It also means you own the whole post-training pipeline. If a light fine-tune of the instruct model reaches your goal, that is the cheaper path.

## What about the license?

The model is released under the LFM Open License v1.0. The model card links the license file but the terms are not reproduced in the sources here, so check them directly for commercial use, redistribution and any revenue thresholds before building a product.

## Frequently Asked Questions

### How big is LFM3-1.7B-Base?
It has 1.73B parameters across 24 layers and a 128,000-token vocabulary. Weights ship in BF16. Liquid AI says it uses about 1 GB of memory at 4-bit precision.

### Can I use LFM3-1.7B-Base as a chatbot?
It is not recommended. The base model is a pre-trained checkpoint for heavy fine-tuning. For chat, tool calling and agents, Liquid AI directs users to LFM3-1.7B, which is post-trained from this checkpoint.

### What context length does it support?
256K tokens, counting input and output together. Mid-training extended the window in stages, to 32K, then 128K, then 256K.

### What tools does Liquid AI suggest for fine-tuning?
TRL. Liquid AI provides Colab notebooks for SFT and continued pre-training. They start from LFM3-1.7B, and you switch to the base model by changing `model_id`.

### Does it support languages other than English?
Yes. The card lists 16 languages, including Arabic, Chinese, Japanese, Korean, Hindi and Russian. The CPT notebook shows adding another (Turkish) from raw text.