---
title: "How Do You Serve Sage-A1.5 With SGLang or vLLM at 512K Context?"
url: "https://goremy.ai/blog/articles/how-to-serve-sage-a1-5-sglang-vllm"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-10-23T06:20:08.514Z"
updated: "2026-10-23T06:20:08.514Z"
topic: Local & Open-Weight Models
description: "Serve Sage-A1.5 with SGLang or vLLM: standard and MTP launch commands, parsers, and the YaRN factor 2.0 setup that extends 256K context to 512K."
---

# How Do You Serve Sage-A1.5 With SGLang or vLLM at 512K Context?

> Serve Sage-A1.5 with SGLang or vLLM: standard and MTP launch commands, parsers, and the YaRN factor 2.0 setup that extends 256K context to 512K.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-10-23

## How do you serve Sage-A1.5 locally with SGLang or vLLM?

Install SGLang or vLLM in a Python 3.12 environment built with uv, then launch the server pointing at `InternScience/Sage-A1.5`. According to the card, the launch commands create API endpoints at `http://localhost:8000/v1`. For SGLang, set a 262144-token context, the `qwen3` reasoning parser and the `qwen3_coder` tool-call parser. To reach 512K, enable YaRN with `factor: 2.0` through a RoPE override. Optional EAGLE flags add MTP speculative decoding.

## TL;DR

- Sage-A1.5 is an **agentic model** built for long-horizon work, and its Hugging Face Transformers-format weights work with Transformers, vLLM and SGLang.
- The model **natively supports a 256K context window**, so the standard launch uses `--context-length 262144`.
- Use `--reasoning-parser qwen3` and `--tool-call-parser qwen3_coder` so reasoning traces and function calls come back as structured API output.
- **MTP speculative decoding** in SGLang uses the EAGLE algorithm with 3 steps, top-k 1 and 4 draft tokens.
- **512K context needs YaRN with factor 2.0**, an `original_max_position_embeddings` of 262144, an environment variable override and a higher memory fraction (0.88).
- The card's vLLM section was cut off in the material reviewed here, so check the model card for the exact vLLM flags before copying anything.

## What is Sage-A1.5, and why does the serving setup matter?

Sage-A1.5 is built for long-horizon agentic work: orchestrating subagents with isolated contexts, verifying its own progress, and doing exploratory scientific research. Those workloads produce long trajectories full of tool calls and reasoning text. That is why the launch commands care about two things: parsing reasoning and tool calls correctly, and having enough context.

The card describes the repository as weights and config files in the Hugging Face Transformers format, compatible with Transformers, vLLM and SGLang. The parser names (`qwen3`, `qwen3_coder`) tell you the model follows the Qwen3-style conventions for thinking blocks and tool-call formatting.

## How do you install SGLang for Sage-A1.5?

The card uses uv with Python 3.12:

```shell
uv venv --python 3.12 --seed --managed-python
source .venv/bin/activate

uv pip install sglang
```

SGLang's own install documentation covers CUDA and platform details. Check it if your driver or GPU setup is unusual.

## How do you run the standard SGLang server?

The standard launch from the card:

```shell
python -m sglang.launch_server \
  --model-path InternScience/Sage-A1.5 \
  --port 8000 \
  --mem-fraction-static 0.8 \
  --context-length 262144 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder
```

What each flag does:

| Flag | Value | Purpose |
|---|---|---|
| `--model-path` | `InternScience/Sage-A1.5` | Pulls the weights from Hugging Face |
| `--port` | 8000 | API at `http://localhost:8000/v1` |
| `--mem-fraction-static` | 0.8 | Share of GPU memory reserved for weights and KV cache |
| `--context-length` | 262144 | The native 256K window |
| `--reasoning-parser` | `qwen3` | Separates thinking content from the final answer |
| `--tool-call-parser` | `qwen3_coder` | Converts tool-call text into structured calls |

Skipping the parsers is the most common way to get a working server that behaves badly in an agent loop. Tool calls arrive as raw text and your client never sees a function call.

## How do you enable MTP speculative decoding?

The card's multi-token prediction variant adds EAGLE speculative decoding to the standard command:

```shell
python -m sglang.launch_server \
  --model-path InternScience/Sage-A1.5 \
  --port 8000 \
  --mem-fraction-static 0.8 \
  --context-length 262144 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4
```

Speculative decoding drafts several tokens ahead and verifies them in one pass of the main model. When the drafts are accepted, you get more tokens per forward pass without changing the output distribution. The card sets 3 steps, top-k 1 and 4 draft tokens. The card gives no speed figures, so measure throughput on your own hardware and prompts. Gains depend on batch size and on how predictable your outputs are.

## How do you extend Sage-A1.5 to 512K context?

Sage-A1.5 supports 256K natively. To double that, the card serves the model with YaRN (`factor: 2.0`) by overriding the RoPE parameters at launch:

```shell
SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 \
python -m sglang.launch_server \
  --model-path InternScience/Sage-A1.5 \
  --port 8000 \
  --context-length 524288 \
  --json-model-override-args '{
      "text_config": {
          "rope_parameters": {
              "mrope_interleaved": true,
              "mrope_section": [11, 11, 10],
              "rope_type": "yarn",
              "rope_theta": 10000000,
              "partial_rotary_factor": 0.25,
              "factor": 2.0,
              "original_max_position_embeddings": 262144
          }
      }
  }' \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --mem-fraction-static 0.88
```

Three things make this work:

1. **The environment variable.** `SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1` lets SGLang accept a context length longer than the one in the model config.
2. **The YaRN override.** `rope_type` is `yarn`, `factor` is 2.0, and `original_max_position_embeddings` stays at 262144. 262144 times 2 gives the 524288 you pass to `--context-length`.
3. **The kept RoPE settings.** The multimodal RoPE fields (`mrope_interleaved`, `mrope_section`) and the `rope_theta` and `partial_rotary_factor` values are restated inside the override. Copy them exactly. Leaving them out risks replacing the model's defaults with incomplete settings.

The card also raises `--mem-fraction-static` from 0.8 to 0.88. A 512K window needs a larger KV cache, so more GPU memory goes to it. The card does not publish VRAM requirements, so treat 0.88 as a starting point and watch for out-of-memory errors.

## Should you always run with the 512K configuration?

Probably not. A general caveat about YaRN as implemented in most serving engines is that the scaling applies to every request, whether short or long. That can slightly change behavior on short inputs, and the larger context reservation costs memory either way. The card's own benchmark note says its SWE-bench evaluations used the mini-SWE-agent harness with a 200K-token context window and automatic context compaction enabled. Use the standard config by default and switch to the 512K config only for jobs that really need it.

## What about vLLM?

The card includes a vLLM section. It says to install vLLM from the main branch with uv:

```shell
uv venv --python 3.12 --seed --managed-python
source .venv/bin/activate

uv pip install vllm --torch-backend=auto
```

It also says the commands create the same `http://localhost:8000/v1` endpoint. The card's vLLM launch command is `vllm serve InternScience/Sage-A1.5 --port 8000 --max-model-len 262144`. In general, vLLM's `vllm serve` takes equivalents of the SGLang options: a max model length, a reasoning parser, a tool-call parser with auto tool choice enabled, tensor parallelism for multi-GPU setups, and a speculative decoding config for MTP. Take the exact names and values from the model card rather than translating from SGLang. Parser names and speculative settings differ between engines and versions.

## How do you check that the server works?

Once the server is up, any OpenAI-compatible client can point at `http://localhost:8000/v1` with the model name you served. Send a short chat request first. Then send one with a tool definition and confirm the response contains a structured tool call rather than text. If you enabled the reasoning parser, check that thinking content arrives separately from the final answer.

## Frequently Asked Questions

### What context length does Sage-A1.5 support?
It supports 256K tokens natively, which is the `--context-length 262144` value in the standard commands. With YaRN at factor 2.0 and the RoPE override shown above, the card extends this to 512K (524288 tokens).

### Which parsers should I use with Sage-A1.5?
The card uses `--reasoning-parser qwen3` and `--tool-call-parser qwen3_coder` in its SGLang commands. They separate reasoning output from answers and turn tool calls into structured API responses, which agent frameworks depend on.

### What does the MTP option change?
It adds EAGLE speculative decoding to SGLang through four extra flags: algorithm EAGLE, 3 steps, top-k 1 and 4 draft tokens. It targets faster generation. The card does not quote speedups, so benchmark it on your own workload.

### How much GPU memory do I need?
The sources do not give VRAM requirements. The card only shows `--mem-fraction-static` values of 0.8 for standard serving and 0.88 for the 512K setup. Plan hardware by testing, and expect the long-context setup to need more memory for KV cache.

### Can I use Hugging Face Transformers instead?
The card says the weights are in Transformers format and compatible with it, along with vLLM and SGLang. For an API server with parsers and speculative decoding, the card's commands target SGLang and vLLM.