How Do You Serve Sage-A1.5 With SGLang or vLLM at 512K Context?

How do you serve Sage-A1.5 locally with SGLang or vLLM?

Install SGLang or vLLM in a Python 3.12 environment built with uv, then launch the server pointing at InternScience/Sage-A1.5. According to the card, the launch commands create API endpoints at http://localhost:8000/v1. For SGLang, set a 262144-token context, the qwen3 reasoning parser and the qwen3_coder tool-call parser. To reach 512K, enable YaRN with factor: 2.0 through a RoPE override. Optional EAGLE flags add MTP speculative decoding.

What is Sage-A1.5, and why does the serving setup matter?

Sage-A1.5 is built for long-horizon agentic work: orchestrating subagents with isolated contexts, verifying its own progress, and doing exploratory scientific research. Those workloads produce long trajectories full of tool calls and reasoning text. That is why the launch commands care about two things: parsing reasoning and tool calls correctly, and having enough context.

The card describes the repository as weights and config files in the Hugging Face Transformers format, compatible with Transformers, vLLM and SGLang. The parser names (qwen3, qwen3_coder) tell you the model follows the Qwen3-style conventions for thinking blocks and tool-call formatting.

How do you install SGLang for Sage-A1.5?

The card uses uv with Python 3.12:

uv venv --python 3.12 --seed --managed-python
source .venv/bin/activate

uv pip install sglang

SGLang's own install documentation covers CUDA and platform details. Check it if your driver or GPU setup is unusual.

How do you run the standard SGLang server?

The standard launch from the card:

python -m sglang.launch_server \
  --model-path InternScience/Sage-A1.5 \
  --port 8000 \
  --mem-fraction-static 0.8 \
  --context-length 262144 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

What each flag does:

Skipping the parsers is the most common way to get a working server that behaves badly in an agent loop. Tool calls arrive as raw text and your client never sees a function call.

How do you enable MTP speculative decoding?

The card's multi-token prediction variant adds EAGLE speculative decoding to the standard command:

python -m sglang.launch_server \
  --model-path InternScience/Sage-A1.5 \
  --port 8000 \
  --mem-fraction-static 0.8 \
  --context-length 262144 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4

Speculative decoding drafts several tokens ahead and verifies them in one pass of the main model. When the drafts are accepted, you get more tokens per forward pass without changing the output distribution. The card sets 3 steps, top-k 1 and 4 draft tokens. The card gives no speed figures, so measure throughput on your own hardware and prompts. Gains depend on batch size and on how predictable your outputs are.

How do you extend Sage-A1.5 to 512K context?

Sage-A1.5 supports 256K natively. To double that, the card serves the model with YaRN (factor: 2.0) by overriding the RoPE parameters at launch:

SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 \
python -m sglang.launch_server \
  --model-path InternScience/Sage-A1.5 \
  --port 8000 \
  --context-length 524288 \
  --json-model-override-args '{
      "text_config": {
          "rope_parameters": {
              "mrope_interleaved": true,
              "mrope_section": [11, 11, 10],
              "rope_type": "yarn",
              "rope_theta": 10000000,
              "partial_rotary_factor": 0.25,
              "factor": 2.0,
              "original_max_position_embeddings": 262144
          }
      }
  }' \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --mem-fraction-static 0.88

Three things make this work:

1. The environment variable. SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 lets SGLang accept a context length longer than the one in the model config.
2. The YaRN override. rope_type is yarn, factor is 2.0, and original_max_position_embeddings stays at 262144. 262144 times 2 gives the 524288 you pass to --context-length.
3. The kept RoPE settings. The multimodal RoPE fields (mrope_interleaved, mrope_section) and the rope_theta and partial_rotary_factor values are restated inside the override. Copy them exactly. Leaving them out risks replacing the model's defaults with incomplete settings.

The card also raises --mem-fraction-static from 0.8 to 0.88. A 512K window needs a larger KV cache, so more GPU memory goes to it. The card does not publish VRAM requirements, so treat 0.88 as a starting point and watch for out-of-memory errors.

Should you always run with the 512K configuration?

Probably not. A general caveat about YaRN as implemented in most serving engines is that the scaling applies to every request, whether short or long. That can slightly change behavior on short inputs, and the larger context reservation costs memory either way. The card's own benchmark note says its SWE-bench evaluations used the mini-SWE-agent harness with a 200K-token context window and automatic context compaction enabled. Use the standard config by default and switch to the 512K config only for jobs that really need it.

What about vLLM?

The card includes a vLLM section. It says to install vLLM from the main branch with uv:

uv venv --python 3.12 --seed --managed-python
source .venv/bin/activate

uv pip install vllm --torch-backend=auto

It also says the commands create the same http://localhost:8000/v1 endpoint. The card's vLLM launch command is vllm serve InternScience/Sage-A1.5 --port 8000 --max-model-len 262144. In general, vLLM's vllm serve takes equivalents of the SGLang options: a max model length, a reasoning parser, a tool-call parser with auto tool choice enabled, tensor parallelism for multi-GPU setups, and a speculative decoding config for MTP. Take the exact names and values from the model card rather than translating from SGLang. Parser names and speculative settings differ between engines and versions.

How do you check that the server works?

Once the server is up, any OpenAI-compatible client can point at http://localhost:8000/v1 with the model name you served. Send a short chat request first. Then send one with a tool definition and confirm the response contains a structured tool call rather than text. If you enabled the reasoning parser, check that thinking content arrives separately from the final answer.