---
title: Open Model Run Guides
url: "https://goremy.ai/blog/topic/open-model-run-guides"
type: topic
updated: "2026-10-23T11:20:07.415Z"
posts: 15
---

# Open Model Run Guides

Run guides for specific named open-weight models: how to run each one locally, with the right runtime (Ollama, llama.cpp, vLLM, SGLang, MLX), quantization, VRAM and context needs. General setup, tooling and hardware coverage stays in Local & Open-Weight Models.

## Posts (15)

- [How Do You Run Recursive Flint Locally? Hardware and Setup](https://goremy.ai/blog/articles/recursive-flint-local-coding-model-setup) (2026-10-23): Run Recursive Flint, a 40B MoE coding model with 2.5B active parameters, locally with llama.cpp or Rec. Hardware, GGUF size, settings and OpenCode fix.
- [How to Run Flint 40B Locally: Ollama, GGUF, SGLang and VRAM Needs](https://goremy.ai/blog/articles/how-to-run-flint-40b-locally-ollama) (2026-10-23): Install Flint 40B locally with Ollama, llama.cpp or SGLang. Covers VRAM needs (about 32GB for the quant), GGUF Q4, NVFP4, MLX and BF16 on H200 or B300.
- [How Do You Serve Sage-A1.5 With SGLang or vLLM at 512K Context?](https://goremy.ai/blog/articles/how-to-serve-sage-a1-5-sglang-vllm) (2026-10-23): Serve Sage-A1.5 with SGLang or vLLM: standard and MTP launch commands, parsers, and the YaRN factor 2.0 setup that extends 256K context to 512K.
- [How to Run Raon-1 mini Locally: NVFP4, RTX 5090 and Context Needs](https://goremy.ai/blog/articles/raon-1-mini-run-locally-rtx-5090) (2026-10-23): Raon-1 mini is a 28.8B MoE with 4.3B active parameters. See what NVFP4, MXFP8, DFlash2 and an 8 KiB per token KV cache mean for local hardware.
- [What is Raon-1 mini and can it run on one RTX 5090?](https://goremy.ai/blog/articles/raon-1-mini-preview-krafton-open-model) (2026-10-23): Raon-1 mini preview is KRAFTON AI's Apache 2.0 30B-A4B MoE with hybrid attention and 262K context. Specs, NVFP4 on a 32GB RTX 5090, and benchmarks.
- [Can You Run Fish Audio Drama 3 Locally? VRAM and Setup Guide](https://goremy.ai/blog/articles/fish-audio-drama-3-run-locally-vram) (2026-10-23): Fish Audio Drama 3 is an open-weight MoE text-to-speech model. Here are the VRAM needs, the 4-bit quant, bracket directions, and how it ranks.
- [How to Run Recursive Flint Locally: SGLang, GPUs and Setup](https://goremy.ai/blog/articles/recursive-flint-run-locally-sglang-hardware) (2026-10-22): Self-host Recursive Flint, a 39.8B MoE coding model, with SGLang on one H200 or B300. Covers the 80 GB download, Docker steps, sampling settings and licence.
- [How to Run Ornith-1.5-122B Locally with llama.cpp and vLLM](https://goremy.ai/blog/articles/run-ornith-1-5-122b-locally-llama-cpp) (2026-10-22): Run Ornith-1.5-122B locally: llama.cpp GGUF setup, VRAM and context trade-offs, runtime versions to check, and how to handle thinking-mode output.
- [How to Run Hy-Memory-30B-A3B Locally with vLLM for Mem0 and MemOS](https://goremy.ai/blog/articles/run-hy-memory-30b-a3b-vllm-mem0) (2026-10-22): Serve Tencent's Hy-Memory-30B-A3B with vLLM as hy-memory, call it via the OpenAI-compatible API, and use it as the write path in Mem0 or MemOS.
- [How to Run Spark-X2.6-4B Locally: Ollama, llama.cpp, MLX, vLLM](https://goremy.ai/blog/articles/spark-x2-6-4b-on-device-agent-model) (2026-10-22): Run Spark-X2.6-4B, an Apache 2.0 4B on-device agent model with 1M-token context, via Ollama, llama.cpp, LM Studio, MLX, vLLM or SGLang.
- [How to Run Nemotron Dory 1.0 4B Locally and Pick a Loop Count](https://goremy.ai/blog/articles/run-nemotron-dory-4b-locally-loop-count) (2026-10-22): Run NVIDIA Dory1.0-4B locally with vLLM or llama.cpp on Apple Silicon, choose 1, 2 or 4 loops, and cut KV cache memory with last-loop caching.
- [How to Run DeepSeek-V4.1-Flash-1017 Without a Jinja Chat Template](https://goremy.ai/blog/articles/run-deepseek-v4-1-flash-1017-locally) (2026-10-22): DeepSeek-V4.1-Flash-1017 ships MIT-licensed with no Jinja template. See sampling settings, 1M context, the encoding folder and deepseek-recipe.
- [How to Run Distilled AI Models Locally (DeepSeek-R1, Ollama)](https://goremy.ai/blog/articles/how-to-run-distilled-ai-models-locally-deepseek-r1-ollama) (2026-10-21): Learn how to pick, quantize, and run distilled models like DeepSeek-R1 locally with Ollama or llama.cpp, and when self-hosting beats API costs.
- [How to Run DeepSeek R1 671B Locally by Streaming Experts Off SSD](https://goremy.ai/blog/articles/how-to-run-deepseek-r1-671b-locally-by-streaming-experts-off-ssd) (2026-10-21): Learn how to run 671B MoE models like DeepSeek R1 locally with llama.cpp, KTransformers and Unsloth quants, with hardware tiers and tokens per second.
- [How to Run GLM-5.3-Flash Locally on CPU (No GPU Required)](https://goremy.ai/blog/articles/run-glm-5-3-flash-locally-on-cpu-no-gpu-needed) (2026-08-31): How to run the 320B-parameter GLM-5.3-Flash model on CPU with 128GB RAM using Unsloth's dynamic GGUF quants, plus the hardware math and exact commands.