---
title: "How to Run Apex-35B Locally: Hardware, Setup and Results"
url: "https://goremy.ai/blog/articles/apex-35b-run-locally-qwen-3-6-moe"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-11-19T19:20:11.789Z"
updated: "2026-11-19T19:20:11.789Z"
topic: Local AI
description: Apex-35B is a 35B sparse MoE built on Qwen 3.6 with 3B active parameters. Here is how to run it with llama.cpp, what VRAM it needs, and how it tested.
---

# How to Run Apex-35B Locally: Hardware, Setup and Results

> Apex-35B is a 35B sparse MoE built on Qwen 3.6 with 3B active parameters. Here is how to run it with llama.cpp, what VRAM it needs, and how it tested.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-11-19

## How do you run Apex-35B locally?

You download a GGUF build of Apex-35B and serve it with llama.cpp, an open-source program that runs language models on your own hardware. The 8-bit version (Q8_0) is 37.8 GB. In one public demo it used just over 38 GB of VRAM (the memory on a graphics card) on an 80 GB H100. Smaller quantized versions, down to 9.0 GB, exist for lower-end cards. The model is released under Apache 2.0, so you can use it commercially.

## TL;DR

- **Apex-35B is a sparse mixture-of-experts model** with 35 billion total parameters but only about 3 billion active for each token, built on Qwen3.6-35B-A3B by a new lab called Apex Intelligence.
- **The Q8_0 file is 37.8 GB**, and a demo run used just over 38 GB of VRAM on an H100. The smallest quantization (IQ1_S) is 9.0 GB.
- **You can serve it with llama.cpp**, then connect an agent such as Hermes so it can read code, run commands and write files.
- **In one creator's tests it fixed a hidden billing bug in about 5.5 minutes**, cleaned a messy spreadsheet well, and produced a middling visual coding result.
- **On the published benchmarks it beats the Qwen model it was built on** in every research and agentic row shown, but larger models still lead on most tests.
- **Treat the results as a starting point.** They come from a model card and one reviewer's hands-on tests, not from your own workload.

## What is Apex-35B?

Apex-35B-V1.0 is the first model from Apex Intelligence. It is a general-purpose model aimed at coding, tool use, question answering, deep search and research tasks, including computational science and robotics simulation. It is trained on top of Qwen3.6-35B-A3B, and the model card says it works with the Claude Code harness.

Three details matter for a practical reader:

- **Sparse architecture.** It has 35B parameters in total, but roughly 3B are activated per token.
- **Long context.** The window is 262,144 tokens, shared between your input and the model's output. That is enough for a large code repository or a long multi-step tool session.
- **Dynamic thinking.** The model decides on each turn whether to reason and for how long. You do not flip a switch between "fast" and "thinking" modes.

## How was Apex-35B made?

The unusual part is the process. Apex Intelligence built an automated research system called Apex Research. According to the model card, it contributed to training-data construction, automated experiments, training-recipe exploration, GPU kernel optimization and inference optimization. It generated training data for the model's weak areas and refined the recipe based on experimental feedback. The company calls this recursive self-improvement.

In plain terms: AI helped do parts of the work of building the AI, and humans validated the results before they fed into later development. That does not make the model magic. It is a claim about how the training pipeline was run, and the real test is how the model behaves on your tasks.

## What hardware does Apex-35B need?

A mixture-of-experts model is fast because only about 3B parameters work on each token. It is not small in memory. All 35B parameters still have to be loaded, so your VRAM requirement follows the file size, plus room for the context cache.

The Apex GGUF repository offers BF16 plus 28 quantizations. A quantization stores the model's numbers at lower precision, which shrinks the file at some cost in quality.

| Version | File size | Notes |
|---|---|---|
| Q8_0 | 37.8 GB | Used in the demo; just over 38 GB of VRAM on an 80 GB H100 |
| IQ1_S | 9.0 GB | Smallest option listed; expect a quality drop |
| BF16 | Larger than Q8_0 | Full precision; size not given in the material I reviewed |
| 26 others | Between the two | Check the repository for exact sizes |

Practical reading of that table:

- **A single 48 GB or 80 GB workstation or cloud GPU** is a comfortable home for Q8_0.
- **A 24 GB consumer card** means choosing a lower quantization from the middle of the range. Check the file size against your VRAM and leave headroom for context.
- **Longer context uses more memory.** If you run out, reduce the context length before you reduce the quantization.

If you do not own a big GPU, you can rent one by the hour from a GPU cloud provider. In the demo, the reviewer served the model with llama.cpp on a local system.

## How do you set it up with llama.cpp?

1. **Install llama.cpp** using its current build instructions for your system (CUDA for Nvidia cards).
2. **Pick a quantization** from the Apex-35B-V1.0-GGUF repository on Hugging Face that fits your VRAM.
3. **Start llama-server** pointing at that file. A typical pattern looks like this, but confirm flags against the model card and your llama.cpp version:

```
llama-server -hf Apexintelligence-AI/Apex-35B-V1.0-GGUF:Q8_0 -c 32768 -ngl 99
```

4. **Open the built-in web interface** (llama-server includes one) to chat with the model, or call its OpenAI-compatible API from other tools.
5. **Raise the context length gradually.** The model supports 262,144 tokens, but you will only reach that if you have the memory for it.

Unsloth and other local runtimes are also supported, according to the model card.

## How do you use Apex-35B with an agent like Hermes?

A chat window can only talk. An agent harness lets the model read files, run commands, edit code and check its own work. In the demo, the model was configured with the Hermes agent, pointed at the local llama.cpp server, and given a task in plain language.

The setup logic is the same for most harnesses: point the agent at your local server's address, choose the model, and give it a working folder. Because the server speaks an OpenAI-style API, many agent tools can use it without special support. The model card also lists Claude Code as a supported harness.

## How did Apex-35B perform in real tests?

The reviewer ran four tasks. These are single runs by one person, so read them as signals, not measurements.

**1. Fixing a hidden bug in a full-stack app (passed).** The app was an operations console for a rooftop solar installer. Two screens showed symptoms of one bug: a customer in Pune was told the system paid for itself in 1.8 years, and a tariff chart showed a staircase instead of a smooth line. The model was not told what the bug was. In about 5.5 minutes it rewrote the function, rebuilt only the API service, added a regression test and gave verification evidence. The Pune payback became 2.7 years, the 25-year savings fell from about $44,000 to about $30,000, and the chart smoothed out.

**2. Building a rain-on-a-window animation (mixed).** The task was a single HTML file, no libraries, with sliding drops, wipeable fog and lightning. It took about 24 minutes and the features worked, but the look missed the brief. Trails were thick like dark ropes, and there were noisy square patches.

**3. Cleaning a messy spreadsheet (strong).** The file was a year of online store orders with duplicates, four date formats, inconsistent country names and bad values. The model produced four files: a cleaned file of around 1,400 orders, a log of every problem, a chart and a findings report. It found that electronics brought in about half of revenue, but roughly one in five electronics orders was returned. A few chart labels overlapped.

**4. A genetics probability problem (passed).** With no code or tools, the model thought for about five minutes and got all three answers right, including 66.7% for the first. The reviewer noted one figure skipped a tiny correction.

## How does Apex-35B compare on benchmarks?

The model card compares Apex-35B-V1.0 against its base model and several others. A selection of the published scores:

| Benchmark | Apex-35B | Qwen3.6-35B-A3B | GLM 5.3 (753B) | DeepSeek V4.1-Flash-1017 |
|---|---|---|---|---|
| SciCode (main / sub) | 62.6 / 75.6 | 37.5 / 60.6 | 79.2 / 91.3 | 68.8 / 81.9 |
| FrontierScience | 40.0 | 21.7 | 30.7 | 61.7 |
| BioMysteryBench (human-difficult) | 35.3 | 11.8 | 39.1 | 29.4 |
| PaperBench | 60.9 | 45.8 | 42.5 | 87.1 |
| BrowseComp | 69.4 | 42.1 | 80.9 | 90.5 |

The pattern: Apex-35B clearly beats the model it was built on, and the larger models still lead on most rows. It edges ahead of GLM 5.3 on FrontierScience and PaperBench, and ahead of DeepSeek on the hardest biology set. It also beats Recursive Flint (39.8B), a model of similar size. These are the lab's own reported results, so independent testing matters.

## What are the limits?

- **Speed of thinking.** Dynamic thinking can run for minutes on hard problems. That is fine for a background task, less so for a quick question.
- **Polish.** Visual and creative output was acceptable, not impressive.
- **Memory.** "Only 3B active" does not mean it fits on a small card. Plan around the file size.
- **Evidence.** The hands-on tests are one person's, and the benchmarks come from the model's maker.

## What does this mean for you?

**If you work with sensitive data**, a local model means your files never leave your machine or your rented server. The spreadsheet-cleaning result is the relevant one: messy exports of orders, customers or expenses are common work, and the output included a log of every change, which is what a reviewer wants.

**If you are an analyst, marketer or finance person**, try the cleaning task first. Take a copy of a messy export, ask for a cleaned file, a log of problems, one chart and three findings, and say explicitly that no number may be invented. Then check ten rows by hand.

**If you manage a technical team**, the Apache 2.0 license and the long context make this worth a trial for internal code-fixing agents. Run it against a real bug in a non-critical repo and look at whether it adds tests.

**If you have no GPU**, rent one for an afternoon before buying anything.

## Could you build a tool for this kind of work yourself?

The spreadsheet test points at a common need: a repeatable place to upload a messy export and get back a clean table, a problem log and a few findings. With [Remy](https://goremy.ai), you can describe that as a small internal app in plain English, including an upload form, a results dashboard and a login for your team, and it builds and deploys it. That is a different job from running a model on your own GPU, and it can call AI models for the cleaning step. It suits someone who wants a working tool rather than a one-off chat.

## Frequently Asked Questions

### Can Apex-35B run on a 24 GB graphics card?

Not at Q8_0, which is 37.8 GB. You would need a smaller quantization from the 28 available, and the smallest (IQ1_S) is 9.0 GB. Lower quantizations lose some quality, so test on your own tasks before relying on one.

### Is Apex-35B free for commercial use?

It is released under the Apache 2.0 license, which generally permits commercial use. Read the license file in the Hugging Face repository for the exact terms before you ship anything.

### Why does a model with only 3B active parameters need so much VRAM?

All 35B parameters must be loaded into memory because any of the experts might be needed for the next token. The "active" number affects speed, not file size. Context length adds more memory on top.

### Is Apex-35B better than Qwen3.6-35B-A3B?

On the published benchmarks in the model card, yes, by a wide margin on research and agentic tests such as SciCode and FrontierScience. It also thinks for longer. Run both on your own tasks if cost or speed matters.

### What is Hermes in this setup?

Hermes is the agent software the reviewer connected to the local model. It lets the model act on files and commands instead of only chatting. Any agent tool that can talk to a local OpenAI-style server could fill the same role.
