---
title: Open-Weight Model Benchmarks
url: "https://goremy.ai/blog/topic/open-weight-model-benchmarks"
type: topic
updated: "2026-11-05T23:00:12.496Z"
posts: 32
---

# Open-Weight Model Benchmarks

Find open-weight model benchmarks, hands-on tests and head-to-head scores for newly released open models like Kimi K3.1 Pro, Nemotron and Flint.

Answers: How do open-weight models like Kimi K3.1 Pro and Nemotron compare on benchmarks?

## Posts (32)

- [Ripple-65B Tested: How Good Is This Dense Open Model?](https://goremy.ai/blog/articles/ripple-65b-dense-model-tested-agentic-coding) (2026-11-05): Ripple-65B is a 65B dense Apache 2.0 model aimed at agentic coding. See its benchmark wins and gaps, plus a bug-fix, simulation and SQL test.
- [How Good Is Tencent Hy4 at Long Agentic Tasks? Hands-On Tests](https://goremy.ai/blog/articles/tencent-hy4-hands-on-test-blender-finance) (2026-11-05): Tencent's Hy4 was tested on a sketch-to-Blender scene, a multi-agent IPO report with a planted error, and 82-language output. Results and caveats.
- [MOSS-Image Review: Layout Control and Text Rendering Tested](https://goremy.ai/blog/articles/moss-image-layout-control-text-rendering-test) (2026-11-03): Hands-on MOSS-Image results: bounding-box layout control, English, Chinese and Urdu text, plus portraits and landscapes from many regions.
- [Why Did Open Models Fall 12 Points Behind in Index v5?](https://goremy.ai/blog/articles/artificial-analysis-index-v5-open-models-gap) (2026-11-02): Artificial Analysis Intelligence Index v5 uses 60% private tests and harder agentic coding. The open-model gap grew from 3 to 12 points. Here is what it means.
- [How Accurate Is Phonon 2? 5.21% WER in a 164MB ASR Model](https://goremy.ai/blog/articles/phonon-2-word-error-rate-open-asr-benchmark) (2026-11-01): Phonon 2 averages 5.21% WER on Open ASR leaderboard English sets in 164MB, 15x smaller than Parakeet. See where it holds up and where it fails.
- [How Accurate Is Surya OCR 2.5? Tested on Real Documents](https://goremy.ai/blog/articles/surya-ocr-2-5-test-results-languages) (2026-10-31): Surya OCR 2.5 tested on invoices, handwriting, an 1883 newspaper, LaTeX, Hindi, Arabic, Urdu and Yoruba. See where it works and where it fails.
- [What GPU Does Sage A1.5 Need? Local VRAM and Agent Tests](https://goremy.ai/blog/articles/sage-a1-5-35b-agentic-model-local-test) (2026-10-30): Sage A1.5 is a 35B MoE agentic model from Intern. See the reported 38 GB VRAM with llama.cpp, how it was trained, and how it handled sub-agent tasks.
- [Is Clover-35B-A3B-Preview Any Good? Diffusion LLM Tested](https://goremy.ai/blog/articles/clover-35b-a3b-preview-diffusion-llm-review) (2026-10-30): Hands-on test of DiffuSpace's Clover-35B-A3B-Preview diffusion LLM: bug fix, coding, multilingual and logic puzzle results versus its benchmark claims.
- [Does LFM3-1.7B Match 9B Models on Agent Benchmarks?](https://goremy.ai/blog/articles/lfm3-1-7b-benchmarks-vs-qwen-gemma-minicpm) (2026-10-29): LFM3-1.7B is pitched as matching 9B models on agent benchmarks. Here is what the card claims, what hands-on tests showed, and where it fell short.
- [How Does Unlimited OCR 2 Score on 100-Page Documents?](https://goremy.ai/blog/articles/unlimited-ocr-2-long-document-benchmark-results) (2026-10-28): Unlimited OCR 2 scores about 77 on a 100-page document test while most rivals fall under 30. How SR-SWA attention works, plus where the model still fails.
- [Is Nanbeige4.3 33B Claude-Level? Benchmarks vs Real Tests](https://goremy.ai/blog/articles/nanbeige4-3-33b-test-results-claude-level-claim) (2026-10-27): Nanbeige4.3 33B tops Qwen 3.8 27B and Gemma 4 on most charts, but looped endlessly in a bug-fix and an HTML test. Here is how the claims hold up.
- [How Good Is Moss Image 5B, the Apache 2.0 Image Model?](https://goremy.ai/blog/articles/moss-image-5b-open-text-to-image-review) (2026-10-27): Moss Image is a 5B open-weight text-to-image model under Apache 2.0. See its text rendering, layout control, benchmark claims, and where it falls short.
- [Kimi K3.1 Pro vs Opus 5.5: Has Open Source Caught Up?](https://goremy.ai/blog/articles/kimi-k3-1-pro-vs-opus-5-5-open-source-caught-up) (2026-10-27): Kimi K3.1 Pro ranks fourth on Artificial Analysis and sits under a point behind Opus 5.5 on a coding benchmark. Here is what that means for builders.
- [How Good Is Nemotron-3.5 Bolt E4B? Tested Against Gemma 4](https://goremy.ai/blog/articles/nemotron-3-5-bolt-e4b-hands-on-test-results) (2026-10-27): Hands-on results for NVIDIA's Nemotron-3.5 Bolt E4B: a bug-hunt agent task that failed, a broken HTML scene, and passes on a riddle, physics math and rostering.
- [How Does Nanbeige4.3-33B Score vs Qwen3.8-27B and Gemma4-31B?](https://goremy.ai/blog/articles/nanbeige-4-3-33b-benchmarks-vs-qwen-gemma) (2026-10-27): Nanbeige4.3-33B leads 17 of 20 benchmarks against Qwen3.8-27B, Gemma4-31B and Muse-Glimmer-30B. See the full table and where it loses.
- [Kimi K3.1 Pro vs Opus 5.5: Which Is Better for Agentic Tasks?](https://goremy.ai/blog/articles/kimi-k3-1-pro-vs-opus-5-5-claude-code) (2026-10-27): A 13-task test of Kimi K3.1 Pro against Claude Opus 5.5 in the same Claude Code harness: where Kimi matches Opus, where it cuts corners, and what to check.
- [Kimi K3.1 Pro vs Opus 5.5 vs GPT 6.1 Soul: Which Is Better?](https://goremy.ai/blog/articles/kimi-k3-1-pro-vs-opus-5-5-gpt-6-1-soul) (2026-10-25): Kimi K3.1 Pro tested against Opus 5.5 and GPT 6.1 Soul: pricing, open-weight caveats, and hands-on demos from a Rubik's cube to a Fall Guys clone.
- [Kimi K3.1 Pro Benchmark: Does It Match Claude Opus 5.5?](https://goremy.ai/blog/articles/kimi-k3-1-pro-kingbench-results) (2026-10-25): Kimi K3.1 Pro claims near-Opus results, but a six-task KingBench run scored it 5.37/10. See task scores, cost, and how it compares to rivals.
- [Flint vs Qwen 3.8 27B: Which Local Coding Model Wins?](https://goremy.ai/blog/articles/flint-vs-qwen-3-8-27b-coding-benchmarks) (2026-10-24): Flint is roughly 10x faster than dense Qwen 3.8 27B but scores 55 vs 73 on Terminal Bench. Here is where each local coding model fits.
- [How Good Is Ling-3.0 Flash Santé at Medical Reasoning?](https://goremy.ai/blog/articles/ling-3-0-flash-sante-healthcare-model-tested) (2026-10-24): InclusionAI's MIT-licensed Ling-3.0 Flash Santé is a 124B MoE healthcare model. See clinical test results, benchmark standing, and its lack of vision.
- [How Accurate Is Carbon-X on DNA Tasks vs GPT and Opus?](https://goremy.ai/blog/articles/carbon-x-benchmarks-vs-gpt-opus-dna-variant-effects) (2026-10-23): Carbon-X, a 14.8B DNA model, averages 66.7% on five benchmarks versus 67.5% for GPT-6.1 Sol at far lower cost. Results, weak spots, and limits.
- [How Good Is Flint 40B at Coding? Benchmarks vs Qwen 3.8](https://goremy.ai/blog/articles/flint-40b-benchmarks-terminal-bench-swe-bench) (2026-10-23): Flint 40B scores 54.9% on Terminal-Bench 2.1 and 74.2% on SWE-bench Verified. See M4 Pro speed, NVFP4 vs BF16, and where Qwen 3.8 still wins.
- [How Does Sage-A1.5 Compare to Kimi-K3, GLM-5.3 and GPT-5.6-sol?](https://goremy.ai/blog/articles/sage-a1-5-benchmarks-vs-kimi-k3-glm-5-3) (2026-10-23): Sage-A1.5, a 35B MoE agentic model, hits 95.1 on GAIA and 61.2 on SWE-bench Pro. Here is where it beats or trails Kimi-K3, GLM-5.3 and GPT-5.6-sol.
- [Does Vast-10M's 10-Million-Token Context Window Work in Practice?](https://goremy.ai/blog/articles/vast-10m-10-million-token-context-review) (2026-10-23): Hands-on look at Vast-10M, which swaps attention in DeepSeek V4 and GLM 5.2 for VSA. Results from a game build, HTML art, a legal corpus and 81 languages.
- [Sage-A1.5 Benchmarks: How Good Is Shanghai AI Lab's 35B Agent Model?](https://goremy.ai/blog/articles/sage-a1-5-shanghai-ai-lab-agentic-model) (2026-10-23): Sage-A1.5 is a 35B MoE agentic model from Shanghai AI Lab. See GAIA, BrowseComp and SWE-bench Pro scores against larger models, plus SGLang and vLLM serving.
- [How does Ornith-1.5-122B score against Qwen 3.8 and DeepSeek V4 Flash?](https://goremy.ai/blog/articles/ornith-1-5-122b-benchmarks-vs-qwen-deepseek-nemotron) (2026-10-22): Ornith-1.5-122B posts 76.9 on SWE-bench Pro and 84.5 on Verified. Here is how it compares with Qwen3.8, Nemotron 3.5 Super VL, DeepSeek-V4-Flash and Beam.
- [How does Hy-Memory-30B-A3B score against Claude Opus 5?](https://goremy.ai/blog/articles/hy-memory-30b-a3b-benchmark-results) (2026-10-22): Tencent's Hy-Memory-30B-A3B memory writer averages 55.49 across four harnesses vs Claude Opus 5 at 54.62. Per-harness and per-benchmark scores inside.
- [How does Spark-X2.6-4B compare with Qwen3.5-4B and Gemma-E4B?](https://goremy.ai/blog/articles/spark-x2-6-4b-vs-qwen-3-5-4b-gemma-e4b) (2026-10-22): Spark-X2.6-4B claims best-in-class small-model results and efficiency on par with Qwen3.5-4B and Gemma-E4B. What the card says and what it leaves out.
- [How Good Is DeepSeek-V4.1-Flash-1017 vs Opus 5 and GPT-5.6 Sol?](https://goremy.ai/blog/articles/deepseek-v4-1-flash-1017-benchmarks) (2026-10-22): DeepSeek-V4.1-Flash-1017 gains 11.1 points on Terminal-Bench 4.0 over V4.1-Flash. See how it stacks up against Opus 5, GPT-5.6 Sol, GLM-5.3 and K3.
- [What Is Nemotron 3.5 Super VL? Architecture, Speed and 1M Context](https://goremy.ai/blog/articles/nemotron-3-5-super-vl-architecture-speed-context) (2026-10-21): NVIDIA's open-weight Nemotron 3.5 Super VL uses Mamba-2, MoE and attention with 120B total, 12B active parameters and 1M context. Here is where it fits.
- [How Good Is Nemotron 3.5 Super VL for Agentic Coding and Vision?](https://goremy.ai/blog/articles/nemotron-3-5-super-vl-hands-on-test) (2026-10-21): Hands-on results for NVIDIA's open Nemotron 3.5 Super VL (120B, 12B active): bug fixing, vision, translation, circuits, and benchmarks.
- [How Does Ornith-1.5-122B Perform on Real Tasks? Hands-On Test Results](https://goremy.ai/blog/articles/ornith-1-5-122b-hands-on-review) (2026-10-21): Ornith-1.5-122B tested on a Docker bug fix, vision, screenshot reasoning, creative HTML and 80-language translation. Here is where it held up and failed.