How to Run the Edited Qwen3.5 0.8B Locally With vLLM
Serve Hugging Face's rebuilt weight-edited Qwen3.5 0.8B beside the original in vLLM and compare looping on a 2^100 mod 7 prompt and MATH-500 problems.
AI News & Models
Local AI
Run AI models on your own laptop, GPU or server: setup guides, hardware, benchmarks and when local beats the cloud.
150 articles · Updated Nov 12, 2026
Most read by people in Engineering, IT and Legal & Compliance.
Northka Computer Use Model: Which Size and How Good?
Northka is an open-weight computer-use model family (397B Pro, 35B Mini, 9B Nano) scoring 78.9 to 88 on OSWorld Verified. Sizes, scores and harness explained.
Northka computer use modelNorthka OSWorldopen source computer use agent
How Do You Fix a Small Reasoning Model Stuck in Loops?
Small reasoning models often loop until the token limit cuts them off. Count truncated answers first, then adjust mode, temperature and output limit.
reasoning model loopingsmall Qwen thinking mode loopfix repetitive reasoning LLM
Can You Run Qwen 3.8 27B Locally With Ollama? VRAM Needs
Run Qwen 3.8 27B locally with Ollama and Simon Willison's LLM tool. Observed VRAM use is about 26 GB, plus how it handled the weights-reading puzzle.
Qwen 3.8 27BQwen 3.8 27B OllamaQwen 3.8 27B VRAM
Did GPT-6 Astra Really Make Qwen 6x Better at Math?
An MIT-linked paper says GPT-6 Astra edited 0.016% of Qwen3.5-0.8B's weights for a sixfold math gain. Here is what changed and what to doubt.
How to Run Ripple-65B Locally: VRAM and Setup Requirements
What it takes to self-host Ripple-65B: about 75 GB VRAM quantized on a 96 GB RTX Pro 6000 Blackwell, inference code changes, a Hermes agent, and DGX Spark fit.
run Ripple-65B locallyRipple 65B VRAMRipple quantized
Ripple-65B Tested: How Good Is This Dense Open Model?
Ripple-65B is a 65B dense Apache 2.0 model aimed at agentic coding. See its benchmark wins and gaps, plus a bug-fix, simulation and SQL test.
Ripple-65BRicursive Intelligence RippleRipple model benchmark
How Good Is Tencent Hy4 at Long Agentic Tasks? Hands-On Tests
Tencent's Hy4 was tested on a sketch-to-Blender scene, a multi-agent IPO report with a planted error, and 82-language output. Results and caveats.
How to Run Qwen3.5-4B GLB 3D Coder Locally and Use It in Godot
Run CCORE's Qwen3.5-4B GLB 3D Coder locally from a GGUF, set the sampling and system prompt, then export Three.js scenes as GLB files for Godot.
Qwen3.5-4B GLB 3D Codertext to 3D GLB modelThree.js GLB export
How Do You Run NVIDIA Dory Locally? Loops, KV Cache, Formats
Run NVIDIA's 3.97B Dory looped model locally: pick a loop count in vLLM, choose per-loop or last-loop KV cache, and pick BF16, MXFP8, NVFP4 or GGUF.
run Dory locallyNVIDIA Dory vLLMDory GGUF
Frontier vs Open-Weight Model: How to Decide With Step-Down Tests
How to decide if you can replace a frontier model with a self-hosted open-weight one: build a swap layer, step down generation by generation, then fine-tune.
choose LLM for businessopen-weight vs frontier modelself-host open-weight model
How to Run MOSS-Image 5B Locally in ComfyUI (VRAM, Setup)
Install the 5B MOSS-Image text-to-image model in ComfyUI: where the files go, the node setting that matters, about 28 GB VRAM use and layout mode.
MOSS-ImageMOSS-Image ComfyUIrun MOSS-Image locally
How to Install Mesh LLM and Pool GPUs Across Machines
Install Mesh LLM with Docker, start a node with a Gemma model, edit and validate config, and use one OpenAI-compatible API across pooled GPUs.
Mesh LLMpool GPUs across machinesOpenAI compatible API local
How to Run SenseNova-Loop-9B-Preview: Setup and Serving
Run SenseNova-Loop-9B-Preview with Transformers 5.18, LightLLM or an SGLang PR build: environment, BF16 loading, thinking mode, MTP and fine-tuning commands.
SenseNova-Loop-9B-Previewrun SenseNova Loop locallySenseNova Loop SGLang
MOSS-Image Review: Layout Control and Text Rendering Tested
Hands-on MOSS-Image results: bounding-box layout control, English, Chinese and Urdu text, plus portraits and landscapes from many regions.
MOSS-Image reviewMOSS-Image testAI image layout control
LFM3-1.7B-Base: Specs, Memory Needs and Fine-Tuning Guide
Liquid AI's LFM3-1.7B-Base is a 1.73B model with 256K context that takes about 1 GB at 4-bit. Specs, inference code, thinking toggle and fine-tuning with TRL.
LFM3-1.7B-BaseLiquid AI LFM3LFM3 fine-tuning
How to Run DeepSeek and Qwen on One GPU With Colibri 2.1
Colibri 2.1 runs large MoE models on one GPU by spreading experts across VRAM, RAM and disk. See the install flow, GGUF import, expert map and profiling tab.
Colibri 2.1run DeepSeek locallyGGUF import
Why Did Open Models Fall 12 Points Behind in Index v5?
Artificial Analysis Intelligence Index v5 uses 60% private tests and harder agentic coding. The open-model gap grew from 3 to 12 points. Here is what it means.
Artificial Analysis Intelligence Index v5open models vs closed modelsFrontier Code 2
Shared On-Prem AI Box vs Cloud APIs: When Does It Win?
When does one shared on-prem GB300 workstation beat cloud APIs or a Spark per person? Privacy, token costs, model policy control, and scaling.
on-prem AI vs cloud APIshared local AI server teamprivate LLM hosting
How Accurate Is Phonon 2? 5.21% WER in a 164MB ASR Model
Phonon 2 averages 5.21% WER on Open ASR leaderboard English sets in 164MB, 15x smaller than Parakeet. See where it holds up and where it fails.
Phonon 2 benchmarkOpen ASR leaderboardPhonon 2 word error rate
How to Run Phonon 2 Locally on CPU for Speech Recognition
Run Fermion Research's 164 MB Phonon 2 ASR model on CPU only. Install steps, CPU timings, the OpenAI-compatible server, accuracy numbers and license notes.
Phonon 2Phonon 2 speech recognitionrun ASR locally CPU
How to Run Surya OCR 2.5 Locally on CPU or GPU
Install Surya OCR 2.5 with pip, launch the Streamlit GUI, and run it on CPU or about 1.8 GB of VRAM. Covers backends, output formats, speed, and limits.
Surya OCR 2.5run OCR locallySurya OCR install
How Accurate Is Surya OCR 2.5? Tested on Real Documents
Surya OCR 2.5 tested on invoices, handwriting, an 1883 newspaper, LaTeX, Hindi, Arabic, Urdu and Yoruba. See where it works and where it fails.
Surya OCR 2.5 reviewOCR model testhandwriting OCR accuracy
HP ZGX Fury Review: Can One GB300 Box Serve a Whole Team?
HP's ZGX Fury puts Nvidia's GB300 under a desk with 748GB of memory and 7.1 TB/s HBM. Here are the specs, ZRT software and measured multi-user throughput.
HP ZGX FuryDGX Station GB300ZGX Fury review