Edge0-145B: a 145B model on one 24GB GPU, with catches
Edge0-145B claims a 145B model on one 24GB card or a Mac. What it really needs (a 106 GB download, its own engine) and the licence catch.
· Updated · 3 min read
Edge0-145B, released on 13 November by the Edge0 team, is a 145-billion-parameter mixture-of-experts model that its makers say runs on one NVIDIA graphics card (they measured 24GB cards) or an Apple silicon Mac. According to the Edge0 README (read 15 November 2026), it is derived from Qwen3.8 Flash Next, activates 10.12 billion parameters per token, and runs on Edge0's own engine. Every speed and benchmark figure so far is Edge0's own.
Updated 15 November 2026: Edge0 added a 0.9 GB vision encoder to the Hugging Face folder; the size below is revised.
What did Edge0 release?
Two models, Edge0-145B and a smaller Edge0-120B, each in Q3_K and NVFP4. The Edge0 model page describes 48 layers with 512 experts each, 10 routed plus one shared per token, a 256K context, and a speculative-decoding layer. Its experts work on a compressed copy of each token, small enough that the engine keeps thousands on the card and streams the rest from system memory.
The base model is Qwen's Qwen3.8 Flash Next, which Qwen's card lists at 125B parameters with 6B active, plus a 51B n-gram embedding table. See our Qwen3.8 Flash Next brief for that model in Ollama.
What hardware does Edge0-145B need?
The README gives NVIDIA GPUs with CUDA 13 on Linux or Windows, and Metal on Apple silicon. The edge0-infer guide for the 145B asks for at least 16 GiB of GPU memory and 20 GiB of system memory on Linux and Windows. On a Mac only the Q3_K build works, with a 32,768-token context. AMD and Intel graphics are not listed.
Edge0's own measurements, from the model page on 15 November 2026:
| Machine (Edge0's test) | Build | Agent decode | 16K prefill |
|---|---|---|---|
| RTX 5090 D 24 GB | Q3_K | 121.2 tok/s | 3,554.9 tok/s |
| RTX 4090 24 GB | Q3_K | 91.7 tok/s | 3,179.4 tok/s |
| Apple M2 Ultra (memory not stated) | Q3_K | 57.0 tok/s | 800.3 tok/s |
| RTX 5090 D capped at 16 GiB VRAM + 20 GiB RAM | Q3_K | 44.9 tok/s | 335 tok/s |
It is not living in 24GB alone: with every expert resident, the model page says the Q3_K build occupied 23.14 GiB of graphics memory plus 31.28 GiB of system memory. The capped row streams uncached experts from the SSD.
Do the benchmark scores hold up?
No independent run had been published as of 15 November that we could find. The README's table puts Edge0-145B at 86.5 on Terminal-Bench 2.1, ahead of the 83.3 it lists for Qwen3.8 Flash Next. Those are Edge0's figures: Qwen's own Flash Next card does not report a Terminal-Bench 2.1 score, and the README gives no harness settings.
What it means if you run models yourself
- A new engine. The model runs on edge0-strata, a fork of the MIT-licensed Strata engine, started by
edge0-infer. You build both from source, or use the Edge0 Chat desktop app (Windows, Apple silicon). Edge0 documents no Ollama, LM Studio or llama.cpp support, and the folder is a custom expert pack, not a single GGUF. - An OpenAI- and Anthropic-compatible server.
edge0-infer servelistens on127.0.0.1:8000with the model nameedge0_145b, so a coding agent can talk to it:
build/cpu/bin/edge0-infer serve --model /path/to/Edge0-145B-q3k
- The licence is not Apache. The Edge0 Community Model License allows personal, research and internal use, and commercial use of outputs, but needs written permission for paid model services and for "independent commercial coding or office assistants". Qwen's community licence still applies on top; read both before building anything on it.
Choosing by memory? See the best local LLM per memory tier.
Using it from your other machines
Because edge0-infer serve speaks the OpenAI chat format, Corewire can adopt it with Add endpoint in the window's Runtimes panel, so a laptop on the train can reach the model on the desktop with the 24GB card. The OpenCode guide shows pointing an agent at a model on another of your computers.