Models
16 of the most-used open-weight models right now, chosen from Hugging Face data on 2026-10-16. Pick one to see memory needs, which machines and GPUs fit, and measured speed.
Small (about 7–14B)
| Model | Params | 4-bit (est.) | 8-bit (est.) | Devices that fit | With measured tok/s |
|---|---|---|---|---|---|
| Gemma 4 12B | 12B | ≈7.3 GB | ≈12.7 GB | 24 | 2 |
| Qwen3.5 9B | 9.7B | ≈5.9 GB | ≈10.3 GB | 24 | 0 |
| Qwen3 8B | 8.2B | ≈5 GB | ≈8.7 GB | 24 | 0 |
| Llama 3.1 8B | 8B | ≈4.9 GB | ≈8.5 GB | 24 | 2 |
Mid (about 20–35B)
| Model | Params | 4-bit (est.) | 8-bit (est.) | Devices that fit | With measured tok/s |
|---|---|---|---|---|---|
| Qwen3.8 27B | 27.8B | ≈16.8 GB | ≈29.5 GB | 24 | 10 |
| Qwen3.6 35B-A3B | 36B / 3B act. | ≈21.8 GB | ≈38.2 GB | 22 | 13 |
| Qwen3-Coder 30B-A3B | 30.5B / 3B act. | ≈18.5 GB | ≈32.4 GB | 22 | 16 |
| Qwen3.6 27B | 27.8B | ≈16.8 GB | ≈29.5 GB | 24 | 3 |
| Gemma 4 26B-A4B | 25.8B / 4B act. | ≈15.6 GB | ≈27.4 GB | 24 | 12 |
| Muse Glimmer 30B | 29.8B | ≈18.1 GB | ≈31.6 GB | 24 | 0 |
| gpt-oss-20b | 20.9B / 3.6B act. | ≈12.1 GB | ≈22.2 GB | 24 | 8 |
| Qwen3 32B | 32.8B | ≈19.9 GB | ≈34.8 GB | 22 | 1 |
Large (70B+ and big MoE)
| Model | Params | 4-bit (est.) | 8-bit (est.) | Devices that fit | With measured tok/s |
|---|---|---|---|---|---|
| gpt-oss-120b | 116.8B / 5.1B act. | ≈63.4 GB | ≈124.1 GB | 16 | 15 |
| Llama 3.3 70B | 70.6B | ≈42.8 GB | ≈75 GB | 21 | 0 |
| Qwen3.8-Flash-Next | 180B / 6B act. | ≈109.1 GB | ≈191.2 GB | 1 | 0 |
| GLM-5.3-Flash | 321.3B | ≈194.8 GB | ≈341.4 GB | 0 | 0 |
Too big to run locally (yet)
Hugely popular open-weight models whose 4-bit weights exceed 256GB, the most memory on any tracked machine. Estimates only; no speeds.
| Model | Params | 4-bit (est.) | 8-bit (est.) | Machines needed (est.) |
|---|---|---|---|---|
| GLM-5.3 | 753B | ≈457 GB | ≈800 GB | 3 × 256GB or 5 × 128GB |
| Muse Spark Open 816B-A42B | 822B | ≈498 GB | ≈873 GB | 3 × 256GB or 6 × 128GB |
| DeepSeek-V4.1-Flash | 763B | ≈463 GB | ≈811 GB | 3 × 256GB or 5 × 128GB |
| DeepSeek-V4-Pro | 1,599B | ≈969 GB | ≈1,699 GB | 6 × 256GB or 11 × 128GB |
| Kimi K3 | 2,780B | ≈1,685 GB | ≈2,954 GB | 9 × 256GB or 18 × 128GB |
| Qwen3.8 2.4T-A95B | 2,446B | ≈1,483 GB | ≈2,599 GB | 8 × 256GB or 16 × 128GB |
How we chose these models
Snapshot of the Hugging Face API on 2026-10-16: (1) top 500 GGUF repos by downloads, with each quant repo credited to its base model via the base_model:quantized tag and downloads summed per base model; (2) top 60 text-generation models by downloads, likes and trendingScore. We kept official open-weight text LLMs from the original publisher that rank high on either list, dropped audio/image/embedding models, community fine-tunes and uncensored merges, and dropped models whose 4-bit weights exceed 256GB (the largest tracked machine). Older but still heavily downloaded families (Qwen3, Llama 3.x) are kept for size coverage. Mistral and DeepSeek distills did not rank in either list on this date.
- Qwen 4.1 Flash: No Hugging Face repo named Qwen4.1 or Qwen 4.1 Flash on 2026-10-16. Nearest match: Qwen/Qwen3.8-Flash-Next (architecture tag qwen4_exp), included.
- GLM 5.3: zai-org/GLM-5.3 exists (753B, license 'glm-5.3', 2026-08-25): ~4-bit is ~450GB, beyond every tracked machine, so excluded. zai-org/GLM-5.3-Flash (321B, MIT, 2026-08-25) included.
- Muse (released ~yesterday): meta-models/Muse-Spark-Open-816B-A42B (822B, 'muse-model-license', manually gated; repo created 2026-09-15, community quants appeared 2026-10-15/16). ~4-bit is ~490GB, so not runnable on any tracked machine; excluded. The smaller meta-models/Muse-Glimmer-30B (Apache-2.0, 2026-08-09) is included.
Memory and fit
Weights in GB = parameters × bits per weight ÷ 8 (Q4_K_M ≈ 4.85 bpw, Q8_0 ≈ 8.5 bpw; gpt-oss uses its published MXFP4 file size). ESTIMATE. A device fits if weights × 1.15 + 2 GB (context, runtime) ≤ usable memory: VRAM − 1 GB for graphics cards, 85% of unified memory for machines.
Also see the original benchmark pages: Llama-2 7B Q4_0 (small-model reference).