Can you run Qwen3.8 2.4T-A95B locally?
Qwen/Qwen3.8-2.4T-A95B · 2,446B parameters, 95B active (MoE) · license qwen3.8-max · on Hugging Face since 2026-08-08
Short answer: not on one tracked machine
The weights alone need about 1,483 GB at Q4_K_M and 2,599 GB at Q8_0 (estimates from parameter count × bits per weight). The largest tracked machine has 256GB, so no single box here can hold it at 4-bit.
What it would take (estimate)
Assumes 4-bit weights × 1.15 + 2 GB ≈ 1,707 GB, spread over machines with 85% of their memory usable. Clustering needs software such as llama.cpp RPC and fast networking; we have no public speed result for this model on any tracked setup.
| Option | Hardware cost | EUR |
|---|---|---|
| 16 × Framework Desktop 128GB (DIY) (128GB each) | ≈$55,184 | ≈€62,224 |
| 8 × Mac Studio M5 Ultra 80-core GPU, 256GB / 1TB (256GB each) | ≈$86,392 | ≈€100,232 |
Or rent it
| Cloud GPUs (80GB each) | USD | EUR |
|---|---|---|
| 22 × H100 80GB, RunPod (Community Cloud (Secure $2.89)) | ≈$43.78/hour | ≈€38.91/hour |
| 22 × H100 80GB, Vast.ai (median of 2 live offers) | ≈$71.70/hour | ≈€63.72/hour |
Hourly cost = GPUs needed × tracked H100 PCIe hourly price (see Rent instead). Multi-GPU machines with that many cards may not be offered at this price.
Smaller siblings that fit
Qwen3.8-Flash-Next (≈109.1 GB at 4-bit) · Qwen3.8 27B (≈16.8 GB at 4-bit)
Why it's listed: Qwen's flagship open-weight model.