Graphics cards (needs a host PC)

Card-only prices. Add the cost of a PC, power supply and cooling before comparing with the complete machines on the main ranking.

CardUSDEURSmall-model tok/sper $1kper €1k
NVIDIA RTX 3090 24GB (used)$700≈€622 172246.1276.9
NVIDIA RTX 5090 32GB (new)n/an/a 314––
NVIDIA RTX 4090 24GB (used)n/an/a 192––

Small model: llama.cpp llama-bench, Llama-2 7B Q4_0, tg128 (text generation tok/s); best of FA on/off as posted.

Which models fit each card, and how fast

Numbers are generation speed in tokens per second (tok/s), measured by others and linked to the source. Fit assumes 4-bit weights (estimate) on a single card; llm-speed.com data CC BY 4.0.

ModelRTX 5090 32GB (new)RTX 4090 24GB (used)RTX 3090 24GB (used)
Qwen3.8 27B ≈16.8 GBfitsno result yetfitsno result yetfitsno result yet
Qwen3.6 35B-A3B ≈21.8 GB220.4 tok/sdoesn't fitdoesn't fit
Gemma 4 12B ≈7.3 GBfitsno result yetfitsno result yetfitsno result yet
Qwen3-Coder 30B-A3B ≈18.5 GB259.9 tok/sdoesn't fitdoesn't fit
Qwen3.5 9B ≈5.9 GBfitsno result yetfitsno result yetfitsno result yet
Qwen3.6 27B ≈16.8 GB70.6 tok/sfitsno result yetfitsno result yet
Gemma 4 26B-A4B ≈15.6 GBfitsno result yetfitsno result yetfitsno result yet
Muse Glimmer 30B ≈18.1 GBfitsno result yetfitsno result yetfitsno result yet
gpt-oss-20b ≈12.1 GB193.9 tok/sfitsno result yetfitsno result yet
gpt-oss-120b ≈63.4 GBdoesn't fitdoesn't fitdoesn't fit
Qwen3 8B ≈5 GBfitsno result yetfitsno result yetfitsno result yet
Llama 3.1 8B ≈4.9 GB230.8 tok/sfitsno result yet136.2 tok/s
Qwen3 32B ≈19.9 GB69.5 tok/sdoesn't fitdoesn't fit
Llama 3.3 70B ≈42.8 GBdoesn't fitdoesn't fitdoesn't fit
Qwen3.8-Flash-Next ≈109.1 GBdoesn't fitdoesn't fitdoesn't fit
GLM-5.3-Flash ≈194.8 GBdoesn't fitdoesn't fitdoesn't fit