Best GPU for AI: ranked for running models locally
For running AI models on your own computer (chatbots, coding assistants, vision models), the GPU that matters is the one with the most memory you can afford, then the fastest memory. We rank every consumer card by how many of the 60 most-downloaded open models it runs fully in memory at 4-bit or better with an 8K context. Picks are computed from our data and update automatically as models change.
- Best overall: RTX 5090 (32 GB, $1,999): runs 43 of the top 60 models, with 1792 GB/s of bandwidth.
- Best under $800: RTX 5070 Ti (16 GB, $749): runs 32 of the top 60 models.
- Cheapest way to the 24 GB tier: RX 7900 XTX (24 GB, $999): runs 39 of the top 60 models. Used RTX 3090s are popular for the same reason; we use launch prices only.
- Best budget GPU for AI: Arc A770 16GB (16 GB, $349): runs 32 of the top 60 models.
Launch MSRP, not street prices. "Runs" = fits fully in GPU memory at 4-bit or better with 8K context; models that need more can still run more slowly with CPU offload. Ties go to the faster card, then the cheaper one.
Best budget GPU for AI
Under about $350, a 12–16 GB card beats an 8 GB one for AI even when the 8 GB card is newer: the extra memory lets 7–9B models run at 8-bit and makes room for 12–15B models. Our pick is the Arc A770 16GB. The under-$300 and under-$500 pages list every card with speed estimates, and Arc B580 vs RTX 4060 and RTX 3060 vs 4060 compare the usual candidates.
Best NVIDIA GPU for AI
NVIDIA has the widest software support: every local AI tool supports CUDA first. The strongest consumer card is the RTX 5090 (32 GB, $1,999): runs 43 of the top 60 models. At $500 or less, the best NVIDIA option is the RTX 5060 Ti 16GB (16 GB, $429): runs 32 of the top 60 models. The 16 GB RTX 4060 Ti and 5060 Ti are slower than the 12 GB RTX 4070 on games but fit more models; see RTX 4060 Ti 16GB vs RTX 4070.
Best AMD GPU for AI
AMD cards work with Ollama, LM Studio and llama.cpp through ROCm or Vulkan; ROCm support depends on the model and operating system, so check the supported list first. The best AMD card for local models is the RX 7900 XTX (24 GB, $999): runs 39 of the top 60 models. AMD often gives more memory per dollar:
RX 7900 XTX: 24 GB, $999, 39 modelsRX 7900 XT: 20 GB, $899, 33 modelsRX 7800 XT: 16 GB, $499, 32 modelsRX 7900 GRE: 16 GB, $549, 32 modelsRX 9070: 16 GB, $549, 32 modelsRX 9070 XT: 16 GB, $599, 32 models
Head to head: RX 7900 XTX vs RTX 4090 · RX 9070 XT vs RTX 5070 Ti.
Intel GPUs for AI
Intel Arc cards run local models through Vulkan (Ollama, LM Studio, llama.cpp) or llama.cpp's SYCL build. Support is newer and less tested than CUDA, but the price per GB of memory is low: the Arc A770 16GB has 16 GB at $349 and runs 32 of the top 60, the Arc B580 has 12 GB at $249 and runs 29 of the top 60.
NVIDIA DGX Spark, Macs and other unified-memory machines
For models too big for any gaming card (70B and up, or large mixture-of-experts models), machines with one big shared memory pool are the practical home option. They hold much larger models but run them more slowly than a high-end GPU, because their memory bandwidth is lower.
| Machine | Memory | Usable for models | Bandwidth | Top-60 models that fit |
|---|---|---|---|---|
| Mac M3 Ultra 512GB | 512 GB | 384 GB | 819 GB/s | 51 |
| Mac M3 Ultra 256GB | 256 GB | 192 GB | 819 GB/s | 49 |
| Mac M2 Ultra 192GB | 192 GB | 144 GB | 800 GB/s | 46 |
| Mac M4 Max 128GB | 128 GB | 96 GB | 546 GB/s | 45 |
| Mac M3 Max 128GB | 128 GB | 96 GB | 400 GB/s | 45 |
| DGX Spark | 128 GB | 109 GB | 273 GB/s | 45 |
| Ryzen AI Max+ 395 128GB | 128 GB | 109 GB | 256 GB/s | 45 |
| Mac M2 Max 96GB | 96 GB | 72 GB | 400 GB/s | 44 |
See DGX Spark vs Mac M4 Max 128GB, DGX Spark vs RTX 5090 and the Mac vs PC guide.
Data-center GPUs: H100, A100, H200
Data-center cards are built for serving many users and for training, not for one person at a desk. For inference alone, here is what each holds:
| GPU | Memory | Bandwidth | Top-60 models that fit | Largest that fits |
|---|---|---|---|---|
| A100 40GB | 40 GB | 1,555 GB/s | 43 | Qwen3.6 35B A3B (36B) |
| RTX A6000 | 48 GB | 768 GB/s | 43 | Qwen3.6 35B A3B (36B) |
| L40S | 48 GB | 864 GB/s | 43 | Qwen3.6 35B A3B (36B) |
| RTX 6000 Ada | 48 GB | 960 GB/s | 43 | Qwen3.6 35B A3B (36B) |
| H100 PCIe | 80 GB | 2,000 GB/s | 44 | gpt-oss 120B (117B) |
| A100 80GB | 80 GB | 2,039 GB/s | 44 | gpt-oss 120B (117B) |
| H100 SXM | 80 GB | 3,350 GB/s | 44 | gpt-oss 120B (117B) |
| RTX PRO 6000 Blackwell | 96 GB | 1,792 GB/s | 45 | gpt-oss 120B (117B) |
| H200 | 141 GB | 4,800 GB/s | 46 | Qwen3.8 Flash Next (180B) |
Training needs much more memory
Everything above is about running models. Training is different: full fine-tuning with the Adam optimizer in mixed precision needs about 16 bytes per parameter before activations (ZeRO paper), so about 120 GB for an 8B model that runs in about 6 GB at 4-bit. That is why full training happens on multi-GPU data-center systems. Parameter-efficient methods such as LoRA and QLoRA cut this a lot and can fit small models on a 16–24 GB card, but our numbers don't cover them.
Every consumer GPU, ranked for local AI
Popular comparisons: RTX 4060 Ti 16GB vs RTX 4060 Ti 8GBRTX 4060 Ti 16GB vs RTX 4070RTX 4070 Super vs RTX 4070 Ti SuperRTX 4080 vs RTX 4090RTX 5060 Ti 16GB vs RTX 5070RTX 5070 Ti vs RTX 5080RTX 5080 vs RTX 5090RTX 3090 vs RTX 3090 TiRTX 3060 12GB vs RTX 4060RTX 3060 12GB vs RTX 4060 Ti 16GB all
New to this? Start with how to run an LLM locally, then check a specific model with Can I run…? or the VRAM calculator.
Automated: model data from the Hugging Face Hub API, GPU specs and launch prices from vendor pages, formulas on the methodology page. No affiliate links. Report an error.