RunsOnMy ModelsGPUs & MacsBy VRAMBest GPUCan I run…?OllamaStart hereAPI pricesVRAM calculatorNew

Best GPU for AI: ranked for running models locally

For running AI models on your own computer (chatbots, coding assistants, vision models), the GPU that matters is the one with the most memory you can afford, then the fastest memory. We rank every consumer card by how many of the 60 most-downloaded open models it runs fully in memory at 4-bit or better with an 8K context. Picks are computed from our data and update automatically as models change.

RTX 5090best overall (≤ $2,000)
RTX 5070 Tibest under $800
Arc A770 16GBbest budget (≤ $350)

Launch MSRP, not street prices. "Runs" = fits fully in GPU memory at 4-bit or better with 8K context; models that need more can still run more slowly with CPU offload. Ties go to the faster card, then the cheaper one.

Best budget GPU for AI

Under about $350, a 12–16 GB card beats an 8 GB one for AI even when the 8 GB card is newer: the extra memory lets 7–9B models run at 8-bit and makes room for 12–15B models. Our pick is the Arc A770 16GB. The under-$300 and under-$500 pages list every card with speed estimates, and Arc B580 vs RTX 4060 and RTX 3060 vs 4060 compare the usual candidates.

Best NVIDIA GPU for AI

NVIDIA has the widest software support: every local AI tool supports CUDA first. The strongest consumer card is the RTX 5090 (32 GB, $1,999): runs 43 of the top 60 models. At $500 or less, the best NVIDIA option is the RTX 5060 Ti 16GB (16 GB, $429): runs 32 of the top 60 models. The 16 GB RTX 4060 Ti and 5060 Ti are slower than the 12 GB RTX 4070 on games but fit more models; see RTX 4060 Ti 16GB vs RTX 4070.

Best AMD GPU for AI

AMD cards work with Ollama, LM Studio and llama.cpp through ROCm or Vulkan; ROCm support depends on the model and operating system, so check the supported list first. The best AMD card for local models is the RX 7900 XTX (24 GB, $999): runs 39 of the top 60 models. AMD often gives more memory per dollar:

RX 7900 XTX: 24 GB, $999, 39 modelsRX 7900 XT: 20 GB, $899, 33 modelsRX 7800 XT: 16 GB, $499, 32 modelsRX 7900 GRE: 16 GB, $549, 32 modelsRX 9070: 16 GB, $549, 32 modelsRX 9070 XT: 16 GB, $599, 32 models

Head to head: RX 7900 XTX vs RTX 4090 · RX 9070 XT vs RTX 5070 Ti.

Intel GPUs for AI

Intel Arc cards run local models through Vulkan (Ollama, LM Studio, llama.cpp) or llama.cpp's SYCL build. Support is newer and less tested than CUDA, but the price per GB of memory is low: the Arc A770 16GB has 16 GB at $349 and runs 32 of the top 60, the Arc B580 has 12 GB at $249 and runs 29 of the top 60.

NVIDIA DGX Spark, Macs and other unified-memory machines

For models too big for any gaming card (70B and up, or large mixture-of-experts models), machines with one big shared memory pool are the practical home option. They hold much larger models but run them more slowly than a high-end GPU, because their memory bandwidth is lower.

MachineMemoryUsable for modelsBandwidthTop-60 models that fit
Mac M3 Ultra 512GB512 GB384 GB819 GB/s51
Mac M3 Ultra 256GB256 GB192 GB819 GB/s49
Mac M2 Ultra 192GB192 GB144 GB800 GB/s46
Mac M4 Max 128GB128 GB96 GB546 GB/s45
Mac M3 Max 128GB128 GB96 GB400 GB/s45
DGX Spark128 GB109 GB273 GB/s45
Ryzen AI Max+ 395 128GB128 GB109 GB256 GB/s45
Mac M2 Max 96GB96 GB72 GB400 GB/s44

See DGX Spark vs Mac M4 Max 128GB, DGX Spark vs RTX 5090 and the Mac vs PC guide.

Data-center GPUs: H100, A100, H200

Data-center cards are built for serving many users and for training, not for one person at a desk. For inference alone, here is what each holds:

GPUMemoryBandwidthTop-60 models that fitLargest that fits
A100 40GB40 GB1,555 GB/s43Qwen3.6 35B A3B (36B)
RTX A600048 GB768 GB/s43Qwen3.6 35B A3B (36B)
L40S48 GB864 GB/s43Qwen3.6 35B A3B (36B)
RTX 6000 Ada48 GB960 GB/s43Qwen3.6 35B A3B (36B)
H100 PCIe80 GB2,000 GB/s44gpt-oss 120B (117B)
A100 80GB80 GB2,039 GB/s44gpt-oss 120B (117B)
H100 SXM80 GB3,350 GB/s44gpt-oss 120B (117B)
RTX PRO 6000 Blackwell96 GB1,792 GB/s45gpt-oss 120B (117B)
H200141 GB4,800 GB/s46Qwen3.8 Flash Next (180B)

Training needs much more memory

Everything above is about running models. Training is different: full fine-tuning with the Adam optimizer in mixed precision needs about 16 bytes per parameter before activations (ZeRO paper), so about 120 GB for an 8B model that runs in about 6 GB at 4-bit. That is why full training happens on multi-GPU data-center systems. Parameter-efficient methods such as LoRA and QLoRA cut this a lot and can fit small models on a 16–24 GB card, but our numbers don't cover them.

Every consumer GPU, ranked for local AI

GPUVRAMBandwidthLaunch MSRPTop-60 models that runLargest that fits
RTX 509032 GB1792 GB/s$1,99943Qwen3.6 35B A3B
RTX 409024 GB1008 GB/s$1,59939GLM 4.7 Flash
RTX 3090 Ti24 GB1008 GB/s$1,99939GLM 4.7 Flash
RX 7900 XTX24 GB960 GB/s$99939GLM 4.7 Flash
RTX 309024 GB936 GB/s$1,49939GLM 4.7 Flash
RX 7900 XT20 GB800 GB/s$89933DiffusionGemma 26B A4B IT
RTX 508016 GB960 GB/s$99932gpt-oss 20B
RTX 5070 Ti16 GB896 GB/s$74932gpt-oss 20B
RTX 4080 Super16 GB736 GB/s$99932gpt-oss 20B
RTX 408016 GB717 GB/s$1,19932gpt-oss 20B
RTX 4070 Ti Super16 GB672 GB/s$79932gpt-oss 20B
RX 907016 GB640 GB/s$54932gpt-oss 20B
RX 9070 XT16 GB640 GB/s$59932gpt-oss 20B
RX 7800 XT16 GB624 GB/s$49932gpt-oss 20B
RX 7900 GRE16 GB576 GB/s$54932gpt-oss 20B
Arc A770 16GB16 GB560 GB/s$34932gpt-oss 20B
RTX 5060 Ti 16GB16 GB448 GB/s$42932gpt-oss 20B
RTX 4060 Ti 16GB16 GB288 GB/s$49932gpt-oss 20B
RTX 3080 10GB10 GB760 GB/s$69929Qwen3.5 9B
RTX 507012 GB672 GB/s$54929Qwen3.5 9B
RTX 407012 GB504 GB/s$59929Qwen3.5 9B
RTX 4070 Super12 GB504 GB/s$59929Qwen3.5 9B
RTX 4070 Ti12 GB504 GB/s$79929Qwen3.5 9B
Arc B58012 GB456 GB/s$24929Qwen3.5 9B
RX 7700 XT12 GB432 GB/s$44929Qwen3.5 9B
RTX 3060 12GB12 GB360 GB/s$32929Qwen3.5 9B
RTX 50608 GB448 GB/s$29928Qwen3 VL 8B Instruct
RTX 3060 Ti8 GB448 GB/s$39928Qwen3 VL 8B Instruct
RTX 30708 GB448 GB/s$49928Qwen3 VL 8B Instruct
RX 76008 GB288 GB/s$26928Qwen3 VL 8B Instruct
RTX 4060 Ti 8GB8 GB288 GB/s$39928Qwen3 VL 8B Instruct
RTX 40608 GB272 GB/s$29928Qwen3 VL 8B Instruct

Popular comparisons: RTX 4060 Ti 16GB vs RTX 4060 Ti 8GBRTX 4060 Ti 16GB vs RTX 4070RTX 4070 Super vs RTX 4070 Ti SuperRTX 4080 vs RTX 4090RTX 5060 Ti 16GB vs RTX 5070RTX 5070 Ti vs RTX 5080RTX 5080 vs RTX 5090RTX 3090 vs RTX 3090 TiRTX 3060 12GB vs RTX 4060RTX 3060 12GB vs RTX 4060 Ti 16GB all

New to this? Start with how to run an LLM locally, then check a specific model with Can I run…? or the VRAM calculator.

Automated: model data from the Hugging Face Hub API, GPU specs and launch prices from vendor pages, formulas on the methodology page. No affiliate links. Report an error.