RunsOnMy ModelsGPUs & MacsAPI pricesVRAM calculatorNew

GPUs and Macs for running AI models locally

Pick your hardware to see every open model that fits, the best quant, and an estimated speed ceiling.

Consumer GPUs

HardwareMemoryUsableBandwidthModels that fit (8K)
RTX 3060 12GB12 GB11.4 GB360 GB/s171
RTX 3060 Ti8 GB7.6 GB448 GB/s153
RTX 30708 GB7.6 GB448 GB/s153
RTX 3080 10GB10 GB9.5 GB760 GB/s161
RTX 3080 12GB12 GB11.4 GB912 GB/s171
RTX 309024 GB22.8 GB936 GB/s202
RTX 3090 Ti24 GB22.8 GB1008 GB/s202
RTX 40608 GB7.6 GB272 GB/s153
RTX 4060 Ti 8GB8 GB7.6 GB288 GB/s153
RTX 4060 Ti 16GB16 GB15.2 GB288 GB/s194
RTX 407012 GB11.4 GB504 GB/s171
RTX 4070 Super12 GB11.4 GB504 GB/s171
RTX 4070 Ti12 GB11.4 GB504 GB/s171
RTX 4070 Ti Super16 GB15.2 GB672 GB/s194
RTX 408016 GB15.2 GB717 GB/s194
RTX 4080 Super16 GB15.2 GB736 GB/s194
RTX 409024 GB22.8 GB1008 GB/s202
RTX 50608 GB7.6 GB448 GB/s153
RTX 5060 Ti 16GB16 GB15.2 GB448 GB/s194
RTX 507012 GB11.4 GB672 GB/s171
RTX 5070 Ti16 GB15.2 GB896 GB/s194
RTX 508016 GB15.2 GB960 GB/s194
RTX 509032 GB30.4 GB1792 GB/s207
RX 76008 GB7.6 GB288 GB/s153
RX 7700 XT12 GB11.4 GB432 GB/s171
RX 7800 XT16 GB15.2 GB624 GB/s194
RX 7900 GRE16 GB15.2 GB576 GB/s194
RX 7900 XT20 GB19.0 GB800 GB/s202
RX 7900 XTX24 GB22.8 GB960 GB/s202
RX 907016 GB15.2 GB640 GB/s194
RX 9070 XT16 GB15.2 GB640 GB/s194
Arc B58012 GB11.4 GB456 GB/s171
Arc A770 16GB16 GB15.2 GB560 GB/s194

Multi-GPU rigs

HardwareMemoryUsableBandwidthModels that fit (8K)
2x RTX 30902×24 GB45.6 GB936 GB/s214
2x RTX 40902×24 GB45.6 GB1008 GB/s214
2x RTX 50902×32 GB60.8 GB1792 GB/s217

Apple Silicon Macs

HardwareMemoryUsableBandwidthModels that fit (8K)
Mac M1 16GB16 GB10.7 GB68 GB/s165
Mac M2 24GB24 GB16.1 GB100 GB/s199
Mac M3 24GB24 GB16.1 GB100 GB/s199
Mac M4 16GB16 GB10.7 GB120 GB/s165
Mac M4 32GB32 GB21.4 GB120 GB/s202
Mac M1 Pro 32GB32 GB21.4 GB200 GB/s202
Mac M3 Pro 36GB36 GB24.1 GB150 GB/s202
Mac M4 Pro 24GB24 GB16.1 GB273 GB/s199
Mac M4 Pro 48GB48 GB32.2 GB273 GB/s207
Mac M4 Pro 64GB64 GB48.0 GB273 GB/s215
Mac M1 Max 64GB64 GB48.0 GB400 GB/s215
Mac M2 Max 96GB96 GB72.0 GB400 GB/s219
Mac M3 Max 128GB128 GB96.0 GB400 GB/s227
Mac M4 Max 64GB64 GB48.0 GB546 GB/s215
Mac M4 Max 128GB128 GB96.0 GB546 GB/s227
Mac M2 Ultra 192GB192 GB144 GB800 GB/s245
Mac M3 Ultra 256GB256 GB192 GB819 GB/s248
Mac M3 Ultra 512GB512 GB384 GB819 GB/s278

Unified-memory PCs

HardwareMemoryUsableBandwidthModels that fit (8K)
DGX Spark128 GB109 GB273 GB/s235
Ryzen AI Max+ 395 128GB128 GB109 GB256 GB/s235

Workstation GPUs

HardwareMemoryUsableBandwidthModels that fit (8K)
RTX A600048 GB45.6 GB768 GB/s214
RTX 6000 Ada48 GB45.6 GB960 GB/s214
RTX PRO 6000 Blackwell96 GB91.2 GB1792 GB/s227

Data-center GPUs

HardwareMemoryUsableBandwidthModels that fit (8K)
L40S48 GB45.6 GB864 GB/s214
A100 40GB40 GB38.0 GB1555 GB/s207
A100 80GB80 GB76.0 GB2039 GB/s220
H100 PCIe80 GB76.0 GB2000 GB/s220
H100 SXM80 GB76.0 GB3350 GB/s220
H200141 GB134 GB4800 GB/s243