RunsOnMy ModelsGPUs & MacsBy VRAMBest GPUCan I run…?OllamaStart hereAPI pricesVRAM calculatorNew

How to run an LLM locally

Running a large language model on your own computer means no subscription, no data leaving your machine, and no rate limits. You need three things: enough GPU memory for the model, a program to run it, and the model file. This guide covers all three, with numbers from our database of 298 open models.

1. What hardware you need

The single number that matters most is memory the GPU can use: VRAM on a graphics card, or unified memory on a Mac. A model has to fit in it to run at full speed. Speed then depends on memory bandwidth. Here is how much of the 60 most-downloaded open models fits at each size:

GPU memoryTop-60 models that fit (4-bit)…at near-lossless 8-bitLargest popular model that fits
8 GB29 of 6021Qwen3 VL 8B Instruct (8.8B)
12 GB31 of 6029Qwen3.5 9B (9.7B)
16 GB32 of 6030gpt-oss 20B (21B)
24 GB43 of 6032GLM 4.7 Flash (31B)
32 GB43 of 6037Qwen3.6 35B A3B (36B)
48 GB43 of 6043Qwen3.6 35B A3B (36B)

8K context, real GGUF file sizes. From the Local LLM Hardware Index (2026-11-17).

On a Mac, about two-thirds to three-quarters of unified memory is available to the GPU by default:

MacTop-60 models that fitLargest that fits
Mac M4 16GB29 of 60Qwen3.5 9B
Mac M4 Pro 24GB32 of 60gpt-oss 20B
Mac M4 Pro 48GB43 of 60Qwen3.6 35B A3B
Mac M4 Max 128GB45 of 60gpt-oss 120B

No GPU? Small models (1–4B) run on a CPU alone, slowly. A laptop with 16 GB of RAM can manage them; anything bigger really wants a GPU or Apple Silicon.

Local AI hardware: building a PC for local AI

2. Which tool to use: Ollama vs LM Studio vs llama.cpp

OllamaLM Studiollama.cpp
What it isBackground service plus a simple command line and appDesktop app with a model browser and chat windowThe engine itself: command-line tools and a server
Best forGetting started fast; powering other apps through its APIBrowsing and trying models without a terminalFull control: offload splits, context and cache settings
ModelsOllama library (one command per model)GGUF from Hugging Face; MLX on MacAny GGUF file
Runs onmacOS, Windows, LinuxmacOS (Apple Silicon), Windows, LinuxEverything, including CPU only
Our guide70 Ollama models with sizes and VRAMWhich models fit, quants explainedOffload calculator and commands

All three use the same model format (GGUF) and similar code underneath, so memory needs are about the same whichever you pick. If you're unsure, start with Ollama or LM Studio.

3. Run your first model

  1. Check what fits. Look up your card on the GPU list, or use the VRAM calculator.
  2. Pick the largest model that fits fully, at 4-bit (Q4_K_M) to begin with. A bigger model at 4-bit usually beats a smaller one at 8-bit.
  3. Install and run. With Ollama that is one command, for example ollama run qwen3:8b (6.7 GB at 4K context, fine on an 8 GB card). In LM Studio, search for the model and pick a file marked for full GPU offload.

Run AI locally or use an API?

Local is private, works offline and costs nothing per token once you own the hardware. Hosted APIs run the largest models and need no hardware. Many people do both. Compare costs on our API price tracker, which lists the same models where they're available.

Questions

Can I run an LLM locally with 8 GB of VRAM?

Yes. 29 of the 60 most popular open models fit in 8 GB at 4-bit, including 7–9B chat models. See every model that fits in 8 GB.

Is a Mac good for local LLMs?

For big models, yes: unified memory lets a 64–128 GB Mac hold models no gaming card can, though more slowly than a high-end GPU. See Mac vs PC for local LLMs.

Numbers on this page are computed automatically from the Hugging Face Hub API, GGUF file sizes and GPU specs, using the formulas on the methodology page. Text drafted with AI assistance; see how this site is made.