RunsOnMy ModelsGPUs & MacsAPI pricesVRAM calculatorNew

What AI models run on the RTX 4060?

The RTX 4060 has 8 GB of VRAM (272 GB/s). About 7.6 GB is usable for a model. 153 of 285 open models we track fit at 8K context.

153models fit (8K ctx)
112still fit at 32K ctx
16Blargest that fits
272 GB/smemory bandwidth

Models that fit, largest first

ModelParamsBest quant (8K)Needs32K contextMax tok/s (est.)
Kimi VL A3B Thinking 250616B (3.0B act.)Q2_K6.7 GBQ2_K242details
Kimi VL A3B Instruct16B (3.0B act.)Q2_K7.1 GBno226details
Kimi VL A3B Thinking16B (3.0B act.)Q2_K6.7 GBQ2_K242details
LLaDA2.1 mini16B (1.4B act.)Q2_K6.7 GBno506details
Moonlight 16B A3B Instruct16B (3.0B act.)Q2_K7.1 GBno220details
DeepSeek Coder V2 Lite Instruct16B (2.7B act.)Q2_K6.3 GBQ2_K273details
DeepSeek V2 Lite16B (2.7B act.)Q2_K6.8 GBQ2_K250details
DeepSeek V2 Lite Chat16B (2.7B act.)Q2_K6.8 GBQ2_K250details
Qwen2.5 Coder 14B Instruct15BQ2_K7.3 GBno49details
DeepSeek R1 Distill Qwen 14B15BQ2_K7.6 GBno47details
Qwen3 14B15BQ2_K7.1 GBno49details
phi 415BQ2_K7.4 GBno48details
gemma 3 12b it12BQ2_K6.2 GBno57details
Llama Guard 4 12B12BQ2_K6.5 GBno60details
Llama 3.2 11B Vision Instruct11BQ3_K_M6.8 GBno52details
Llama 3.2 11B Vision11BQ3_K_M6.7 GBno52details
GLM 4.6V Flash10BQ4_K_M6.7 GBQ3_K_M44details
GLM 4.1V 9B Thinking10BQ4_K_M6.7 GBQ3_K_M44details
Step3 VL 10B10BQ4_K_M6.5 GBno54details
SenseNova Loop 9B Preview9.7BQ4_K_M6.4 GBQ3_K_M48details
Qwen3.5 9B9.7BQ4_K_M6.4 GBQ3_K_M48details
Ornith 1.5 9B9.7BQ4_K_M6.6 GBQ3_K_M46details
Qwen3.5 9B Base9.7BQ4_K_M6.4 GBQ3_K_M48details
MiMo V2.6 Distill Qwen 9B9.4BQ4_K_M6.6 GBQ3_K_M47details
internlm3 8b instruct8.8BQ4_K_M6.0 GBQ3_K_M51details
granite 4.2 8b8.8BQ4_K_M6.8 GBno51details
granite 4.1 8b8.8BQ4_K_M6.8 GBno51details
Qwen3 VL 8B Instruct8.8BQ4_K_M6.5 GBno54details
Molmo2 8B8.7BQ4_K_M6.5 GBno54details
LFM2.5 8B A1B8.5B (1.0B act.)Q5_K_M6.4 GBQ5_K_M382details
Qwen2.5 VL 7B Instruct8.3BQ5_K_M6.2 GBQ4_K_M50details
UI TARS 1.5 7B8.3BQ5_K_M6.2 GBQ4_K_M50details
olmOCR 2 7B 10258.3BQ5_K_M6.2 GBQ4_K_M50details
olmOCR 7B 0225 preview8.3BQ5_K_M6.2 GBQ4_K_M50details
UI TARS 7B DPO8.3BQ5_K_M6.2 GBQ4_K_M50details
Qwen3 8B8.2BQ4_K_M6.5 GBno54details
DeepSeek R1 0528 Qwen3 8B8.2BQ4_K_M6.5 GBno54details
granite 3.3 8b instruct8.2BQ4_K_M6.3 GBno55details
granite 3.1 8b instruct8.2BQ4_K_M6.3 GBno55details
granite 3.2 8b instruct8.2BQ4_K_M6.3 GBno55details
Apertus 8B Instruct 25098.1BQ4_K_M6.4 GBno54details
Llama 3.1 8B Instruct8.0BQ4_K_M6.2 GBno55details
Llama 3.1 8B8.0BQ4_K_M6.2 GBQ2_K55details
Hermes 3 Llama 3.1 8B8.0BQ4_K_M6.2 GBno55details
DeepSeek R1 Distill Llama 8B8.0BQ4_K_M6.2 GBno55details
Meta Llama 3 8B Instruct8.0BQ4_K_M6.2 GBno55details
Meta Llama Guard 2 8B8.0BQ4_K_M6.2 GBno55details
Meta Llama 3.1 8B Instruct8.0BQ4_K_M6.2 GBQ2_K55details
Ling 3.0 tiny7.9B (1.4B act.)Q5_K_M6.2 GBQ4_K_M272details
MiMo 7B RL7.8BQ4_K_M6.2 GBno58details
internlm2 5 7b chat7.7BQ5_K_M6.7 GBQ2_K49details
internlm2 chat 7b7.7BQ5_K_M6.7 GBQ2_K49details
Qwen2.5 7B Instruct7.6BQ5_K_M6.2 GBQ4_K_M50details
Qwen2.5 Coder 7B Instruct7.6BQ5_K_M6.2 GBQ4_K_M50details
DeepSeek R1 Distill Qwen 7B7.6BQ5_K_M6.2 GBQ4_K_M50details
Falcon H1R 7B7.6BQ6_K6.8 GBQ4_K_M44details
Olmo 3 7B Instruct7.3BQ3_K_M6.5 GBno74details
Olmo 3 1025 7B7.3BQ3_K_M6.4 GBno76details
Olmo 3 7B Think7.3BQ3_K_M6.5 GBno74details
Mistral 7B v0.17.2BQ5_K_M6.2 GBQ2_K53details
Mistral 7B Instruct v0.27.2BQ5_K_M6.3 GBno53details
Mistral 7B Instruct v0.17.2BQ5_K_M6.2 GBQ2_K53details
granite 4.0 h tiny6.9B (1.5B act.)Q6_K6.0 GBQ6_K226details
OLMoE 1B 7B 09246.9B (1.3B act.)Q6_K6.8 GBQ2_K258details
deepseek coder 7b instruct v1.56.9BQ3_K_M7.6 GBno79details
deepseek coder 6.7b instruct6.7BQ3_K_M7.5 GBno82details
deepseek coder 6.7b base6.7BQ2_K6.8 GBno107details
Llama 2 7b hf6.7BQ2_K6.8 GBno107details
Yi 6B 200K6.1BQ6_K5.7 GBQ5_K_M55details
Yi 6B6.1BQ6_K5.7 GBQ5_K_M55details
Molmo2 4B4.9BQ8_06.2 GBQ3_K_M58details
WeVisOne VL 4B4.8BQ8_06.2 GBQ3_K_M58details
Mage VL Cowork4.7BQ8_06.2 GBQ3_K_M58details
Qianfan OCR4.7BQ8_06.2 GBQ3_K_M58details
Qwen3.5 4B4.7BQ8_05.3 GBQ8_061details
S5 nano4.7BQ8_05.4 GBQ8_059details
LightOnOCR 3 4B4.5BQ8_05.3 GBQ8_061details
SingVL S1 4B4.5BQ8_05.3 GBQ8_061details
Intern Decision 4B4.5BQ8_05.3 GBQ8_061details
Qwen3 VL 4B Instruct4.4BQ8_05.8 GBQ2_K64details
gemma 3 4b it4.3BQ8_05.0 GBQ8_066details
medgemma 1.5 4b it4.3BQ8_05.0 GBQ8_066details
medgemma 4b it4.3BQ8_05.0 GBQ8_066details
gemma 3 4b pt4.3BQ8_05.4 GBQ8_060details
Phi 3.5 vision instruct4.1BQ6_K6.6 GBno80details
Qwen3 4B4.0BQ8_05.8 GBQ2_K64details
Qwen3 4B Instruct 25074.0BQ8_05.8 GBQ2_K64details
granite vision 4.1 4b4.0BQ8_04.6 GBQ8_075details
Phi 4 mini instruct3.8BQ8_05.6 GBQ3_K_M67details
Phi 3 mini 4k instruct3.8BQ6_K6.4 GBno87details
Phi 3.5 mini instruct3.8BQ6_K6.4 GBno87details
Phi 3 mini 128k instruct3.8BQ6_K6.4 GBno87details
Qwen2.5 VL 3B Instruct3.8BF166.7 GBQ8_044details
granite 4.2 3b3.7BQ8_04.8 GBQ8_070details
granite 4.1 3b llmon preview3.7BQ8_04.8 GBQ8_070details
Raon 1 nano3.5BQ8_04.3 GBQ8_080details
granite 4.1 3b3.4BQ8_04.6 GBQ8_075details
granite 4.0 micro3.4BQ8_04.6 GBQ8_075details
DeepSeek OCR 23.4B (1.2B act.)F166.6 GBQ8_0131details
deepseek vl2 tiny3.4B (1.2B act.)Q8_04.5 GBQ8_0218details
Unlimited OCR 23.3B (1.1B act.)F166.6 GBQ8_0135details
Unlimited OCR3.3B (1.1B act.)F166.6 GBQ8_0135details
DeepSeek OCR3.3B (1.1B act.)F166.6 GBQ8_0135details
granite 3.1 3b a800m instruct3.3B (883M act.)Q8_04.3 GBQ8_0289details
Llama 3.2 3B Instruct3.2BQ8_04.7 GBQ6_K79details
Llama 3.2 3B3.2BQ8_05.1 GBQ5_K_M71details
Hermes 3 Llama 3.2 3B3.2BQ8_04.7 GBQ6_K79details
d1 3B3.1BQ8_03.4 GBQ8_095details
LFM2.5 VL 3B3.1BQ8_03.4 GBQ8_095details
Qwen2.5 3B Instruct3.1BF166.7 GBQ8_044details
SmolLM3 3B3.1BQ8_04.3 GBQ8_083details
SmolLM3 3B Base3.1BQ8_04.3 GBQ8_083details
phi 22.8BQ8_05.8 GBno92details
LFM2.5 2.6B2.7BQ8_03.4 GBQ8_095details
granite 3.3 2b instruct2.5BF165.8 GBQ8_054details
MiniCPM5 2B2.5BF165.7 GBF1654details
WeVisOne VL 2B2.4BF165.4 GBQ8_067details
Qwen3.5 2B2.3BF164.5 GBF1672details
SmolVLM Instruct2.2BF165.4 GBQ4_K_M75details
Qwen3 1.7B2.0BF164.8 GBQ8_079details
Qwen3.8 27B DSpine b162.0BF164.8 GBF1667details
d1 2B1.9BQ8_02.5 GBQ8_0147details
LFM3 VL 2B1.9BQ8_02.5 GBQ8_0147details
DeepSeek R1 Distill Qwen 1.5B1.8BF164.2 GBF1676details
LFM3 1.7B1.7BQ8_02.5 GBQ8_0147details
LFM3 1.7B Base1.7BF164.0 GBF1679details
SmolLM2 1.7B Instruct1.7BF165.2 GBQ4_K_M79details
SmolLM2 1.7B1.7BF165.2 GBQ4_K_M79details
SmolLM 1.7B1.7BF165.2 GBQ4_K_M79details
granite 4.0 1b1.6BF164.3 GBF1683details
Qwen2.5 1.5B Instruct1.5BF163.8 GBF1688details
OLMo 2 0425 1B1.5BF164.4 GBQ8_091details
deepseek coder 1.3b instruct1.3BF164.5 GBQ6_K101details
Llama 3.2 1B1.2BF163.2 GBF16110details
Llama 3.2 1B Instruct1.2BF165.5 GBF1655details
LFM2.5 1.2B Instruct1.2BF165.0 GBF1658details
HunyuanOCR1.1BF163.5 GBF16121details
PaddleOCR VL 1.6959MF162.5 GBF16142details
Qwen3.5 0.8B873MF162.4 GBF16179details
LightOnOCR 3 0.8B853MF162.4 GBF16179details
FastVLM 0.5B759MF162.2 GBF16179details
Qwen3 0.6B752MF162.7 GBF16227details
GOT OCR2 0716MF162.8 GBF16190details
Nemotron 3.5 Parse631MF161.9 GBF16208details
GOT OCR 2.0 hf561MF162.5 GBF16243details
SmolVLM2 500M Video Instruct507MF161.6 GBF16332details
SmolVLM 500M Instruct507MF161.6 GBF16332details
Qwen2.5 0.5B Instruct494MF161.7 GBF16275details
LFM2.5 VL 450M449MF161.9 GBF16191details
SmolLM2 360M Instruct362MF161.5 GBF16375details
SmolLM2 360M362MF161.5 GBF16375details
LFM2.5 350M354MF161.9 GBF16191details
LFM2 350M354MF161.4 GBF16384details

Just too big at 4-bit

These need up to 35% more memory than the RTX 4060 has at Q4_K_M and 8K context. A smaller quant or shorter context may work.

ModelQ4_K_M needsShort by
Llama 3.2 11B Vision8.0 GB0.4 GB
Llama 2 7b hf8.3 GB0.7 GB
deepseek coder 6.7b base8.3 GB0.7 GB
deepseek coder 6.7b instruct8.3 GB0.7 GB
deepseek coder 7b instruct v1.58.3 GB0.7 GB
gemma 3 12b it8.5 GB0.9 GB
Llama Guard 4 12B9.1 GB1.5 GB
falcon 7b instruct9.4 GB1.8 GB
falcon 7b9.6 GB2.0 GB
DeepSeek V2 Lite9.8 GB2.2 GB

Questions

What is the biggest model that runs on the RTX 4060?

Of the models we track, the largest that fits at 8K context is Kimi VL A3B Thinking 2506 (16B) at Q2_K, needing about 6.7 GB.

How much of the RTX 4060's memory can a model use?

About 7.6 GB: we assume 95% of the 8 GB is usable after driver and display overhead.

Spec source: NVIDIA specifications. Model data: Hugging Face Hub API. Speed is a bandwidth ceiling (bandwidth ÷ bytes read per token), not a benchmark. Methodology · Report an error.