What AI models run on the H200?
The H200 has 141 GB of VRAM (4,800 GB/s). About 134 GB is usable for a model. 243 of 285 open models we track fit at 8K context.
243models fit (8K ctx)
241still fit at 32K ctx
358Blargest that fits
4800 GB/smemory bandwidth
Models that fit, largest first
| Model | Params | Best quant (8K) | Needs | 32K context | Max tok/s (est.) | |
|---|---|---|---|---|---|---|
| GLM 4.5 | 358B (36B act.) | Q2_K | 126 GB | no | 369 | details |
| GLM 4.7 | 358B (36B act.) | Q2_K | 126 GB | no | 369 | details |
| GLM 4.6 | 357B (34B act.) | Q2_K | 125 GB | Q2_K | 385 | details |
| GLM 5.3 Flash | 321B (18B act.) | Q2_K | 113 GB | Q2_K | 698 | details |
| Nex N2.6 | 321B (18B act.) | Q2_K | 118 GB | Q2_K | 670 | details |
| MiMo V2.6 Flash RL | 311B (6.2B act.) | Q2_K | 110 GB | Q2_K | — | details |
| MiMo V2.5 | 311B (17B act.) | Q2_K | 110 GB | Q2_K | 735 | details |
| MiMo V2.6 Flash MOPD | 311B (6.2B act.) | Q2_K | 110 GB | Q2_K | — | details |
| MiMo V2 Flash | 310B (16B act.) | Q2_K | 106 GB | Q2_K | 806 | details |
| DeepSeek V4 Flash Vision Exp | 305B (15B act.) | Q2_K | 108 GB | Q2_K | 840 | details |
| DeepSeek V4 Flash 0731 | 304B (27B act.) | Q2_K | 108 GB | Q2_K | 468 | details |
| DeepSeek V4 Flash DSpark | 304B (27B act.) | Q2_K | 108 GB | Q2_K | 468 | details |
| Hy3 | 299B (21B act.) | Q2_K | 102 GB | Q2_K | 641 | details |
| Hy3 preview | 299B (21B act.) | Q2_K | 107 GB | Q2_K | 611 | details |
| DeepSeek V4 Flash | 291B (14B act.) | Q2_K | 103 GB | Q2_K | 906 | details |
| ERNIE 5.5 Flash Base | 286B (5.7B act.) | Q2_K | 101 GB | Q2_K | — | details |
| Intern S1 | 241B (28B act.) | Q3_K_M | 112 GB | Q3_K_M | 353 | details |
| DeepSeek V2 Chat | 236B (21B act.) | Q3_K_M | 108 GB | Q3_K_M | 459 | details |
| MiniMax M2.5 | 229B (11B act.) | Q3_K_M | 104 GB | Q3_K_M | 911 | details |
| MiniMax M2.7 | 229B (11B act.) | Q3_K_M | 97.0 GB | Q3_K_M | 983 | details |
| MiniMax M2 | 229B (11B act.) | Q3_K_M | 104 GB | Q3_K_M | 911 | details |
| MiniMax M2.1 | 229B (11B act.) | Q3_K_M | 104 GB | Q3_K_M | 911 | details |
| MiniMax M3.1 Flash | 220B (4.4B act.) | Q3_K_M | 102 GB | Q3_K_M | — | details |
| Step 3.7 Flash | 201B | Q4_K_M | 119 GB | Q3_K_M | 39 | details |
| Step 3.5 Flash | 199B | Q4_K_M | 116 GB | Q3_K_M | 40 | details |
| Qwen3.8 Flash Next | 180B (62B act.) | Q5_K_M | 121 GB | Q4_K_M | 109 | details |
| Ling 3.0 flash Sante | 127B (5.6B act.) | Q6_K | 105 GB | Q6_K | 980 | details |
| Ling 3.0 flash | 127B (5.6B act.) | Q6_K | 103 GB | Q6_K | 999 | details |
| Ornith 1.5 122B | 125B (13B act.) | Q6_K | 96.6 GB | Q6_K | 459 | details |
| gpt oss 120b | 117B (5.8B act.) | F16 | 61.9 GB | F16 | 1492 | details |
| gpt oss safeguard 120b | 117B (5.8B act.) | Q8_0 | 60.1 GB | Q8_0 | 1538 | details |
| GLM 4.5 Air | 110B (15B act.) | Q8_0 | 112 GB | Q8_0 | 302 | details |
| Llama 4 Scout 17B 16E Instruct | 109B (18B act.) | Q8_0 | 109 GB | Q8_0 | 252 | details |
| Llama 4 Scout 17B 16E | 109B (18B act.) | Q8_0 | 110 GB | Q8_0 | 250 | details |
| GLM 4.5V | 108B (14B act.) | Q8_0 | 108 GB | Q8_0 | 319 | details |
| GLM 4.6V | 108B (14B act.) | Q8_0 | 108 GB | Q8_0 | 319 | details |
| Hunyuan A13B Instruct | 80B (13B act.) | Q8_0 | 81.2 GB | Q8_0 | 347 | details |
| Kimi Dev 72B | 73B | Q8_0 | 75.1 GB | Q8_0 | 62 | details |
| Llama 3.3 70B Instruct | 71B | Q8_0 | 73.0 GB | Q8_0 | 64 | details |
| DeepSeek R1 Distill Llama 70B | 71B | Q8_0 | 73.0 GB | Q8_0 | 64 | details |
| Llama 3.1 70B | 71B | Q8_0 | 73.0 GB | Q8_0 | 64 | details |
| Kimi Linear 48B A3B Instruct | 49B (3.0B act.) | F16 | 92.5 GB | F16 | 800 | details |
| Nous Hermes 2 Mixtral 8x7B DPO | 47B (13B act.) | F16 | 88.5 GB | F16 | 186 | details |
| falcon 40b | 42B | F16 | 79.4 GB | F16 | 57 | details |
| flint | 40B (1.3B act.) | F16 | 75.1 GB | F16 | 1920 | details |
| Maxwell V1 Flash | 37B (2.3B act.) | F16 | 69.7 GB | F16 | 1057 | details |
| Hermes 4.3 36B | 36B | F16 | 70.0 GB | F16 | 66 | details |
| Seed OSS 36B Instruct | 36B | F16 | 70.0 GB | F16 | 66 | details |
| Qwen3.6 35B A3B | 36B (3.0B act.) | F16 | 67.2 GB | F16 | 809 | details |
| Ornith 1.5 35B A3B | 36B (3.0B act.) | F16 | 67.2 GB | F16 | 809 | details |
| Qwen3.5 35B A3B | 36B (3.0B act.) | F16 | 65.6 GB | F16 | 829 | details |
| Axion 1.0 35B | 35B (3.9B act.) | F16 | 65.6 GB | F16 | 623 | details |
| Qwen AgentWorld 35B A3B | 35B (3.0B act.) | F16 | 65.6 GB | F16 | 799 | details |
| Yi 34B 200K | 34B | F16 | 66.5 GB | F16 | 70 | details |
| Yi 34B | 34B | F16 | 66.5 GB | F16 | 70 | details |
| Yi 34B Chat | 34B | F16 | 66.5 GB | F16 | 70 | details |
| Yi 1.5 34B Chat | 34B | F16 | 66.5 GB | F16 | 70 | details |
| Nous Hermes 2 Yi 34B | 34B | F16 | 66.5 GB | F16 | 70 | details |
| DeepSeek R1 Distill Qwen 32B | 33B | F16 | 63.7 GB | F16 | 73 | details |
| Qwen3 32B | 33B | F16 | 63.7 GB | F16 | 73 | details |
| GLM Z1 32B 0414 | 33B | F16 | 61.8 GB | F16 | 74 | details |
| GLM 4 32B 0414 | 33B | F16 | 61.8 GB | F16 | 74 | details |
| granite 4.0 h small | 32B (8.8B act.) | F16 | 60.7 GB | F16 | 272 | details |
| gemma 4 31B it | 31B | F16 | 60.1 GB | F16 | 78 | details |
| GLM 4.7 Flash | 31B (4.6B act.) | F16 | 56.9 GB | F16 | 543 | details |
| Qwen3 Coder 30B A3B Instruct | 31B (3.0B act.) | F16 | 58.3 GB | F16 | 800 | details |
| North Mini Code 1.0 | 30B (3.3B act.) | F16 | 58.2 GB | F16 | 726 | details |
| Hy Memory 30B A3B | 30B (3.0B act.) | F16 | 57.4 GB | F16 | 800 | details |
| Muse Glimmer 30B | 30B | F16 | 52.9 GB | F16 | 86 | details |
| granite 4.2 30b | 29B | F16 | 57.1 GB | F16 | 82 | details |
| Qwen3.8 27B | 28B | F16 | 52.3 GB | F16 | 88 | details |
| Qwen3.6 27B | 28B | F16 | 51.5 GB | F16 | 89 | details |
| Qwen3.5 27B | 28B | F16 | 51.5 GB | F16 | 89 | details |
| gemma 3 27b it | 27B | F16 | 52.2 GB | F16 | 89 | details |
| Arete 27B | 27B | F16 | 51.5 GB | F16 | 89 | details |
| MiniMax H3 Image PE | 27B | F16 | 52.3 GB | F16 | 88 | details |
| Trinity Mini | 26B (2.0B act.) | F16 | 49.7 GB | F16 | 1222 | details |
| GLiDE 27B | 26B | F16 | 50.0 GB | F16 | 92 | details |
| diffusiongemma 26B A4B it | 26B (4.0B act.) | F16 | 48.5 GB | F16 | 613 | details |
| gemma 4 26B A4B it | 26B (4.0B act.) | F16 | 48.4 GB | F16 | 613 | details |
| LFM2.5 24B A2B | 24B (2.0B act.) | F16 | 90.1 GB | F16 | 600 | details |
| ERNIE 4.5 21B A3B PT | 22B | F16 | 41.7 GB | F16 | 110 | details |
| ERNIE 4.5 21B A3B Thinking | 22B | F16 | 41.7 GB | F16 | 110 | details |
| gpt oss safeguard 20b | 22B (4.8B act.) | F16 | 13.8 GB | F16 | 1563 | details |
| gpt oss 20b | 21B (4.2B act.) | F16 | 13.8 GB | F16 | 1736 | details |
| Groot 19B A3B 261023 | 19B (3.0B act.) | F16 | 36.4 GB | F16 | 818 | details |
| Kimi VL A3B Thinking 2506 | 16B (3.0B act.) | F16 | 60.4 GB | F16 | 411 | details |
| Kimi VL A3B Instruct | 16B (3.0B act.) | F16 | 31.5 GB | F16 | 800 | details |
| Kimi VL A3B Thinking | 16B (3.0B act.) | F16 | 31.5 GB | F16 | 800 | details |
| deepseek moe 16b base | 16B (2.8B act.) | F16 | 32.8 GB | F16 | 848 | details |
| LLaDA2.1 mini | 16B (1.4B act.) | F16 | 31.3 GB | F16 | 1675 | details |
| Moonlight 16B A3B Instruct | 16B (3.0B act.) | F16 | 30.7 GB | F16 | 800 | details |
| DeepSeek Coder V2 Lite Instruct | 16B (2.7B act.) | F16 | 30.1 GB | F16 | 902 | details |
| DeepSeek V2 Lite | 16B (2.7B act.) | F16 | 30.1 GB | F16 | 902 | details |
| DeepSeek V2 Lite Chat | 16B (2.7B act.) | F16 | 30.1 GB | F16 | 902 | details |
| Qwen2.5 Coder 14B Instruct | 15B | F16 | 29.7 GB | F16 | 162 | details |
| DeepSeek R1 Distill Qwen 14B | 15B | F16 | 29.7 GB | F16 | 162 | details |
| Qwen3 14B | 15B | F16 | 29.4 GB | F16 | 163 | details |
| phi 4 | 15B | F16 | 29.5 GB | F16 | 164 | details |
| gemma 3 12b it | 12B | F16 | 23.6 GB | F16 | 204 | details |
| Llama Guard 4 12B | 12B | F16 | 24.6 GB | F16 | 200 | details |
| Llama 3.2 11B Vision Instruct | 11B | F16 | 20.1 GB | F16 | 245 | details |
| Llama 3.2 11B Vision | 11B | F16 | 21.7 GB | F16 | 226 | details |
| GLM 4.6V Flash | 10B | F16 | 18.5 GB | F16 | 255 | details |
| GLM 4.1V 9B Thinking | 10B | F16 | 18.5 GB | F16 | 255 | details |
| Step3 VL 10B | 10B | F16 | 32.3 GB | F16 | 146 | details |
| SenseNova Loop 9B Preview | 9.7B | F16 | 17.8 GB | F16 | 268 | details |
| Qwen3.5 9B | 9.7B | F16 | 17.8 GB | F16 | 268 | details |
| Ornith 1.5 9B | 9.7B | F16 | 17.8 GB | F16 | 268 | details |
| Qwen3.5 9B Base | 9.7B | F16 | 17.8 GB | F16 | 268 | details |
| MiMo V2.6 Distill Qwen 9B | 9.4B | F16 | 17.8 GB | F16 | 268 | details |
| glm 4 9b | 9.4B | F16 | 23.2 GB | F16 | 255 | details |
| internlm3 8b instruct | 8.8B | F16 | 17.4 GB | F16 | 273 | details |
| granite 4.2 8b | 8.8B | F16 | 18.2 GB | F16 | 273 | details |
| granite 4.1 8b | 8.8B | F16 | 18.2 GB | F16 | 273 | details |
| Qwen3 VL 8B Instruct | 8.8B | F16 | 17.1 GB | F16 | 293 | details |
| Molmo2 8B | 8.7B | F16 | 17.1 GB | F16 | 293 | details |
| LFM2.5 8B A1B | 8.5B (1.0B act.) | F16 | 32.3 GB | F16 | 1199 | details |
| Qwen2.5 VL 7B Instruct | 8.3B | F16 | 15.3 GB | F16 | 315 | details |
| UI TARS 1.5 7B | 8.3B | F16 | 15.3 GB | F16 | 315 | details |
| olmOCR 2 7B 1025 | 8.3B | F16 | 15.3 GB | F16 | 315 | details |
| olmOCR 7B 0225 preview | 8.3B | F16 | 16.6 GB | F16 | 289 | details |
| UI TARS 7B DPO | 8.3B | F16 | 15.3 GB | F16 | 315 | details |
| Qwen3 8B | 8.2B | F16 | 17.1 GB | F16 | 293 | details |
| DeepSeek R1 0528 Qwen3 8B | 8.2B | F16 | 17.1 GB | F16 | 293 | details |
| granite 3.3 8b instruct | 8.2B | F16 | 17.0 GB | F16 | 294 | details |
| granite 3.1 8b instruct | 8.2B | F16 | 17.0 GB | F16 | 294 | details |
| granite 3.2 8b instruct | 8.2B | F16 | 17.0 GB | F16 | 294 | details |
| Apertus 8B Instruct 2509 | 8.1B | F16 | 16.7 GB | F16 | 298 | details |
| Llama 3.1 8B Instruct | 8.0B | F16 | 16.6 GB | F16 | 299 | details |
| Llama 3.1 8B | 8.0B | F16 | 16.6 GB | F16 | 299 | details |
| Hermes 3 Llama 3.1 8B | 8.0B | F16 | 16.6 GB | F16 | 299 | details |
| DeepSeek R1 Distill Llama 8B | 8.0B | F16 | 31.6 GB | F16 | 149 | details |
| Meta Llama 3 8B Instruct | 8.0B | F16 | 16.6 GB | F16 | 299 | details |
| Meta Llama Guard 2 8B | 8.0B | F16 | 16.6 GB | F16 | 299 | details |
| Meta Llama 3.1 8B Instruct | 8.0B | F16 | 16.6 GB | F16 | 299 | details |
| Ling 3.0 tiny | 7.9B (1.4B act.) | F16 | 15.6 GB | F16 | 1735 | details |
| MiMo 7B RL | 7.8B | F16 | 16.4 GB | F16 | 306 | details |
| internlm2 5 7b chat | 7.7B | F16 | 16.0 GB | F16 | 310 | details |
| internlm2 chat 7b | 7.7B | F16 | 16.0 GB | F16 | 310 | details |
| Qwen2.5 7B Instruct | 7.6B | F16 | 15.3 GB | F16 | 315 | details |
| Qwen2.5 Coder 7B Instruct | 7.6B | F16 | 15.3 GB | F16 | 315 | details |
| DeepSeek R1 Distill Qwen 7B | 7.6B | F16 | 15.3 GB | F16 | 315 | details |
| Falcon H1R 7B | 7.6B | F16 | 15.1 GB | F16 | 316 | details |
| Olmo 3 7B Instruct | 7.3B | F16 | 16.7 GB | F16 | 329 | details |
| Olmo 3 1025 7B | 7.3B | F16 | 16.7 GB | F16 | 329 | details |
| Olmo 3 7B Think | 7.3B | F16 | 16.7 GB | F16 | 329 | details |
| Mistral 7B v0.1 | 7.2B | F16 | 14.9 GB | F16 | 331 | details |
| Mistral 7B Instruct v0.2 | 7.2B | F16 | 14.9 GB | F16 | 331 | details |
| Mistral 7B Instruct v0.1 | 7.2B | F16 | 14.9 GB | F16 | 331 | details |
| falcon 7b | 7.2B | F16 | 18.4 GB | F16 | 333 | details |
| falcon 7b instruct | 7.2B | F16 | 18.4 GB | F16 | 333 | details |
| granite 4.0 h tiny | 6.9B (1.5B act.) | F16 | 13.6 GB | F16 | 1636 | details |
| OLMoE 1B 7B 0924 | 6.9B (1.3B act.) | F16 | 14.4 GB | F16 | 1871 | details |
| deepseek coder 7b instruct v1.5 | 6.9B | F16 | 17.2 GB | F16 | 347 | details |
| deepseek coder 6.7b instruct | 6.7B | F16 | 17.0 GB | F16 | 356 | details |
| deepseek coder 6.7b base | 6.7B | F16 | 17.0 GB | F16 | 356 | details |
| Llama 2 7b hf | 6.7B | F16 | 17.0 GB | F16 | 356 | details |
| Yi 6B 200K | 6.1B | F16 | 12.3 GB | F16 | 396 | details |
| Yi 6B | 6.1B | F16 | 12.3 GB | F16 | 396 | details |
| Molmo2 4B | 4.9B | F16 | 10.0 GB | F16 | 544 | details |
| WeVisOne VL 4B | 4.8B | F16 | 10.0 GB | F16 | 543 | details |
| Mage VL Cowork | 4.7B | F16 | 10.0 GB | F16 | 544 | details |
| Qianfan OCR | 4.7B | F16 | 10.1 GB | F16 | 543 | details |
| Qwen3.5 4B | 4.7B | F16 | 9.0 GB | F16 | 570 | details |
| S5 nano | 4.7B | F16 | 9.2 GB | F16 | 554 | details |
| LightOnOCR 3 4B | 4.5B | F16 | 9.0 GB | F16 | 570 | details |
| SingVL S1 4B | 4.5B | F16 | 9.0 GB | F16 | 570 | details |
| Intern Decision 4B | 4.5B | F16 | 9.0 GB | F16 | 570 | details |
| Qwen3 VL 4B Instruct | 4.4B | F16 | 9.3 GB | F16 | 596 | details |
| gemma 3 4b it | 4.3B | F16 | 8.4 GB | F16 | 618 | details |
| medgemma 1.5 4b it | 4.3B | F16 | 8.4 GB | F16 | 618 | details |
| medgemma 4b it | 4.3B | F16 | 8.4 GB | F16 | 618 | details |
| gemma 3 4b pt | 4.3B | F16 | 9.2 GB | F16 | 558 | details |
| Phi 3.5 vision instruct | 4.1B | F16 | 11.2 GB | F16 | 579 | details |
| Qwen3 4B | 4.0B | F16 | 9.3 GB | F16 | 597 | details |
| Qwen3 4B Instruct 2507 | 4.0B | F16 | 9.3 GB | F16 | 596 | details |
| granite vision 4.1 4b | 4.0B | F16 | 7.6 GB | F16 | 705 | details |
| Phi 4 mini instruct | 3.8B | F16 | 8.9 GB | F16 | 625 | details |
| Phi 3 mini 4k instruct | 3.8B | F16 | 10.6 GB | F16 | 628 | details |
| Phi 3.5 mini instruct | 3.8B | F16 | 10.6 GB | F16 | 628 | details |
| Phi 3 mini 128k instruct | 3.8B | F16 | 10.6 GB | F16 | 628 | details |
| Qwen2.5 VL 3B Instruct | 3.8B | F16 | 6.7 GB | F16 | 777 | details |
| granite 4.2 3b | 3.7B | F16 | 8.0 GB | F16 | 655 | details |
| granite 4.1 3b llmon preview | 3.7B | F16 | 8.0 GB | F16 | 656 | details |
| Raon 1 nano | 3.5B | F16 | 7.0 GB | F16 | 765 | details |
| granite 4.1 3b | 3.4B | F16 | 7.6 GB | F16 | 705 | details |
| granite 4.0 micro | 3.4B | F16 | 7.6 GB | F16 | 705 | details |
| DeepSeek OCR 2 | 3.4B (1.2B act.) | F16 | 6.6 GB | F16 | 2318 | details |
| deepseek vl2 tiny | 3.4B (1.2B act.) | F16 | 7.4 GB | F16 | 2042 | details |
| Unlimited OCR 2 | 3.3B (1.1B act.) | F16 | 6.6 GB | F16 | 2374 | details |
| Unlimited OCR | 3.3B (1.1B act.) | F16 | 6.6 GB | F16 | 2388 | details |
| DeepSeek OCR | 3.3B (1.1B act.) | F16 | 6.6 GB | F16 | 2388 | details |
| granite 3.1 3b a800m instruct | 3.3B (883M act.) | F16 | 7.1 GB | F16 | 2716 | details |
| Llama 3.2 3B Instruct | 3.2B | F16 | 13.5 GB | F16 | 373 | details |
| Llama 3.2 3B | 3.2B | F16 | 7.5 GB | F16 | 747 | details |
| Hermes 3 Llama 3.2 3B | 3.2B | F16 | 7.5 GB | F16 | 746 | details |
| d1 3B | 3.1B | F16 | 10.8 GB | F16 | 444 | details |
| LFM2.5 VL 3B | 3.1B | F16 | 10.8 GB | F16 | 444 | details |
| Qwen2.5 3B Instruct | 3.1B | F16 | 6.7 GB | F16 | 778 | details |
| SmolLM3 3B | 3.1B | F16 | 6.9 GB | F16 | 779 | details |
| SmolLM3 3B Base | 3.1B | F16 | 6.9 GB | F16 | 780 | details |
| phi 2 | 2.8B | F16 | 8.2 GB | F16 | 863 | details |
| LFM2.5 2.6B | 2.7B | F16 | 10.8 GB | F16 | 444 | details |
| granite 3.3 2b instruct | 2.5B | F16 | 5.8 GB | F16 | 947 | details |
| MiniCPM5 2B | 2.5B | F16 | 5.7 GB | F16 | 953 | details |
| WeVisOne VL 2B | 2.4B | F16 | 5.4 GB | F16 | 1178 | details |
| Qwen3.5 2B | 2.3B | F16 | 4.5 GB | F16 | 1271 | details |
| SmolVLM Instruct | 2.2B | F16 | 5.4 GB | F16 | 1324 | details |
| Qwen3 1.7B | 2.0B | F16 | 4.8 GB | F16 | 1392 | details |
| Qwen3.8 27B DSpine b16 | 2.0B | F16 | 4.8 GB | F16 | 1182 | details |
| d1 2B | 1.9B | F16 | 7.2 GB | F16 | 692 | details |
| LFM3 VL 2B | 1.9B | F16 | 7.2 GB | F16 | 692 | details |
| DeepSeek R1 Distill Qwen 1.5B | 1.8B | F16 | 4.2 GB | F16 | 1348 | details |
| LFM3 1.7B | 1.7B | F16 | 7.2 GB | F16 | 692 | details |
| LFM3 1.7B Base | 1.7B | F16 | 4.0 GB | F16 | 1388 | details |
| SmolLM2 1.7B Instruct | 1.7B | F16 | 5.2 GB | F16 | 1402 | details |
| SmolLM2 1.7B | 1.7B | F16 | 5.2 GB | F16 | 1402 | details |
| SmolLM 1.7B | 1.7B | F16 | 5.2 GB | F16 | 1402 | details |
| granite 4.0 1b | 1.6B | F16 | 4.3 GB | F16 | 1469 | details |
| Qwen2.5 1.5B Instruct | 1.5B | F16 | 3.8 GB | F16 | 1555 | details |
| OLMo 2 0425 1B | 1.5B | F16 | 4.4 GB | F16 | 1614 | details |
| deepseek coder 1.3b instruct | 1.3B | F16 | 4.5 GB | F16 | 1782 | details |
| Llama 3.2 1B | 1.2B | F16 | 3.2 GB | F16 | 1942 | details |
| Llama 3.2 1B Instruct | 1.2B | F16 | 5.5 GB | F16 | 968 | details |
| LFM2.5 1.2B Instruct | 1.2B | F16 | 5.0 GB | F16 | 1024 | details |
| HunyuanOCR | 1.1B | F16 | 3.5 GB | F16 | 2143 | details |
| PaddleOCR VL 1.6 | 959M | F16 | 2.5 GB | F16 | 2504 | details |
| Qwen3.5 0.8B | 873M | F16 | 2.4 GB | F16 | 3165 | details |
| LightOnOCR 3 0.8B | 853M | F16 | 2.4 GB | F16 | 3165 | details |
| FastVLM 0.5B | 759M | F16 | 2.2 GB | F16 | 3163 | details |
| Qwen3 0.6B | 752M | F16 | 2.7 GB | F16 | 4006 | details |
| GOT OCR2 0 | 716M | F16 | 2.8 GB | F16 | 3352 | details |
| Nemotron 3.5 Parse | 631M | F16 | 1.9 GB | F16 | 3673 | details |
| GOT OCR 2.0 hf | 561M | F16 | 2.5 GB | F16 | 4282 | details |
| SmolVLM2 500M Video Instruct | 507M | F16 | 1.6 GB | F16 | 5851 | details |
| SmolVLM 500M Instruct | 507M | F16 | 1.6 GB | F16 | 5851 | details |
| Qwen2.5 0.5B Instruct | 494M | F16 | 1.7 GB | F16 | 4858 | details |
| LFM2.5 VL 450M | 449M | F16 | 1.9 GB | F16 | 3373 | details |
| SmolLM2 360M Instruct | 362M | F16 | 1.5 GB | F16 | 6616 | details |
| SmolLM2 360M | 362M | F16 | 1.5 GB | F16 | 6616 | details |
| LFM2.5 350M | 354M | F16 | 1.9 GB | F16 | 3373 | details |
| LFM2 350M | 354M | F16 | 1.4 GB | F16 | 6770 | details |
Just too big at 4-bit
These need up to 35% more memory than the H200 has at Q4_K_M and 8K context. A smaller quant or shorter context may work.
| Model | Q4_K_M needs | Short by |
|---|---|---|
| DeepSeek V2 Chat | 135.3 GB | 1.4 GB |
| Intern S1 | 139.2 GB | 5.2 GB |
| ERNIE 5.5 Flash Base | 164.2 GB | 30.2 GB |
| DeepSeek V4 Flash | 167.0 GB | 33.1 GB |
| Hy3 | 172.8 GB | 38.8 GB |
| Hy3 preview | 173.2 GB | 39.3 GB |
| DeepSeek V4 Flash 0731 | 174.6 GB | 40.6 GB |
| DeepSeek V4 Flash DSpark | 174.6 GB | 40.6 GB |
| DeepSeek V4 Flash Vision Exp | 174.8 GB | 40.9 GB |
| MiMo V2 Flash | 175.8 GB | 41.8 GB |
| MiMo V2.6 Flash RL | 178.7 GB | 44.8 GB |
| MiMo V2.6 Flash MOPD | 178.7 GB | 44.8 GB |
| MiMo V2.5 | 179.6 GB | 45.7 GB |
Questions
What is the biggest model that runs on the H200?
Of the models we track, the largest that fits at 8K context is GLM 4.5 (358B) at Q2_K, needing about 126 GB.
How much of the H200's memory can a model use?
About 133.9 GB: we assume 95% of the 141 GB is usable after driver and display overhead.
Spec source: NVIDIA specifications. Model data: Hugging Face Hub API. Speed is a bandwidth ceiling (bandwidth ÷ bytes read per token), not a benchmark. Methodology · Report an error.