Can I run Hermes 4.3 36B locally?
Hermes 4.3 36B by NousResearch has 36B parameters. At 4-bit with an 8K context it needs about 23.0 GB of GPU memory. It fits on 36 of 65 GPUs and Macs we track. See all models that fit in 32 GB of VRAM.
Memory needed by quant and context length
GB of GPU memory = weights + KV cache (FP16) + runtime overhead. Weights use real file sizes from bartowski/NousResearch_Hermes-4.3-36B-GGUF where available (marked •), otherwise parameter count × bits per weight. How we calculate.
| Quant | Quality | 4K ctx | 8K ctx | 32K ctx | 128K ctx |
|---|---|---|---|---|---|
| 2-bit (Q2_K) • | rough | 14.4 GB | 15.4 GB | 21.4 GB | 45.4 GB |
| 3-bit (Q3_K_M) • | usable | 18.1 GB | 19.1 GB | 25.1 GB | 49.1 GB |
| 4-bit (Q4_K_M) • | good, most popular | 22.0 GB | 23.0 GB | 29.0 GB | 53.0 GB |
| 5-bit (Q5_K_M) • | very good | 25.5 GB | 26.5 GB | 32.5 GB | 56.5 GB |
| 6-bit (Q6_K) • | near lossless | 29.3 GB | 30.3 GB | 36.3 GB | 60.3 GB |
| 8-bit (Q8_0) • | effectively lossless | 37.5 GB | 38.5 GB | 44.5 GB | 68.5 GB |
| 16-bit (F16/BF16) • | original weights | 69.0 GB | 70.0 GB | 76.0 GB | 100 GB |
Which GPUs and Macs run Hermes 4.3 36B?
Best quality quant that fits at 8K context. "Tight" = within 10% of usable memory. Speed is a ceiling from memory bandwidth (real speed is typically 50–80% of it).
Questions
How much VRAM does Hermes 4.3 36B need?
About 23.0 GB at 4-bit (Q4_K_M) with an 8K context: 20.3 GB of weights (real GGUF file size), 2.0 GB of KV cache and 0.7 GB of runtime overhead. At 8-bit it needs about 38.5 GB.
What is the smallest GPU that runs Hermes 4.3 36B?
At Q4_K_M and 8K context, the smallest single consumer GPU in our table that fits it is the RTX 5090 (32 GB).
Can I run Hermes 4.3 36B on a Mac?
Yes. At Q4_K_M with 8K context it fits on a Mac M4 Pro 48GB or larger, using the default macOS GPU memory limit.
Similar models
Same family or a similar parameter count, with the VRAM each needs at Q4_K_M and 8K context.
Nous Hermes 2 Yi 34B
34B · 22.1 GB at 4-bitArete 27B
27B · 17.4 GB at 4-bitNous Hermes 2 Mixtral 8x7B DPO
47B · 28.0 GB at 4-bitSeed OSS 36B Instruct
36B · 23.0 GB at 4-bitQwen3.6 35B A3B
36B · 22.1 GB at 4-bitOrnith 1.5 35B A3B
36B · 21.4 GB at 4-bitModel facts
| Hugging Face repo | NousResearch/Hermes-4.3-36B |
| Parameters | 36,151,104,512 (36B) |
| Layers / hidden size | 64 / 5120 |
| Attention | GQA; 80 heads, 8 KV heads, head dim 128 |
| KV cache per token (FP16) | 256.0 KiB |
| Max context | 524,288 tokens |
| License | apache-2.0 |
| Released on HF | Nov 17, 2025 |
| Downloads (30d) / likes | 3,122 / 329 |
Sources: Hugging Face model API · config.json (NousResearch/Hermes-4.3-36B) · GGUF files · GPU specs from vendor pages (list). Data fetched Nov 17, 2026. Spot an error? Report it.