Can I run Inkling 2 Small on my PC?
New release: Inkling 2 Small was published on Hugging Face on Nov 4, 2026 by thinkingmachines. Numbers update hourly as GGUF files appear. See everything new this week.
Inkling 2 Small (594B, 19B active) needs about 346 GB of GPU memory at 4-bit with an 8K context. No single consumer GPU we track runs it fully in VRAM at 4-bit. No Mac we track runs it at 4-bit under the default memory limit.
Inkling 2 Small on every GPU and Mac
Best quant that fits fully in memory at 8K context with a speed ceiling; otherwise the llama.cpp offload plan (system RAM needed, estimated speed from DDR4-3200 to DDR5-6000). Macs and unified-memory PCs share one pool, so offload adds nothing there. How offload is estimated.
Questions
How much VRAM do I need for Inkling 2 Small?
About 346 GB at 4-bit (Q4_K_M) with an 8K context, including KV cache and runtime overhead. Longer contexts need more; the model page has the full table by quant and context.
Why can a mixture-of-experts model run on a smaller card?
Only 19B of its 594B parameters are used per token. llama.cpp's --n-cpu-moe keeps the expert weights of some layers in system RAM while attention and the KV cache stay on the GPU, so each token reads far less from slow RAM than a dense model of the same size would.
Other popular checks: Can I run gpt-oss-120b?Can I run gpt-oss-20b?Can I run Llama 70B?Can I run Qwen3?Can I run DeepSeek R1? · Best GPU by budget · Mac vs PC
Model data: Hugging Face Hub API and GGUF file sizes. GPU specs: vendor pages. Speeds are estimates, not benchmarks. Methodology · Report an error.