RunsOnMy ModelsGPUs & MacsAPI pricesVRAM calculatorNew

Can I run North Mini Code 1.0 locally?

North Mini Code 1.0 by CohereLabs has 30B parameters (3.3B active per token, 128 experts, 8 used per token). At 4-bit with an 8K context it needs about 19.3 GB of GPU memory. It fits on 48 of 65 GPUs and Macs we track.

19.3 GBQ4_K_M · 8K context
RTX 3090smallest consumer GPU that fits
M4 32GBsmallest Mac that fits
488Kmax context
GQA + sliding windowattention type

Memory needed by quant and context length

GB of GPU memory = weights + KV cache (FP16) + runtime overhead. Weights use real file sizes from unsloth/North-Mini-Code-1.0-GGUF where available (marked •), otherwise parameter count × bits per weight. How we calculate.

QuantQuality4K ctx8K ctx32K ctx128K ctx
2-bit (Q2_K)rough11.9 GB12.0 GB12.6 GB15.1 GB
3-bit (Q3_K_M) •usable14.5 GB14.6 GB15.2 GB17.7 GB
4-bit (Q4_K_M) •good, most popular19.2 GB19.3 GB19.9 GB22.3 GB
5-bit (Q5_K_M) •very good22.7 GB22.8 GB23.4 GB25.8 GB
6-bit (Q6_K) •near lossless25.0 GB25.1 GB25.8 GB28.2 GB
8-bit (Q8_0) •effectively lossless31.5 GB31.6 GB32.2 GB34.6 GB
16-bit (F16/BF16) •original weights58.1 GB58.2 GB58.8 GB61.2 GB

Which GPUs and Macs run North Mini Code 1.0?

Best quality quant that fits at 8K context. "Tight" = within 10% of usable memory. Speed is a ceiling from memory bandwidth (real speed is typically 50–80% of it).

HardwareMemoryFits?Best quantNeedsMax tok/s (est.)
RTX 3060 12GB12 GBNo———details
RTX 4060 Ti 16GB16 GBFitsQ2_K12.0 GB232details
RTX 309024 GBFitsQ4_K_M19.3 GB450details
RTX 409024 GBFitsQ4_K_M19.3 GB484details
RTX 509032 GBFitsQ6_K25.1 GB648details
RTX 5070 Ti16 GBFitsQ2_K12.0 GB723details
RX 7900 XTX24 GBFitsQ4_K_M19.3 GB461details
Mac M4 Pro 48GB48 GBFitsQ6_K25.1 GB99details
Mac M4 Max 128GB128 GBFitsF1658.2 GB83details
Mac M3 Ultra 512GB512 GBFitsF1658.2 GB124details
DGX Spark128 GBFitsF1658.2 GB41details
RTX 407012 GBNo———details
RTX 5060 Ti 16GB16 GBFitsQ2_K12.0 GB361details
RTX 40608 GBNo———details
Arc B58012 GBNo———details
RTX 3060 Ti8 GBNo———details
RTX 30708 GBNo———details
RTX 3080 10GB10 GBNo———details
RTX 3080 12GB12 GBNo———details
RTX 3090 Ti24 GBFitsQ4_K_M19.3 GB484details
RTX 4060 Ti 8GB8 GBNo———details
RTX 4070 Super12 GBNo———details
RTX 4070 Ti12 GBNo———details
RTX 4070 Ti Super16 GBFitsQ2_K12.0 GB542details
RTX 408016 GBFitsQ2_K12.0 GB578details
RTX 4080 Super16 GBFitsQ2_K12.0 GB594details
RTX 50608 GBNo———details
RTX 507012 GBNo———details
RTX 508016 GBFitsQ2_K12.0 GB775details
2x RTX 30902×24 GBFitsQ8_031.6 GB266details
2x RTX 40902×24 GBFitsQ8_031.6 GB287details
2x RTX 50902×32 GBFitsQ8_031.6 GB510details
RTX A600048 GBFitsQ8_031.6 GB218details
RTX 6000 Ada48 GBFitsQ8_031.6 GB273details
RTX PRO 6000 Blackwell96 GBFitsF1658.2 GB271details
L40S48 GBFitsQ8_031.6 GB246details
A100 40GB40 GBFitsQ8_031.6 GB442details
A100 80GB80 GBFitsF1658.2 GB308details
H100 PCIe80 GBFitsF1658.2 GB302details
H100 SXM80 GBFitsF1658.2 GB506details
H200141 GBFitsF1658.2 GB726details
RX 76008 GBNo———details
RX 7700 XT12 GBNo———details
RX 7800 XT16 GBFitsQ2_K12.0 GB503details
RX 7900 GRE16 GBFitsQ2_K12.0 GB465details
RX 7900 XT20 GBFitsQ3_K_M14.6 GB519details
RX 907016 GBFitsQ2_K12.0 GB516details
RX 9070 XT16 GBFitsQ2_K12.0 GB516details
Ryzen AI Max+ 395 128GB128 GBFitsF1658.2 GB39details
Arc A770 16GB16 GBFitsQ2_K12.0 GB452details
Mac M1 16GB16 GBNo———details
Mac M2 24GB24 GBFitsQ2_K12.0 GB81details
Mac M3 24GB24 GBFitsQ2_K12.0 GB81details
Mac M4 16GB16 GBNo———details
Mac M4 32GB32 GBFitsQ4_K_M19.3 GB58details
Mac M1 Pro 32GB32 GBFitsQ4_K_M19.3 GB96details
Mac M3 Pro 36GB36 GBFitsQ4_K_M19.3 GB72details
Mac M4 Pro 24GB24 GBFitsQ2_K12.0 GB220details
Mac M4 Pro 64GB64 GBFitsQ8_031.6 GB78details
Mac M1 Max 64GB64 GBFitsQ8_031.6 GB114details
Mac M2 Max 96GB96 GBFitsF1658.2 GB60details
Mac M3 Max 128GB128 GBFitsF1658.2 GB60details
Mac M4 Max 64GB64 GBFitsQ8_031.6 GB155details
Mac M2 Ultra 192GB192 GBFitsF1658.2 GB121details
Mac M3 Ultra 256GB256 GBFitsF1658.2 GB124details

Prefer an API?

Cohere: North Mini Code (free) is on OpenRouter at free per 1M input tokens and free per 1M output tokens. Price details and history →

Questions

How much VRAM does North Mini Code 1.0 need?

About 19.3 GB at 4-bit (Q4_K_M) with an 8K context: 17.9 GB of weights (real GGUF file size), 0.5 GB of KV cache and 0.9 GB of runtime overhead. At 8-bit it needs about 31.6 GB.

What is the smallest GPU that runs North Mini Code 1.0?

At Q4_K_M and 8K context, the smallest single consumer GPU in our table that fits it is the RTX 3090 (24 GB).

Can I run North Mini Code 1.0 on a Mac?

Yes. At Q4_K_M with 8K context it fits on a Mac M4 32GB or larger, using the default macOS GPU memory limit.

Model facts

Hugging Face repoCohereLabs/North-Mini-Code-1.0
Parameters30,484,303,872 (30B) · ~3.3B active (computed from config)
Layers / hidden size49 / 2048
AttentionGQA + sliding window; 32 heads, 4 KV heads, head dim 128; sliding window 4096
KV cache per token (FP16)98.0 KiB (sliding layers capped at the window)
Max context500,000 tokens
Licenseapache-2.0
Released on HFJun 5, 2026
Downloads (30d) / likes8,372 / 588

Sources: Hugging Face model API · config.json (CohereLabs/North-Mini-Code-1.0) · GGUF files · GPU specs from vendor pages (list). Data fetched Nov 17, 2026. Spot an error? Report it.