RunsOnMy ModelsGPUs & MacsAPI pricesVRAM calculatorNew

Can I run Arete 27B locally?

Arete 27B by NousResearch has 27B parameters. At 4-bit with an 8K context it needs about 17.4 GB of GPU memory. It fits on 56 of 65 GPUs and Macs we track.

17.4 GBQ4_K_M · 8K context
RTX 3090smallest consumer GPU that fits
M4 32GBsmallest Mac that fits
256Kmax context
GQAattention type

Memory needed by quant and context length

GB of GPU memory = weights + KV cache (FP16) + runtime overhead. Weights use real file sizes from bartowski/NousResearch_Arete-27B-GGUF where available (marked •), otherwise parameter count × bits per weight. How we calculate.

QuantQuality4K ctx8K ctx32K ctx128K ctx
2-bit (Q2_K) •rough11.0 GB11.2 GB12.7 GB18.7 GB
3-bit (Q3_K_M) •usable13.4 GB13.6 GB15.1 GB21.1 GB
4-bit (Q4_K_M) •good, most popular17.1 GB17.4 GB18.9 GB24.9 GB
5-bit (Q5_K_M) •very good20.4 GB20.6 GB22.1 GB28.1 GB
6-bit (Q6_K) •near lossless23.1 GB23.4 GB24.9 GB30.9 GB
8-bit (Q8_0) •effectively lossless27.8 GB28.1 GB29.6 GB35.6 GB
16-bit (F16/BF16) •original weights51.2 GB51.5 GB53.0 GB59.0 GB

Which GPUs and Macs run Arete 27B?

Best quality quant that fits at 8K context. "Tight" = within 10% of usable memory. Speed is a ceiling from memory bandwidth (real speed is typically 50–80% of it).

HardwareMemoryFits?Best quantNeedsMax tok/s (est.)
RTX 3060 12GB12 GBTightQ2_K11.2 GB34details
RTX 4060 Ti 16GB16 GBFitsQ3_K_M13.6 GB22details
RTX 309024 GBFitsQ4_K_M17.4 GB54details
RTX 409024 GBFitsQ4_K_M17.4 GB59details
RTX 509032 GBFitsQ6_K23.4 GB76details
RTX 5070 Ti16 GBFitsQ3_K_M13.6 GB68details
RX 7900 XTX24 GBFitsQ4_K_M17.4 GB56details
Mac M4 Pro 48GB48 GBFitsQ8_028.1 GB9.5details
Mac M4 Max 128GB128 GBFitsF1651.5 GB10details
Mac M3 Ultra 512GB512 GBFitsF1651.5 GB15details
DGX Spark128 GBFitsF1651.5 GB5.1details
RTX 407012 GBTightQ2_K11.2 GB48details
RTX 5060 Ti 16GB16 GBFitsQ3_K_M13.6 GB34details
RTX 40608 GBNo———details
Arc B58012 GBTightQ2_K11.2 GB43details
RTX 3060 Ti8 GBNo———details
RTX 30708 GBNo———details
RTX 3080 10GB10 GBNo———details
RTX 3080 12GB12 GBTightQ2_K11.2 GB86details
RTX 3090 Ti24 GBFitsQ4_K_M17.4 GB59details
RTX 4060 Ti 8GB8 GBNo———details
RTX 4070 Super12 GBTightQ2_K11.2 GB48details
RTX 4070 Ti12 GBTightQ2_K11.2 GB48details
RTX 4070 Ti Super16 GBFitsQ3_K_M13.6 GB51details
RTX 408016 GBFitsQ3_K_M13.6 GB54details
RTX 4080 Super16 GBFitsQ3_K_M13.6 GB56details
RTX 50608 GBNo———details
RTX 507012 GBTightQ2_K11.2 GB64details
RTX 508016 GBFitsQ3_K_M13.6 GB73details
2x RTX 30902×24 GBFitsQ8_028.1 GB33details
2x RTX 40902×24 GBFitsQ8_028.1 GB35details
2x RTX 50902×32 GBFitsF1651.5 GB33details
RTX A600048 GBFitsQ8_028.1 GB27details
RTX 6000 Ada48 GBFitsQ8_028.1 GB33details
RTX PRO 6000 Blackwell96 GBFitsF1651.5 GB33details
L40S48 GBFitsQ8_028.1 GB30details
A100 40GB40 GBFitsQ8_028.1 GB54details
A100 80GB80 GBFitsF1651.5 GB38details
H100 PCIe80 GBFitsF1651.5 GB37details
H100 SXM80 GBFitsF1651.5 GB62details
H200141 GBFitsF1651.5 GB89details
RX 76008 GBNo———details
RX 7700 XT12 GBTightQ2_K11.2 GB41details
RX 7800 XT16 GBFitsQ3_K_M13.6 GB47details
RX 7900 GRE16 GBFitsQ3_K_M13.6 GB44details
RX 7900 XT20 GBFitsQ3_K_M13.6 GB61details
RX 907016 GBFitsQ3_K_M13.6 GB49details
RX 9070 XT16 GBFitsQ3_K_M13.6 GB49details
Ryzen AI Max+ 395 128GB128 GBFitsF1651.5 GB4.8details
Arc A770 16GB16 GBFitsQ3_K_M13.6 GB43details
Mac M1 16GB16 GBNo———details
Mac M2 24GB24 GBFitsQ3_K_M13.6 GB7.6details
Mac M3 24GB24 GBFitsQ3_K_M13.6 GB7.6details
Mac M4 16GB16 GBNo———details
Mac M4 32GB32 GBFitsQ4_K_M17.4 GB7.0details
Mac M1 Pro 32GB32 GBFitsQ4_K_M17.4 GB12details
Mac M3 Pro 36GB36 GBFitsQ5_K_M20.6 GB7.3details
Mac M4 Pro 24GB24 GBFitsQ3_K_M13.6 GB21details
Mac M4 Pro 64GB64 GBFitsQ8_028.1 GB9.5details
Mac M1 Max 64GB64 GBFitsQ8_028.1 GB14details
Mac M2 Max 96GB96 GBFitsF1651.5 GB7.4details
Mac M3 Max 128GB128 GBFitsF1651.5 GB7.4details
Mac M4 Max 64GB64 GBFitsQ8_028.1 GB19details
Mac M2 Ultra 192GB192 GBFitsF1651.5 GB15details
Mac M3 Ultra 256GB256 GBFitsF1651.5 GB15details

Questions

How much VRAM does Arete 27B need?

About 17.4 GB at 4-bit (Q4_K_M) with an 8K context: 16.0 GB of weights (real GGUF file size), 0.5 GB of KV cache and 0.9 GB of runtime overhead. At 8-bit it needs about 28.1 GB.

What is the smallest GPU that runs Arete 27B?

At Q4_K_M and 8K context, the smallest single consumer GPU in our table that fits it is the RTX 3090 (24 GB).

Can I run Arete 27B on a Mac?

Yes. At Q4_K_M with 8K context it fits on a Mac M4 32GB or larger, using the default macOS GPU memory limit.

Model facts

Hugging Face repoNousResearch/Arete-27B
Parameters27,356,728,560 (27B)
Layers / hidden size64 / 5120
AttentionGQA (hybrid: 48 linear/state layers); 24 heads, 4 KV heads, head dim 256
KV cache per token (FP16)64.0 KiB
Max context262,144 tokens
Licenseapache-2.0
Released on HFNov 6, 2026
Downloads (30d) / likes9,187 / 671

Sources: Hugging Face model API · config.json (NousResearch/Arete-27B) · GGUF files · GPU specs from vendor pages (list). Data fetched Nov 17, 2026. Spot an error? Report it.