RunsOnMy ModelsGPUs & MacsAPI pricesVRAM calculatorNew

Can I run diffusiongemma 26B A4B it locally?

diffusiongemma 26B A4B it by google has 26B parameters (4.0B active per token, 128 experts, 8 used per token). At 4-bit with an 8K context it needs about 17.1 GB of GPU memory. It fits on 58 of 65 GPUs and Macs we track.

17.1 GBQ4_K_M · 8K context
RX 7900 XTsmallest consumer GPU that fits
M4 32GBsmallest Mac that fits
256Kmax context
GQA + sliding windowattention type

Memory needed by quant and context length

GB of GPU memory = weights + KV cache (FP16) + runtime overhead. Weights use real file sizes from unsloth/diffusiongemma-26B-A4B-it-GGUF where available (marked •), otherwise parameter count × bits per weight. How we calculate.

QuantQuality4K ctx8K ctx32K ctx128K ctx
2-bit (Q2_K)rough10.3 GB10.4 GB11.4 GB15.1 GB
3-bit (Q3_K_M)usable13.0 GB13.2 GB14.1 GB17.8 GB
4-bit (Q4_K_M) •good, most popular16.9 GB17.1 GB18.0 GB21.7 GB
5-bit (Q5_K_M) •very good19.1 GB19.2 GB20.2 GB23.9 GB
6-bit (Q6_K) •near lossless22.4 GB22.5 GB23.4 GB27.2 GB
8-bit (Q8_0) •effectively lossless26.3 GB26.4 GB27.4 GB31.1 GB
16-bit (F16/BF16) •original weights48.3 GB48.5 GB49.4 GB53.2 GB

Which GPUs and Macs run diffusiongemma 26B A4B it?

Best quality quant that fits at 8K context. "Tight" = within 10% of usable memory. Speed is a ceiling from memory bandwidth (real speed is typically 50–80% of it).

HardwareMemoryFits?Best quantNeedsMax tok/s (est.)
RTX 3060 12GB12 GBTightQ2_K10.4 GB240details
RTX 4060 Ti 16GB16 GBFitsQ3_K_M13.2 GB147details
RTX 309024 GBFitsQ5_K_M19.2 GB316details
RTX 409024 GBFitsQ5_K_M19.2 GB340details
RTX 509032 GBFitsQ8_026.4 GB430details
RTX 5070 Ti16 GBFitsQ3_K_M13.2 GB458details
RX 7900 XTX24 GBFitsQ5_K_M19.2 GB324details
Mac M4 Pro 48GB48 GBFitsQ8_026.4 GB66details
Mac M4 Max 128GB128 GBFitsF1648.5 GB70details
Mac M3 Ultra 512GB512 GBFitsF1648.5 GB105details
DGX Spark128 GBFitsF1648.5 GB35details
RTX 407012 GBTightQ2_K10.4 GB336details
RTX 5060 Ti 16GB16 GBFitsQ3_K_M13.2 GB229details
RTX 40608 GBNo———details
Arc B58012 GBTightQ2_K10.4 GB304details
RTX 3060 Ti8 GBNo———details
RTX 30708 GBNo———details
RTX 3080 10GB10 GBNo———details
RTX 3080 12GB12 GBTightQ2_K10.4 GB608details
RTX 3090 Ti24 GBFitsQ5_K_M19.2 GB340details
RTX 4060 Ti 8GB8 GBNo———details
RTX 4070 Super12 GBTightQ2_K10.4 GB336details
RTX 4070 Ti12 GBTightQ2_K10.4 GB336details
RTX 4070 Ti Super16 GBFitsQ3_K_M13.2 GB344details
RTX 408016 GBFitsQ3_K_M13.2 GB367details
RTX 4080 Super16 GBFitsQ3_K_M13.2 GB376details
RTX 50608 GBNo———details
RTX 507012 GBTightQ2_K10.4 GB448details
RTX 508016 GBFitsQ3_K_M13.2 GB491details
2x RTX 30902×24 GBFitsQ8_026.4 GB225details
2x RTX 40902×24 GBFitsQ8_026.4 GB242details
2x RTX 50902×32 GBFitsF1648.5 GB229details
RTX A600048 GBFitsQ8_026.4 GB184details
RTX 6000 Ada48 GBFitsQ8_026.4 GB231details
RTX PRO 6000 Blackwell96 GBFitsF1648.5 GB229details
L40S48 GBFitsQ8_026.4 GB208details
A100 40GB40 GBFitsQ8_026.4 GB373details
A100 80GB80 GBFitsF1648.5 GB260details
H100 PCIe80 GBFitsF1648.5 GB255details
H100 SXM80 GBFitsF1648.5 GB428details
H200141 GBFitsF1648.5 GB613details
RX 76008 GBNo———details
RX 7700 XT12 GBTightQ2_K10.4 GB288details
RX 7800 XT16 GBFitsQ3_K_M13.2 GB319details
RX 7900 GRE16 GBFitsQ3_K_M13.2 GB295details
RX 7900 XT20 GBFitsQ4_K_M17.1 GB307details
RX 907016 GBFitsQ3_K_M13.2 GB327details
RX 9070 XT16 GBFitsQ3_K_M13.2 GB327details
Ryzen AI Max+ 395 128GB128 GBFitsF1648.5 GB33details
Arc A770 16GB16 GBFitsQ3_K_M13.2 GB286details
Mac M1 16GB16 GBTightQ2_K10.4 GB45details
Mac M2 24GB24 GBFitsQ3_K_M13.2 GB51details
Mac M3 24GB24 GBFitsQ3_K_M13.2 GB51details
Mac M4 16GB16 GBTightQ2_K10.4 GB80details
Mac M4 32GB32 GBFitsQ5_K_M19.2 GB40details
Mac M1 Pro 32GB32 GBFitsQ5_K_M19.2 GB67details
Mac M3 Pro 36GB36 GBFitsQ5_K_M19.2 GB51details
Mac M4 Pro 24GB24 GBFitsQ3_K_M13.2 GB140details
Mac M4 Pro 64GB64 GBFitsQ8_026.4 GB66details
Mac M1 Max 64GB64 GBFitsQ8_026.4 GB96details
Mac M2 Max 96GB96 GBFitsF1648.5 GB51details
Mac M3 Max 128GB128 GBFitsF1648.5 GB51details
Mac M4 Max 64GB64 GBFitsQ8_026.4 GB131details
Mac M2 Ultra 192GB192 GBFitsF1648.5 GB102details
Mac M3 Ultra 256GB256 GBFitsF1648.5 GB105details

Questions

How much VRAM does diffusiongemma 26B A4B it need?

About 17.1 GB at 4-bit (Q4_K_M) with an 8K context: 15.7 GB of weights (real GGUF file size), 0.5 GB of KV cache and 0.9 GB of runtime overhead. At 8-bit it needs about 26.4 GB.

What is the smallest GPU that runs diffusiongemma 26B A4B it?

At Q4_K_M and 8K context, the smallest single consumer GPU in our table that fits it is the RX 7900 XT (20 GB).

Can I run diffusiongemma 26B A4B it on a Mac?

Yes. At Q4_K_M with 8K context it fits on a Mac M4 32GB or larger, using the default macOS GPU memory limit.

Model facts

Hugging Face repogoogle/diffusiongemma-26B-A4B-it
Parameters25,823,778,864 (26B) · ~4.0B active (from the model name)
Layers / hidden size30 / 2816
AttentionGQA + sliding window; 16 heads, 8 KV heads, head dim 256; sliding window 1024
KV cache per token (FP16)240.0 KiB (sliding layers capped at the window)
Max context262,144 tokens
Licenseapache-2.0
Released on HFJun 9, 2026
Downloads (30d) / likes648,032 / 1,333

Sources: Hugging Face model API · config.json (google/diffusiongemma-26B-A4B-it) · GGUF files · GPU specs from vendor pages (list). Data fetched Nov 17, 2026. Spot an error? Report it.