RunsOnMy ModelsGPUs & MacsAPI pricesVRAM calculatorNew

Can I run flint locally?

flint by RecursiveSI has 40B parameters (1.3B active per token, 256 experts, 6 used per token). At 4-bit with an 8K context it needs about 23.6 GB of GPU memory. It fits on 48 of 65 GPUs and Macs we track.

23.6 GBQ4_K_M · 8K context
RTX 5090smallest consumer GPU that fits
M4 Pro 48GBsmallest Mac that fits
256Kmax context
GQA + sliding windowattention type

Memory needed by quant and context length

GB of GPU memory = weights + KV cache (FP16) + runtime overhead. Weights use real file sizes from unsloth/flint-GGUF where available (marked •), otherwise parameter count × bits per weight. How we calculate.

QuantQuality4K ctx8K ctx32K ctx128K ctx
2-bit (Q2_K)rough14.7 GB14.8 GB15.5 GB18.1 GB
3-bit (Q3_K_M)usable18.9 GB19.0 GB19.7 GB22.3 GB
4-bit (Q4_K_M)good, most popular23.5 GB23.6 GB24.2 GB26.9 GB
5-bit (Q5_K_M)very good27.2 GB27.3 GB28.0 GB30.6 GB
6-bit (Q6_K)near lossless31.2 GB31.3 GB32.0 GB34.6 GB
8-bit (Q8_0) •effectively lossless40.2 GB40.3 GB41.0 GB43.6 GB
16-bit (F16/BF16) •original weights74.9 GB75.1 GB75.7 GB78.3 GB

Which GPUs and Macs run flint?

Best quality quant that fits at 8K context. "Tight" = within 10% of usable memory. Speed is a ceiling from memory bandwidth (real speed is typically 50–80% of it).

HardwareMemoryFits?Best quantNeedsMax tok/s (est.)
RTX 3060 12GB12 GBNo———details
RTX 4060 Ti 16GB16 GBTightQ2_K14.8 GB614details
RTX 309024 GBFitsQ3_K_M19.0 GB1532details
RTX 409024 GBFitsQ3_K_M19.0 GB1650details
RTX 509032 GBFitsQ5_K_M27.3 GB2012details
RTX 5070 Ti16 GBTightQ2_K14.8 GB1911details
RX 7900 XTX24 GBFitsQ3_K_M19.0 GB1571details
Mac M4 Pro 48GB48 GBFitsQ5_K_M27.3 GB306details
Mac M4 Max 128GB128 GBFitsF1675.1 GB218details
Mac M3 Ultra 512GB512 GBFitsF1675.1 GB328details
DGX Spark128 GBFitsF1675.1 GB109details
RTX 407012 GBNo———details
RTX 5060 Ti 16GB16 GBTightQ2_K14.8 GB956details
RTX 40608 GBNo———details
Arc B58012 GBNo———details
RTX 3060 Ti8 GBNo———details
RTX 30708 GBNo———details
RTX 3080 10GB10 GBNo———details
RTX 3080 12GB12 GBNo———details
RTX 3090 Ti24 GBFitsQ3_K_M19.0 GB1650details
RTX 4060 Ti 8GB8 GBNo———details
RTX 4070 Super12 GBNo———details
RTX 4070 Ti12 GBNo———details
RTX 4070 Ti Super16 GBTightQ2_K14.8 GB1433details
RTX 408016 GBTightQ2_K14.8 GB1529details
RTX 4080 Super16 GBTightQ2_K14.8 GB1570details
RTX 50608 GBNo———details
RTX 507012 GBNo———details
RTX 508016 GBTightQ2_K14.8 GB2048details
2x RTX 30902×24 GBFitsQ8_040.3 GB704details
2x RTX 40902×24 GBFitsQ8_040.3 GB758details
2x RTX 50902×32 GBFitsQ8_040.3 GB1348details
RTX A600048 GBFitsQ8_040.3 GB578details
RTX 6000 Ada48 GBFitsQ8_040.3 GB722details
RTX PRO 6000 Blackwell96 GBFitsF1675.1 GB717details
L40S48 GBFitsQ8_040.3 GB650details
A100 40GB40 GBFitsQ6_K31.3 GB1517details
A100 80GB80 GBFitsQ8_040.3 GB1534details
H100 PCIe80 GBFitsQ8_040.3 GB1505details
H100 SXM80 GBFitsQ8_040.3 GB2520details
H200141 GBFitsF1675.1 GB1920details
RX 76008 GBNo———details
RX 7700 XT12 GBNo———details
RX 7800 XT16 GBTightQ2_K14.8 GB1331details
RX 7900 GRE16 GBTightQ2_K14.8 GB1229details
RX 7900 XT20 GBFitsQ2_K14.8 GB1707details
RX 907016 GBTightQ2_K14.8 GB1365details
RX 9070 XT16 GBTightQ2_K14.8 GB1365details
Ryzen AI Max+ 395 128GB128 GBFitsF1675.1 GB102details
Arc A770 16GB16 GBTightQ2_K14.8 GB1195details
Mac M1 16GB16 GBNo———details
Mac M2 24GB24 GBTightQ2_K14.8 GB213details
Mac M3 24GB24 GBTightQ2_K14.8 GB213details
Mac M4 16GB16 GBNo———details
Mac M4 32GB32 GBFitsQ3_K_M19.0 GB196details
Mac M1 Pro 32GB32 GBFitsQ3_K_M19.0 GB327details
Mac M3 Pro 36GB36 GBFitsQ3_K_M19.0 GB246details
Mac M4 Pro 24GB24 GBTightQ2_K14.8 GB582details
Mac M4 Pro 64GB64 GBFitsQ8_040.3 GB205details
Mac M1 Max 64GB64 GBFitsQ8_040.3 GB301details
Mac M2 Max 96GB96 GBFitsQ8_040.3 GB301details
Mac M3 Max 128GB128 GBFitsF1675.1 GB160details
Mac M4 Max 64GB64 GBFitsQ8_040.3 GB411details
Mac M2 Ultra 192GB192 GBFitsF1675.1 GB320details
Mac M3 Ultra 256GB256 GBFitsF1675.1 GB328details

Prefer an API?

Recursive: Flint (free) is on OpenRouter at free per 1M input tokens and free per 1M output tokens. Price details and history →

Questions

How much VRAM does flint need?

About 23.6 GB at 4-bit (Q4_K_M) with an 8K context: 22.6 GB of weights (estimated), 0.3 GB of KV cache and 0.7 GB of runtime overhead. At 8-bit it needs about 40.3 GB.

What is the smallest GPU that runs flint?

At Q4_K_M and 8K context, the smallest single consumer GPU in our table that fits it is the RTX 5090 (32 GB).

Can I run flint on a Mac?

Yes. At Q4_K_M with 8K context it fits on a Mac M4 Pro 48GB or larger, using the default macOS GPU memory limit.

Model facts

Hugging Face repoRecursiveSI/flint
Parameters39,785,287,456 (40B) · ~1.3B active (computed from config)
Layers / hidden size49 / 2048
AttentionGQA + sliding window; 32 heads, 4 KV heads, head dim 128; sliding window 513
KV cache per token (FP16)98.0 KiB (sliding layers capped at the window)
Max context262,144 tokens
Licenseapache-2.0
Released on HFOct 21, 2026
Downloads (30d) / likes57,853 / 1,412

Sources: Hugging Face model API · config.json (RecursiveSI/flint) · GGUF files · GPU specs from vendor pages (list). Data fetched Nov 17, 2026. Spot an error? Report it.