FitMyModel

What local AI models run on 96 GB of VRAM?

Last updated

We earn from qualifying purchases via affiliate links. Disclosure.

13 of 13 ranked open models fit in 96 GB: 13 with 4 GB or more to spare and 0 tight. The best ranked is Gemma 4 31B (19–20 GB).

ModelLMArenaNeedsFitPages
Gemma 4 31BGoogle · Q4_K_M
 #76 19–20 GB Fits Ryzen AI Max+ 395 (128 GB) · Guide · 24–32 GB GPU tier
Gemma 4 26B-A4B (MoE)Google · default tag
 #100 16 GB Fits Ryzen AI Max+ 395 (128 GB) · 24–32 GB GPU tier
Qwen3.8 27BAlibaba Qwen · Q4_K_M
 #102 18 GB Fits Ryzen AI Max+ 395 (128 GB) · 24–32 GB GPU tier
Qwen3.5 122B-A10B (MoE)Alibaba Qwen · Q4_K_M
 #138 81 GB Fits Ryzen AI Max+ 395 (128 GB) · Guide · Unified 128 GB tier
Nemotron 3.5 Super 120B-A12BNVIDIA · Q4_K_M (default tag 80 GB)
 #149 90 GB Fits Ryzen AI Max+ 395 (128 GB) · Unified 128 GB tier
Qwen3.5 27BAlibaba Qwen · Q4_K_M
 #155 17–20 GB Fits Ryzen AI Max+ 395 (128 GB) · Guide · 24–32 GB GPU tier
Qwen3.5 35B-A3B (MoE)Alibaba Qwen · Q4_K_M
 #172 22 GB Fits Ryzen AI Max+ 395 (128 GB) · Guide · 24–32 GB GPU tier
Mistral Small 3.2 24BMistral AI · Q4_K_M
 #212 15 GB Fits Ryzen AI Max+ 395 (128 GB) · 16 GB GPU tier
gpt-oss 120BOpenAI · MXFP4 (as shipped)
 #218 65 GB Fits Ryzen AI Max+ 395 (128 GB) · Guide · Unified 128 GB tier
Llama 3.3 70BMeta · Q4_K_M
 #266 43 GB Fits Ryzen AI Max+ 395 (128 GB) · Guide · Unified 64 GB tier
gpt-oss 20BOpenAI · MXFP4 (as shipped)
 #269 14 GB Fits Ryzen AI Max+ 395 (128 GB) · Guide · 16 GB GPU tier
Phi-4 14BMicrosoft · Q4_K_M
 #322 9.1 GB Fits Ryzen AI Max+ 395 (128 GB) · Guide · 8–12 GB GPU tier
Llama 3.1 8BMeta · Q4_K_M
 #345 4.9 GB Fits Ryzen AI Max+ 395 (128 GB) · Guide · 8–12 GB GPU tier

Fit compares the Ollama library download size with 96 GB. "Fits" means 4 GB or more spare and "Tight" means under 4 GB; both bands are our estimate, and tight models need a short context. Ranks from LMArena Text, checked Oct 27, 2026, 2:16 PM CT. No speeds are listed because we have no sourced numbers.

Hardware with 96 GB for the GPU

Minisforum MS-S1 MAX mini workstation — illustrated product card

Minisforum MS-S1 MAX 128GB / 2TB (Ryzen AI Max+ 395)

$3,919 Minisforum · checked Oct 27, 2026, 12:40 PM CT

Caveat

Store data says estimated shipping late November. The 128GB / 4TB version was $4,199.

Check price at Minisforum (opens in a new tab)

Prices: Newegg checked Oct 27, 2026, about 12:35 PM CT; Minisforum checked Oct 27, 2026, 12:40 PM CT. New condition only.

Other memory sizes

← 64 GB 8 GB 12 GB 16 GB 24 GB 32 GB 48 GB 64 GB 96 GB 128 GB 128 GB →

Try the What fits my memory tool or browse every pairing on Can it run?

FAQ

What local AI models run on 96 GB of VRAM?

13 of the 13 leaderboard-ranked open models we track fit in 96 GB: Gemma 4 31B (19–20 GB), Gemma 4 26B-A4B (MoE) (16 GB), Qwen3.8 27B (18 GB), Qwen3.5 122B-A10B (MoE) (81 GB), Nemotron 3.5 Super 120B-A12B (90 GB), Qwen3.5 27B (17–20 GB), Qwen3.5 35B-A3B (MoE) (22 GB), Mistral Small 3.2 24B (15 GB), gpt-oss 120B (65 GB), Llama 3.3 70B (43 GB), gpt-oss 20B (14 GB), Phi-4 14B (9.1 GB), Llama 3.1 8B (4.9 GB). Sizes are Ollama library downloads; all leave 4 GB or more spare (our estimate).

What is the best-ranked local model for 96 GB?

Gemma 4 31B, #76 overall on LMArena Text (checked Oct 27, 2026, 2:16 PM CT), needs 19–20 GB.

What is the largest model that fits in 96 GB?

Nemotron 3.5 Super 120B-A12B at 90 GB (Q4_K_M (default tag 80 GB)). The runtime and context need roughly 1–4 GB more, which is our estimate.

Does 96 GB of unified memory count as 96 GB of VRAM?

No. On a Ryzen AI Max+ 395 system AMD says the GPU can use up to 48 GB of a 64 GB system and up to 96 GB of a 128 GB system, so use the GPU share when you read this page.