Unified memory vs dedicated VRAM for LLM inference

Disclosure first: I build Outlier, a local AI app that only runs on Apple Silicon, so I have an obvious interest in one side of this. I've stuck to published specs and public benchmark tables, and labeled every number I couldn't verify at the source. Where a discrete GPU wins, it wins.

One physical fact everything follows from

NVIDIA's own inference-optimization guide is blunt: for token generation, the time spent moving weights, keys, values and activations out of memory dominates the time spent computing. NVIDIA calls it "a memory-bound operation."

That splits local inference into two problems. Prompt processing, meaning reading your question, your files, your codebase, is compute-bound. Token generation, the words appearing on screen, tracks memory bandwidth far more closely than TFLOPs. Any comparison quoting one number hides half the story.

On Apple silicon the CPU and GPU share one pool: MLX's docs note arrays live in shared memory, usable by any device without transferring data. A discrete GPU has its own isolated, much faster pool, and everything must cross PCIe into it.

Capacity: what actually fits

Unified memory used to sit in a class of its own here, and that gap has closed a long way. At the March 2025 M3 Ultra launch Apple offered up to 512GB at 819GB/s, and its claim that models over 600 billion parameters run on device was attached to that configuration. You can't buy it any more. Apple withdrew the 512GB option, then 256GB, then 128GB, and as of July 2026 apple.com/mac-studio/specs tops out at 96GB, still at 819GB/s. The largest pool Apple currently sells is the 128GB M5 Max MacBook Pro at 614GB/s. DGX Spark ships 128GB of coherent memory at 273GB/s for inference up to 200B. An RTX 5090 has 32GB; the RTX PRO 6000 has 96GB with ECC.

So on nameplate capacity a Mac Studio you can order today ties the PRO 6000 at 96GB, and loses to it on bandwidth by more than 2x. The unified-memory capacity argument now only really lands against consumer cards, where 96GB or 128GB against a 5090's 32GB is still a three- to four-fold difference and decides whether a model loads at all.

Those are hard ceilings. Spill out of VRAM and the offloaded layers cross PCIe at a fraction of GDDR7 bandwidth, and throughput falls off a cliff. Unified memory degrades gently instead. Here's what my app's tiers need:

Everything through 35B fits a 5090. The 397B tier doesn't, at any quantization worth running; on a 64GB Mac it works only because a paged mixture-of-experts loader reads experts off the SSD as the router calls for them. On my M1 Ultra that measures 11GB peak process memory and 3.6GB peak wired memory at 1.59 tokens per second. The 64GB minimum in the table is not the working set: loading materializes sixty layers in one burst that needs macOS's Metal wired-memory limit raised to roughly 56GB, which the kernel won't grant on a smaller machine, and the paging itself wants file-cache headroom on top. Slow, but it runs. llama.cpp's DGX Spark file shows the same shape on a mainstream model: 59 GiB of gpt-oss-120b at roughly 2,444 tokens/sec prefill, 59 tokens/sec generation. On a 5090 it doesn't load.

One caveat: unified memory is shared. The model competes with macOS, your browser and the app, and Metal caps GPU allocation through a documented working-set limit — the widely repeated 75%-of-RAM figure is community-reported, not published by Apple. Usable budget runs well under the sticker number.

Bandwidth: speed once it fits

llama.cpp keeps two community benchmark threads in its repo, same model (LLaMA 7B Q4_0). Apple silicon: M4 Max 714 tokens/sec prefill and 70 tokens/sec generation; M2 Ultra 1,238 and 94. CUDA: RTX 5090 14,073 and 290; RTX PRO 6000 14,855 and 274.

The arithmetic is mine, not theirs: a 5090 is about 20x faster than an M4 Max at prompt processing but only about 4x faster at generating tokens. That's the split above — compute-bound prefill, where discrete GPUs are an order of magnitude ahead, and bandwidth-bound decode, where the gap collapses toward the bandwidth ratio.

Apple is closing the compute half deliberately: M5 puts a Neural Accelerator in every GPU core and claims over 4x M4's peak AI compute, with M5 Pro and M5 Max at 307GB/s and 614GB/s. Right fix, but it's a 20x gap, not a 2x one.

Side by side

Thermals, power, and sustained load

An RTX 5090 draws 575W for the card alone, and NVIDIA specifies a 1,000W system power supply, measured on a Ryzen 9 9950X build. The PRO 6000 is rated 600W. An entire M3 Ultra Mac Studio, CPU and GPU and memory and storage and fans, is rated 480W maximum continuous. DGX Spark's GB10 is 140W. That shows up as noise, heat, and where the machine can live. For a laptop there's no dedicated-VRAM option in this class at all.

Price per usable gigabyte

Where official prices exist: DGX Spark's Founders Edition went from $3,999 to $4,699 in February 2026, which NVIDIA attributed directly to industry-wide memory supply constraints — about $37 per gigabyte of pool. The 5090's $1,999 launch MSRP is secondary-sourced, roughly $62/GB. Secondary reporting puts the 96GB PRO 6000 at $8,565 at launch, about $89/GB, and higher on NVIDIA's marketplace by mid-2026.

I won't quote a Mac Studio figure: Apple's store pages don't render prices for verification, and I'd be guessing. What moved instead was the configuration list. Tom's Hardware reported Apple withdrawing the 512GB option in March 2026 and raising the 96GB-to-256GB upgrade to $2,000 during the shortage; MacRumors reported the 256GB option going in May 2026, and the 128GB option went after it. That is why the spec page now stops at 96GB, and why any Mac Studio price-per-gigabyte you find written before mid-2026 is describing a machine Apple no longer builds. Date every price you read, mine included. TrendForce forecasts server DRAM contract prices up another 13–18% quarter over quarter in Q3 2026.

Where dedicated VRAM genuinely wins

If your work looks like this list, buy the GPU.

• Capacity at the top end, now. This used to be the Mac's strongest card and it isn't any more. A PRO 6000 matches the biggest Mac Studio you can order at 96GB and moves it at 1,792GB/s against 819GB/s. Unified memory only wins on capacity against consumer cards.
• Time to first token. About 20x on identical llama.cpp runs. For agentic coding or long-context RAG, anything re-reading a big codebase each turn, that's the difference between instant and painful.
• Batching and concurrency. The compute headroom idle during single-stream decode is what serves many requests at once. A Mac serving one user leaves little on the table; a Mac serving eight falls apart.
• Ecosystem depth. vLLM lists CUDA, ROCm and Intel XPU as GPU platforms; Apple silicon GPU support lives in a separate vllm-metal repo. TensorRT-LLM and new kernels land on CUDA first and Metal later, if ever.
• Training and fine-tuning. NVIDIA markets the PRO 6000 for local fine-tuning and DGX Spark up to 70B. The Mac story is overwhelmingly an inference story.
• ECC and repairability. The PRO 6000's 96GB is ECC-protected; Apple's isn't. You can add a card, swap one, or rent the same GPU in a cloud. Apple silicon memory is soldered — what you buy is what you have forever.

The architectures are converging from both directions. NVIDIA now ships its own unified-memory box at 128GB, and Apple has walked its ceiling back from 512GB to 96GB, so the interesting question is drifting away from Mac versus GPU and toward CUDA versus Metal on memory systems that look increasingly alike.

Who should pick which

Pick unified memory if you want to run large models at all on one machine, you value quiet and low power, you work on a laptop, or your workload is single-user chat, writing and coding.

Pick dedicated VRAM if your models fit in 32–96GB, time-to-first-token is what you feel most, you're serving multiple users, or you plan to fine-tune.

Both paths run fully on-device, so cloud privacy questions don't apply to either; the weights' own licenses are the only policy layer. If you land on Apple silicon, prefill is your weak axis — prompt caching and prefix reuse buy more than any spec bump. That's the bet Outlier makes: Apple Silicon only, macOS 12+, no Windows or Linux build.