Unified memory vs dedicated VRAM for LLM inference

Corrected 2026-09-29: Vision 3.8 is a 15.5 GB download; an earlier version gave 19.0 GB, the size of the Vision tier it replaced.

Disclosure first: I build Outlier, a local AI app that only runs on Apple Silicon, so I have an obvious interest in one side of this. I've stuck to published specs and public benchmark tables, and labeled every number I couldn't verify at the source. Where a discrete GPU wins, it wins.

One physical fact everything follows from

NVIDIA's own inference-optimization guide is blunt: for token generation, the time spent moving weights, keys, values and activations out of memory dominates the time spent computing. NVIDIA calls it "a memory-bound operation."

That splits local inference into two problems. Prompt processing, meaning reading your question, your files, your codebase, is compute-bound. Token generation, the words appearing on screen, tracks memory bandwidth far more closely than TFLOPs. Any comparison quoting one number hides half the story.

On Apple silicon the CPU and GPU share one pool: MLX's docs note arrays live in shared memory, usable by any device without transferring data. A discrete GPU has its own isolated, much faster pool, and everything must cross PCIe into it.

Capacity: what actually fits

At the March 2025 M3 Ultra launch Apple offered up to 512GB, and its claim that models over 600 billion parameters run on device was attached to that configuration. apple.com/mac-studio/specs today lists the M5 Ultra at 96GB, configurable to 256GB or 512GB, at 1.2TB/s, and the M5 Max at 36GB, configurable to 128GB, at up to 614GB/s. The largest laptop pool is the 128GB M5 Max MacBook Pro at 614GB/s. DGX Spark ships 128GB of coherent memory at 273GB/s for inference up to 200B. An RTX 5090 has 32GB; the RTX PRO 6000 has 96GB with ECC.

So at its 96GB base a Mac Studio ties the PRO 6000 on nameplate capacity and loses to it on bandwidth, 1.2TB/s against 1,792GB/s. Above 96GB, the M5 Ultra's 256GB option, and the 512GB one Apple lists for late October, are capacity that no single card on the NVIDIA pages I read offers. Against consumer cards, 128GB or 512GB against a 5090's 32GB is a four- to sixteen-fold difference and decides whether a model loads at all.

Those are hard ceilings. Spill out of VRAM and the offloaded layers cross PCIe at a fraction of GDDR7 bandwidth, and throughput falls off a cliff. Unified memory degrades gently instead. Here's what my app's tiers need:

Everything through 35B fits a 5090. The 397B tier doesn't, at any quantization worth running; on a 64GB Mac it works only because a paged mixture-of-experts loader reads experts off the SSD as the router calls for them. On my M1 Ultra that measures about 14GB peak memory at 1.59 tokens per second. The 64GB minimum in the table is not the working set: loading materializes sixty layers in one burst that needs macOS's Metal wired-memory limit raised to roughly 56GB, which the kernel won't grant on a smaller machine, and the paging itself wants file-cache headroom on top. Slow, but it runs. llama.cpp's DGX Spark file shows the same shape on a mainstream model: 59 GiB of gpt-oss-120b at roughly 2,444 tokens/sec prefill, 59 tokens/sec generation. On a 5090 it doesn't load.

One caveat: unified memory is shared. The model competes with macOS, your browser and the app, and Metal caps GPU allocation through a documented working-set limit: the widely repeated percentage-of-RAM figure is community-reported, not published by Apple. Usable budget runs well under the sticker number.

Bandwidth: speed once it fits

llama.cpp keeps two community benchmark threads in its repo, same model (LLaMA 7B Q4_0). Apple silicon: M4 Max 714 tokens/sec prefill and 70 tokens/sec generation; M2 Ultra 1,238 and 94. CUDA: RTX 5090 14,073 and 290; RTX PRO 6000 14,855 and 274.

The arithmetic is mine, not theirs: a 5090 is about 20x faster than an M4 Max at prompt processing but only about 4x faster at generating tokens. That's the split above: compute-bound prefill, where discrete GPUs are an order of magnitude ahead, and bandwidth-bound decode, where the gap collapses toward the bandwidth ratio.

Apple is closing the compute half deliberately: M5 puts a Neural Accelerator in every GPU core and claims over 4x M4's peak AI compute, with M5 Pro and M5 Max at 307GB/s and 614GB/s. Right fix, but it's a 20x gap, not a 2x one.

Side by side

Thermals, power, and sustained load

An RTX 5090 draws 575W for the card alone, and NVIDIA specifies a 1,000W system power supply, measured on a Ryzen 9 9950X build. The PRO 6000 is rated 600W. An entire Mac Studio, CPU and GPU and memory and storage and fans, is rated 480W maximum continuous. DGX Spark's GB10 is 140W. That shows up as noise, heat, and where the machine can live. For a laptop there's no dedicated-VRAM option in this class at all.

Price per usable gigabyte

Where official prices exist: DGX Spark's Founders Edition went from $3,999 to $4,699 in February 2026, which NVIDIA attributed directly to industry-wide memory supply constraints: about $37 per gigabyte of pool. The 5090's page carries a $1,999 price in its structured data (the visible price is a placeholder), roughly $62/GB. I have no vendor price for the 96GB PRO 6000.

Apple's Mac Studio buy page, read 2026-10-19, starts at $2,499 for the 36GB M5 Max and $5,499 for the 96GB M5 Ultra: about $69 and $57 per gigabyte of pool. Date every price you read, mine included. TrendForce forecasts server DRAM contract prices up 13–18% quarter over quarter in Q3 2026.

Where dedicated VRAM genuinely wins

If your work looks like this list, buy the GPU.

• Bandwidth at 96GB. A PRO 6000 matches the Mac Studio's 96GB base configuration and moves it at 1,792GB/s against the M5 Ultra's 1.2TB/s.
• Time to first token. About 20x on identical llama.cpp runs. For agentic coding or long-context RAG, anything re-reading a big codebase each turn, that's the difference between instant and painful.
• Batching and concurrency. The compute headroom idle during single-stream decode is what serves many requests at once. A Mac serving one user leaves little on the table; a Mac serving eight falls apart.
• Ecosystem depth. vLLM lists CUDA, ROCm and Intel XPU as GPU platforms; Apple silicon GPU support lives in a separate vllm-metal repo. TensorRT-LLM and new kernels land on CUDA first and Metal later, if ever.
• Training and fine-tuning. NVIDIA markets the PRO 6000 for local fine-tuning and DGX Spark up to 70B. The Mac story is overwhelmingly an inference story.
• ECC and repairability. The PRO 6000's 96GB is ECC-protected; Apple's isn't. You can add a card, swap one, or rent the same GPU in a cloud. Apple silicon memory is soldered: what you buy is what you have forever.

The architectures are converging. NVIDIA now ships its own unified-memory box at 128GB, and the Mac Studio now has 1.2TB/s of bandwidth, so the interesting question is drifting away from Mac versus GPU and toward CUDA versus Metal on memory systems that look increasingly alike.

Who should pick which

Pick unified memory if you want to run large models at all on one machine, you value quiet and low power, you work on a laptop, or your workload is single-user chat, writing and coding.

Pick dedicated VRAM if your models fit in 32–96GB, time-to-first-token is what you feel most, you're serving multiple users, or you plan to fine-tune.

Both paths run fully on-device, so cloud privacy questions don't apply to either; the weights' own licenses are the only policy layer. If you land on Apple silicon, prefill is your weak axis: prompt caching and prefix reuse buy more than any spec bump. That's the bet Outlier makes: Apple Silicon only, macOS 15+, no Windows or Linux build.