Apple unified memory vs an NVIDIA RTX GPU for local LLMs

Disclosure up front: I build Outlier, a Mac-only local AI app, so my day job sits on the Apple side of this. What follows is what I'd have wanted to read before spending my own money, including where a discrete GPU is the better machine.

Two ways to hold a model

Apple silicon puts one pool of LPDDR5X on the SoC, addressed directly by CPU, GPU and Neural Engine. Apple describes the M3 Ultra's pool as "the most high-bandwidth, low-latency memory ever available in a personal computer." A Mac Studio starts at 36GB with M4 Max and goes to 128GB; M3 Ultra starts at 96GB.

An RTX card inverts that — a smaller pool of quicker memory bolted to far more compute. The 5090 has 32GB of GDDR7 on a 512-bit interface, 1,792 GB/sec of bandwidth and 21,760 CUDA cores at $1,999; below it, 16GB on the 5080 ($999) and 5070 Ti ($749), 12GB on the 5070 ($549). Structure matters more than specs here: a discrete GPU needs a host-to-device copy, so weights cross PCIe into VRAM and whatever doesn't fit stays behind. In Apple's MLX there's no transfer step at all — arrays live in shared memory.

Capacity versus bandwidth

Inference has two phases with different bottlenecks, and conflating them is where hardware advice goes wrong. llama.cpp's benchmark thread puts it plainly: prompt processing at large batch size "is compute bound... speed depends on how many FLOPS you can utilize," while generating one token at a time is bandwidth-bound. Bandwidth sets tokens per second; compute sets how long you wait first.

Apple tops out at 819 GB/s; a 5090 moves 1,792 GB/s. Community-submitted llama.cpp scoreboard results, all Llama 2 7B Q4_0, put a 5090 at roughly 14,000 tok/s prefill and 290 tok/s generation against about 1,240 / 94 on an M2 Ultra and 714 / 70 on an M4 Max — near 11x and 3x. Crowd-sourced, one small model: directional, not vendor-certified.

What actually fits

Here's the wall. A 70B model at 4-bit is roughly 40GB of weights before any KV cache, so it doesn't fit in 32GB. llama.cpp will offload the overflow to system RAM over PCIe and throughput collapses. Apple claims an M3 Ultra Mac Studio runs 600B-parameter models entirely in memory.

Concrete sizes, using Outlier's tiers since I have exact figures. Nano 4B, 2.37GB download, 6GB RAM; Lite 9B, 5.04GB / 12GB; Quick 26B-a4b, 15.61GB / 16GB; Core and Code 27B, 15.13GB / 24GB; Vision 35B-a3b, 19.0GB / 24GB. All fit a 32GB 5090 and would run quicker there than on my M1 Ultra. The tier above flips it: Plus is 397B-a17b, a 209GB download that runs on a 64GB Mac by streaming experts off the SSD at about 11GB peak RSS — 2.1 tok/s, which is slow.

One caveat cuts against Apple: macOS won't hand a process the whole pool. Metal exposes a recommended maximum working set, commonly cited as roughly 75% of RAM and adjustable via iogpu.wired_limit_mb. That figure comes from an Apple Developer Forums thread, not documentation — budget below the sticker number.

Side by side

Where the NVIDIA card genuinely wins

Prompt processing isn't close, and it's the argument I'd lead with. Anything that re-reads a long context every turn — agent loops, RAG, codebase Q&A, tool calls — pays that prefill cost repeatedly, and the gap grows with context.

Software is the second honest win. CUDA is the reference platform: vLLM, TensorRT-LLM and flash-attention target it first, and new architectures and quantization formats have a working CUDA path on day one. vLLM's own docs still route macOS builds to CPU unless you add a Metal plugin. Fine-tuning is the same story — LoRA, QLoRA and full fine-tunes work out of the box with mainstream PyTorch; the Mac path goes through MLX with fewer recipes.

NVIDIA isn't conceding capacity, either. DGX Spark pairs a 20-core Arm CPU with a Blackwell GPU over 128GB of coherent LPDDR5x at 273 GB/s on a 140W SoC; NVIDIA's marketplace has listed it at $4,699 — reported, not confirmed. In May 2026 NVIDIA announced RTX Spark — Grace CPU plus Blackwell RTX GPU over NVLink-C2C, "up to 128GB of unified memory," 1 petaflop of AI performance, and a claim of 120B-parameter models at 1M tokens of context — shipping this fall. Unified memory won't be an Apple-only idea by autumn.

Power, thermals, and price per usable gigabyte

No instrumented sustained-load numbers here — just published envelopes. Apple rates the entire Mac Studio at 480W maximum continuous power. NVIDIA's product listing gives the 5090 a 575W total graphics power for the card alone — off a listing, not a spec table I pulled directly. That means a large PSU, real airflow and fan noise on long runs: a category difference, not a spec difference.

On price, $1,999 buys either a complete Mac Studio with 36GB or one 5090 with 32GB: roughly $55 per gigabyte of unified memory versus $62 per gigabyte of VRAM plus a host. If the ~75% cap holds, Apple's effective number is nearer $74 per usable gigabyte. The math swings to Apple only above 32GB, where consumer GeForce has nothing to sell you — NVIDIA's answer is the RTX PRO 6000 Blackwell at 96GB and up to 600W: workstation money for one card.

Upgrade paths are the underestimated asymmetry: a card is a slot decision you can revisit, resell or double — though by third-party accounts GeForce has no NVLink, so a second card is PCIe-bound with no automatic VRAM pooling. Apple's memory is soldered at purchase, so buying too little is permanent. Tom's Hardware reported in March 2026 that Apple pulled the 512GB M3 Ultra option and raised the 256GB upgrade to $2,000; the same shortage reportedly pushed NVIDIA's SUPER refresh toward 2027.

Who should pick which

Pick an RTX card if your models fit in 32GB and speed is the point; if you fine-tune; if you need day-one support for new formats; if you're prototyping for cloud deployment; or if you own a PC and want the cheapest credible entry — a $549 card, not a whole machine.

Pick Apple silicon if you want models that don't fit in 32GB at all; if the machine is a laptop or sits where 575W of card is a non-starter; if you build or use native macOS software, since CUDA is licensed only for systems with NVIDIA GPUs.

Both share the part I care about most. With llama.cpp, MLX or Ollama, inference runs entirely on your machine — no prompts leave the box, and there's no training policy to compare because there's no vendor endpoint.

From the Apple side, for scale: Core 27B runs at 20.7 tok/s on my M1 Ultra and Nano 4B hits 32 tok/s on an M4 MacBook Air — quicker than you read, with no fans audible. That's a case for capacity per watt, not a claim of winning.