Apple unified memory vs an NVIDIA RTX GPU, for local LLM inference

Disclosure up front: I build Outlier, a Mac-only local AI app, so my day job sits on the Apple side of this. What follows is what I'd have wanted to read before spending my own money — including where a discrete GPU is flatly the better machine.

Two ways to hold a model

Apple silicon puts one pool of LPDDR5X on the SoC package, addressed directly by the CPU, GPU and Neural Engine. Apple calls it the highest-bandwidth, lowest-latency memory ever put in a personal computer. The consequence shows up in MLX, Apple's array framework: arrays live in shared memory, so instead of copying data to a device you pick which device runs each operation. There's no host-to-device transfer step.

A discrete GPU inverts that — on the 5090, 32GB of GDDR7 on a 512-bit bus attached to a far larger GPU (21,760 CUDA cores), across PCIe. Weights get copied into VRAM first, and whatever doesn't fit stays on the far side of that link. llama.cpp can run models bigger than VRAM in CPU+GPU hybrid mode, but that's a fallback, not a plan.

Capacity versus bandwidth: the actual trade

Inference has two phases with different bottlenecks, and conflating them is where hardware advice goes wrong. llama.cpp's benchmark thread states it plainly: prompt processing at large batch sizes is compute-bound — speed depends on how many FLOPS you can use — while generating one token at a time is bandwidth-bound.

On bandwidth the gap is about 2x: 1,792 GB/s against 819 GB/s on an M3 Ultra, or 410–546 GB/s on an M4 Max. On compute it's far wider. Community-submitted results on llama.cpp's official scoreboards — same llama-bench harness, 7B Q4_0 (the Apple thread uses LLaMA 7B, the CUDA thread Llama 2 7B — same shape, near-identical cost) — put the 5090 near 14,000 tok/s prefill and 290 tok/s generation, against roughly 1,240 / 94 for an M2 Ultra and 714 / 70 for an M4 Max. Roughly 11x on prefill and 3x on generation, though it's crowd-sourced on one small model, so read it as directional.

What actually fits

Capacity flips the ordering. Outlier's tiers make a useful yardstick because the sizes are measured: Nano 4B is a 2.37GB download needing 6GB of RAM, Lite 9B is 5.04GB / 12GB, Core and Code 27B are 15.13GB / 24GB, Vision 35B-a3b is 19.0GB / 24GB. All five fit a 32GB card. Nano and Lite fit anything. The 27B tiers at 15.13GB need a 16GB card and leave under a gigabyte for KV cache plus framebuffer, so 16GB is the floor rather than a comfortable fit; Vision at 19.0GB needs more than 16GB. On a 12GB 5070 both the 27B tiers and Vision overflow outright — 15.13GB of weights doesn't fit 12GB of VRAM before you add anything.

Above that it changes fast. A 70B model at 4-bit is roughly 40GB of weights before KV cache, so it doesn't fit a 5090 at all — llama.cpp spills the rest over PCIe and throughput collapses. Apple claims the M3 Ultra Mac Studio runs models of over 600 billion parameters entirely in memory — but read the config that claim is attached to. Apple's March 2025 announcement ties it to a Mac Studio configurable up to 512GB, and that option is gone: Tom's Hardware, MacRumors and Cult of Mac all reported in March 2026 that Apple pulled the $4,000 512GB upgrade during the memory shortage, leaving 256GB as the most you can order. A 600B model at 4-bit is on the order of 300GB of weights, so the headline figure is not achievable on any Mac Studio currently for sale. What a 256GB machine still buys you is real — on this page's own arithmetic (70B at 4-bit ≈ 40GB) it's a few hundred billion parameters, and no GeForce card comes close at any price — but the number to quote is 256GB, not 512GB.

One caveat cuts against Apple: you don't get the whole pool. macOS caps the GPU working set through Metal's recommendedMaxWorkingSetSize, commonly cited at around 75% of RAM and adjustable via iogpu.wired_limit_mb. That figure traces to an Apple Developer Forums thread, not Apple documentation — treat it as unconfirmed, but budget below the number on the box.

Side by side

Where the NVIDIA card genuinely wins

Prompt processing isn't close, and it's the argument I'd lead with. Anything that re-reads long context — agent loops, RAG, codebase Q&A, tool calls — pays that cost every turn, and the gap grows with context.

• The software is written for CUDA first. vLLM, TensorRT-LLM and flash-attention target it on day one. vLLM's own docs still route a macOS source build to CPU unless you add a separate Metal plugin.
• Fine-tuning works out of the box. LoRA, QLoRA and full fine-tunes run on the mainstream PyTorch stack; MLX can do it with a smaller library surface and fewer tested recipes.
• Cheap entry, real resale. $549 at launch MSRP gets you started on a PC you already own, and the same CUDA path runs on rented cloud GPUs later. Apple's floor is a whole machine.
• NVIDIA isn't conceding capacity. DGX Spark pairs a 20-core Arm CPU with a Blackwell GPU over 128GB of coherent LPDDR5x at 273 GB/s on a 140W SoC. And RTX Spark, announced 31 May 2026 and shipping this fall through the major Windows OEMs, brings up to 128GB of unified memory to ordinary PCs. Unified memory won't be an Apple-only idea by autumn.

The counterweights are structural. Past 32GB on one card you're into the RTX PRO 6000 Blackwell — NVIDIA's page lists 96GB of GDDR7 with ECC at 600W, workstation money for a single card. GeForce has no NVLink, so by third-party accounts a two-card rig is PCIe-bound with no VRAM pooling. And CUDA is licensed only for systems with NVIDIA GPUs.

Power, price per usable gigabyte, and upgrade paths

I don't have instrumented sustained-load numbers for both platforms, so here are the published envelopes. Apple rates the entire Mac Studio at 480W maximum continuous power. NVIDIA's own RTX 5090 product page gives the card a 575W total graphics power with a 450W-or-greater PCIe Gen 5 cable. The card alone exceeds the Mac's whole-machine ceiling, and around it you still need a chassis, airflow and tolerance for fan noise.

Price per gigabyte is less lopsided than either camp suggests. $1,999 buys a complete Mac Studio with 36GB — about $56/GB, or nearer $74 per usable gigabyte if the ~75% cap holds. The same $1,999 buys one 5090 with 32GB: about $62/GB, nearly all of it available for weights, plus a host machine. A $549 5070 is about $46/GB, the cheapest rate here. One asymmetry to flag, because it cuts against my own side: the Mac number is a current price, while the GeForce numbers are January 2025 launch MSRP. GDDR7 cards are exposed to the same DRAM squeeze that pushed Apple's 256GB upgrade to $2,000, so treat $46/GB and $62/GB as floors, not quotes — price the actual cards before you compare. Apple's advantage isn't the rate anyway — it's the ceiling.

Upgrade paths aren't symmetric either. A GPU is a component you can sell, swap or double up; Apple's memory is soldered at purchase, so buying too little is permanent. That got worse in 2026: as noted above, the $4,000 512GB M3 Ultra option is gone and the 256GB upgrade rose to $2,000 during the memory shortage, and Apple's 512GB configuration URL no longer resolves. Those are third-party figures — but the ceiling moved down while the price moved up, so the "512GB Mac" line you still see quoted is stale, and buying too little memory is a decision you can't revisit.

Who should pick which

Pick an RTX card if your models fit in 16–32GB and speed is the point; if you fine-tune; if you need day-one support for new formats; or if you own a PC already and want the cheapest entry.

Pick Apple silicon if you want models a consumer card can't hold; if you want one quiet machine that does everything; if you're already on macOS; or if 575W of graphics card is a non-starter where the machine lives.

Either way it's genuinely local. With llama.cpp, MLX or Ollama, inference runs on your machine and no prompt data goes to Apple or NVIDIA — there's no training-data policy to compare, because there's no vendor endpoint. On a Mac, Outlier is one packaged version of that: Apple Silicon only; Nano and Lite free with no account, no caps, and the remaining tiers behind Pro. On a PC none of it applies — llama.cpp or Ollama is where I'd start.