Apple unified memory vs an NVIDIA RTX GPU, for local LLM inference

Disclosure up front: I build Outlier, a Mac-only local AI app, so my day job sits on the Apple side of this. What follows is what I'd have wanted to read before spending my own money, including where a discrete GPU is flatly the better machine.

Two ways to hold a model

Apple silicon puts one pool of LPDDR5X on the SoC package, addressed directly by the CPU, GPU and Neural Engine. Apple's M3 Ultra announcement calls the design "the most high-bandwidth, low-latency memory ever available in a personal computer." The consequence shows up in MLX, Apple's array framework: arrays live in shared memory, so instead of copying data to a device you pick which device runs each operation. There's no host-to-device transfer step.

A discrete GPU inverts that: on the 5090, 32GB of GDDR7 on a 512-bit bus attached to a far larger GPU (21,760 CUDA cores), across PCIe. Weights get copied into VRAM first, and whatever doesn't fit stays on the far side of that link. llama.cpp can run models bigger than VRAM in CPU+GPU hybrid mode, but that's a fallback, not a plan.

Capacity versus bandwidth: the actual trade

Inference has two phases with different bottlenecks, and conflating them is where hardware advice goes wrong. llama.cpp's benchmark thread states it plainly: prompt processing at large batch sizes is compute-bound, speed depends on how many FLOPS you can use, while generating one token at a time is bandwidth-bound.

On bandwidth, NVIDIA lists 1,792 GB/s for the RTX PRO 6000 (its RTX 5090 page gives the 512-bit bus but no bandwidth figure), against 1.2 TB/s on an M5 Ultra and 614 GB/s on an M5 Max: about 1.5x the top Mac and about 2.9x the M5 Max. On compute it's far wider. Community-submitted results on llama.cpp's official scoreboards (same llama-bench harness, 7B Q4_0 (the Apple thread uses LLaMA 7B, the CUDA thread Llama 2 7B) same shape, near-identical cost): put the 5090 near 14,000 tok/s prefill and 290 tok/s generation, against roughly 1,240 / 94 for an M2 Ultra and 714 / 70 for an M4 Max. Roughly 11x on prefill and 3x on generation, though it's crowd-sourced on one small model, so read it as directional.

What actually fits

Capacity flips the ordering. Outlier's tiers make a useful yardstick because the sizes are measured: Nano 4B is a 2.37GB download needing 6GB of RAM, Lite 9B is 5.04GB / 12GB, Core 27B is 15.13GB / 24GB, Vision 3.8 27B is 15.5GB / 24GB. All four fit a 32GB card. Nano and Lite fit anything. The 27B tiers at 15.13GB need a 16GB card and leave under a gigabyte for KV cache plus framebuffer, so 16GB is the floor rather than a comfortable fit; Vision 3.8 at 15.5 GB needs more than 16GB. On a 12GB 5070 both the 27B tiers and Vision overflow outright. 15.13GB of weights doesn't fit 12GB of VRAM before you add anything.

Above that it changes fast. A 70B model at 4-bit is roughly 40GB of weights before KV cache, so it doesn't fit a 5090 at all: llama.cpp spills the rest over PCIe and throughput collapses. Apple's March 2025 announcement said the Mac Studio with M3 Ultra could run models of over 600 billion parameters directly on device, with memory configurable up to 512GB. Apple's Mac Studio spec page today lists 256GB or 512GB as the options above the 96GB base, on the M5 Ultra with the 80-core GPU, and 128GB as the ceiling on the M5 Max. A 600B model at 4-bit is on the order of 300GB of weights, so that class of model needs the 512GB configuration. What a 256GB machine buys you is still real (on this page's own arithmetic (70B at 4-bit ≈ 40GB) it's a few hundred billion parameters, and no GeForce card comes close at any price).

One caveat cuts against Apple: you don't get the whole pool. macOS caps the GPU working set through Metal's recommendedMaxWorkingSetSize. The percentage of RAM people quote for it comes from community threads, not Apple documentation, so I won't repeat one: budget below the number on the box.

Side by side

Where the NVIDIA card genuinely wins

Prompt processing isn't close, and it's the argument I'd lead with. Anything that re-reads long context (agent loops, RAG, codebase Q&A, tool calls) pays that cost every turn, and the gap grows with context.

• The software is written for CUDA first. vLLM, TensorRT-LLM and flash-attention target it on day one. vLLM's own docs still route a macOS source build to CPU unless you add a separate Metal plugin.
• Fine-tuning works out of the box. LoRA, QLoRA and full fine-tunes run on the mainstream PyTorch stack; MLX can do it with a smaller library surface and fewer tested recipes.
• Cheap entry, real resale. The $549 RTX 5070 list price gets you started on a PC you already own, and the same CUDA path runs on rented cloud GPUs later. Apple's floor is a whole machine.
• NVIDIA isn't conceding capacity. DGX Spark pairs a 20-core Arm CPU with a Blackwell GPU over 128GB of coherent LPDDR5x at 273 GB/s on a 140W SoC, and its page now also advertises RTX Spark for Windows PCs, marked "Out Now". Unified memory isn't an Apple-only idea.

The counterweights are structural. Past 32GB on one card you're into the RTX PRO 6000 Blackwell: NVIDIA's page lists 96GB of GDDR7 with ECC at 600W, workstation money for a single card. The RTX 5090 has no NVLink (NVIDIA's spec page lists it as "No"), so by third-party accounts a two-card rig is PCIe-bound with no VRAM pooling. And CUDA is licensed only for systems with NVIDIA GPUs.

Power, price per usable gigabyte, and upgrade paths

I don't have instrumented sustained-load numbers for both platforms, so here are the published envelopes. Apple rates the entire Mac Studio at 480W maximum continuous power. NVIDIA's own RTX 5090 product page gives the card a 575W total graphics power with a 450W-or-greater PCIe Gen 5 cable. The card alone exceeds the Mac's whole-machine ceiling, and around it you still need a chassis, airflow and tolerance for fan noise.

Price per gigabyte is less lopsided than either camp suggests. $2,499 buys a complete Mac Studio with 36GB: about $69/GB, and more per usable gigabyte once macOS takes its share. $1,999 buys one 5090 with 32GB: about $62/GB, nearly all of it available for weights, plus a host machine. A $549 5070 is about $46/GB, the cheapest rate here. One asymmetry to flag: the Mac number is a whole computer, while the GeForce numbers are NVIDIA's list prices for the card alone. NVIDIA itself blamed memory supply constraints when it raised the DGX Spark price in February 2026, so treat $46/GB and $62/GB as floors, not quotes: price the actual cards before you compare. Apple's advantage isn't the rate anyway: it's the ceiling.

Upgrade paths aren't symmetric either. A GPU is a component you can sell, swap or double up; Apple's memory is soldered at purchase, so buying too little is permanent.

Who should pick which

Pick an RTX card if your models fit in 16–32GB and speed is the point; if you fine-tune; if you need day-one support for new formats; or if you own a PC already and want the cheapest entry.

Pick Apple silicon if you want models a consumer card can't hold; if you want one quiet machine that does everything; if you're already on macOS; or if 575W of graphics card is a non-starter where the machine lives.

Either way it's genuinely local. With llama.cpp, MLX or Ollama, inference runs on your machine and no prompt data goes to Apple or NVIDIA: there's no training-data policy to compare, because there's no vendor endpoint. On a Mac, Outlier is one packaged version of that: Apple Silicon only; Nano and Lite free with no account, no caps, and the remaining tiers behind Pro. On a PC none of it applies: llama.cpp or Ollama is where I'd start.