Mac vs PC for local AI: unified memory vs a discrete GPU

Full disclosure: I build Outlier, a local-AI app that only runs on Apple Silicon, so I have a stake in one side of this. I've tried to make the PC case as strongly as I'd want someone to make mine. On raw speed the discrete GPU wins.

One architectural difference drives everything else

A discrete GPU has its own memory: a fixed pool of VRAM on a very wide bus. Apple Silicon has one pool shared by CPU and GPU — MLX, Apple's framework, notes that its arrays live in shared memory and run on either device without a copy. On the PC the model has to fit in VRAM or performance falls off a cliff; spill into system RAM over PCIe and generation collapses. On the Mac, "VRAM" is whatever you bought, minus what macOS is using. No wall at 32 GB — and no 1,792 GB/s bus.

What actually fits

Weights are the floor, not the total — you also need the KV cache, which grows with context, plus the runtime and the OS. Apple's ML research team measured footprints on a 24 GB MacBook Pro — Qwen 8B is 17.46 GB at BF16 and 5.61 GB at 4-bit, Qwen 14B 4-bit is 9.16 GB, GPT-OSS 20B at MXFP4 is 12.08 GB, and a Qwen 30B MoE 4-bit is 17.31 GB.

Our own tiers line up. Nano (4B) is 2.37 GB and wants 6 GB of RAM; Lite (9B) 5.04 GB at 12 GB; Quick (26B-a4b) 15.61 GB at 16 GB; Core and Code (both 27B) 15.13 GB at 24 GB; Vision (35B-a3b) 19.0 GB at 24 GB; Plus (397B-a17b) is 209 GB on a 64 GB machine, paging MoE experts off the SSD at ~11 GB resident. Against a 32 GB card, a 4-bit model up to about 30B is comfortable and quick; a 70B isn't. That's the capacity argument.

Speed: bandwidth is the ceiling, and the PC has more of it

Token generation is bound by memory bandwidth, not compute — Apple's research group says so, and its measurements show it. M4 to M5 lifted bandwidth from 120 GB/s to 153 GB/s and produced time-to-first-token speedups of 3.33×–4.06×, but generation speedups of only 1.19×–1.27×.

So compare bandwidth. Apple lists 410 GB/s for the 32-core M4 Max, 546 GB/s for the 40-core and 819 GB/s for M3 Ultra; on laptops, 307 GB/s for M5 Pro and 614 GB/s for M5 Max. NVIDIA's GeForce news pages put the RTX 5090 at about 1,792 GB/s — the spec page omits it. Call that 2.2× an M3 Ultra, 2.9× an M5 Max.

Prefill is wider still. A third-party community suite puts an H100 at ~7,760 tokens/sec of prompt evaluation on Llama 3 8B against ~1,024 for an M2 Ultra, while generation was 144 vs 76. Long-context RAG and agent loops live on prefill, so that's the gap you feel. Apple claims M5 processes prompts up to 4× faster than M4.

Thermals, power, and sustained load

NVIDIA rates the RTX 5090 at 575 W and recommends a 1,000 W supply — for the card alone. Apple lists 480 W as maximum continuous draw for an entire Mac Studio. Over hours that's fan noise, room temperature and an electricity bill. Nor is there a real laptop equivalent: a discrete-GPU laptop with useful VRAM is heavy, loud and thermally limited, where a MacBook Pro carries 128 GB on battery.

Price per usable gigabyte, and the upgrade path

Apple charges steeply per memory tier and solders it: day-one capacity is what you have until you buy a new computer. But getting past 32 GB on the PC isn't cheap either. NVIDIA's workstation documentation describes an RTX PRO 6000 Blackwell at 96 GB of GDDR7 with ECC at full bandwidth — capacity and bandwidth together, which no Mac matches, at a professional price. NVIDIA's own unified-memory box, DGX Spark, has 128 GB but only 273 GB/s, below an entry M4 Max. AMD documents the Ryzen AI Max+ 395 at 128 GB unified, 96 GB reassignable as VRAM.

Apple announced the M4 Max Mac Studio at $1,999 in March 2025 and the March 2026 MacBook Pro line from $2,199 to $3,899. Third-party reporting describes a 2026 DRAM shortage pushing Apple prices and GPU street prices up, so check live listings. "The PC is cheaper" is worth re-testing, not repeating.

Side by side

Where the PC with a discrete GPU genuinely wins

These advantages are real, and several are decisive.

• Raw generation speed. 1,792 GB/s against 819 GB/s isn't close. Any model that fits streams out faster on the PC.
• Prefill. The prompt-processing gap is far bigger than the generation gap, and agentic work is made of prefill.
• The CUDA software moat. The biggest practical advantage. vLLM treats CUDA, ROCm and Intel XPU as first-class; Apple Silicon lives in a separate plugin, vLLM-Metal, at v0.2.0 as of April 2026. bitsandbytes gives NVIDIA official support and calls Apple Silicon experimental — its macOS wheel is CPU-only. Unsloth describes Apple/MLX support as in progress. New quantization schemes ship CUDA-first.
• Fine-tuning and training. The LoRA/QLoRA/8-bit-optimizer stack is CUDA kernels. Training on your own data is well-trodden on the PC, newer on the Mac.
• Image, video and diffusion. ComfyUI runs on Apple silicon but documents that path as needing a PyTorch nightly build. NVIDIA is the reference implementation.
• Upgradeability. Start mid-range, upgrade later, or add a second card for more VRAM and compute. No Mac does that.
• Production parity. Linux gives you containerized stacks that mirror what runs on a server.
• A used market at every tier. The Mac path has a hard entry price.

The honest counterweight: AMD isn't a clean escape — bitsandbytes treats ROCm as experimental too, and its supported-GPU matrix is narrower and more version-sensitive than CUDA's. Driver and toolkit versions are a recurring tax; Ollama documents a 550+ NVIDIA driver requirement plus separate ROCm and Vulkan paths, where the Mac is one architecture, one driver.

Who should pick which

Build the PC if you want to fine-tune or train, you work with diffusion and video models, your models fit in 32 GB and you want maximum tokens per second, your workload is prefill-heavy, or local dev needs to mirror a Linux/CUDA production target.

Buy the Mac if you need more model than a consumer GPU holds, you want local AI on a laptop and on battery, you care about noise and power at your desk, or you'd rather install an app than maintain a driver stack.

Privacy isn't a differentiator. Both run inference on your own machine, so any privacy claim belongs to the software, not the silicon. And to be plain about my stake: Outlier is Apple Silicon only, M1 or later, macOS 12+ — no Windows, Linux or Intel build. On the PC side, llama.cpp, Ollama and LM Studio have you covered. Pick hardware for the work, then software that runs on it.