GGUF vs MLX: how the two local model formats actually differ

Disclosure first: I build Outlier, a Mac app that runs MLX-format models on-device, so I am on one side of this particular comparison. That's why I'd rather explain the mechanism than sell a conclusion. This page is about the file formats; MLX vs llama.cpp compares the runtimes that read them. Most GGUF-vs-MLX arguments go wrong in the first sentence, because the two things being compared aren't the same kind of thing.

GGUF is a container, not an engine

GGUF's own spec calls it a file format for storing models for inference with GGML and GGML-based executors. Georgi Gerganov created it, he also wrote llama.cpp, and it replaced the earlier GGML, GGMF and GGJT formats. The stated design goals: single-file deployment, extensibility, mmap compatibility, ease of use, and full information, meaning everything needed to load a model lives inside the file. No external config or loader script.

That last property matters most in practice. Hugging Face draws the contrast directly: unlike tensor-only formats such as safetensors, GGUF encodes both the tensors and a standardized set of metadata. A key-value header carries architecture, tokenizer and hyperparameters, so an old loader can open a new file and skip fields it doesn't recognize. The spec is on v3: v2 widened count fields to uint64, v3 added big-endian support.

The type table is where GGUF's reach shows: roughly forty tensor and quantization type codes, from F32, F16 and BF16 down through the Q*_K k-quants, the IQ* importance-matrix quants, ternary TQ1_0/TQ2_0, and MXFP4. llama.cpp advertises 1.5-bit through 8-bit integer quantization and ships an imatrix tool that computes an importance matrix over a text dataset to improve quantized quality. The repo makes no quantified claim about how much it helps; neither will I.

MLX is a runtime, not a file format

MLX is an array framework for machine learning on Apple silicon, from Apple's own machine learning research group. It's NumPy-shaped rather than model-shaped: arrays, lazy computation, dynamic graphs, and Python, C++, C and Swift APIs. The defining choice is that arrays live in shared memory, so operations run on CPU or GPU with no transfer between them. Where both already share one pool of RAM, that removes a copy other stacks pay for.

MLX LM is the language-model package on top. Weights are safetensors plus a config, not a bespoke container. You get mlx_lm.generate, mlx_lm.chat, mlx_lm.convert (with -q for quantized conversion), mlx_lm.cache_prompt, LoRA and full fine-tuning, and distributed work through mx.distributed. There's an OpenAI-style HTTP server too, though its own docs say it isn't recommended for production because it only implements basic security checks.

The hardware floor is real: Apple silicon, macOS 14.0 or later, native Python 3.10+. Intel Macs are out. CUDA-on-Linux and CPU-only Linux builds now exist, so "MLX is Apple-only" isn't strictly true, but the Mac path is what everything is tuned for.

Container vs runtime, side by side

Where each one genuinely wins

GGUF wins on reach, and it isn't close. 212,991 models carry the gguf library tag on Hugging Face against 26,701 for mlx: roughly eight to one. When a new open model drops, the community usually has GGUF builds at half a dozen bit-widths within hours, and the same file runs on your M-series laptop, your NVIDIA box, an AMD card, or plain CPU. One artifact, every machine you own. GGUF also gets Hugging Face tooling with no MLX counterpart: an inline metadata viewer on model and files pages, and a JavaScript parser (@huggingface/gguf) that reads a remote GGUF's metadata without downloading it. Conversion Spaces exist for both: ggml-org/gguf-my-repo and mlx-community/mlx-my-repo.

MLX wins on hardware depth and on training. Unified-memory arrays with no CPU/GPU copy is the design you get when the framework authors work at the company that made the chip. MLX also does what a pure inference format can't: LoRA and full fine-tuning on your own Mac, distributed training over mx.distributed, and a Python API that makes a model something you modify, not just run. Its learned-quantization suite is unusually sophisticated for a local stack: DWQ, AWQ, GPTQ and per-layer dynamic quantization, cascadable, aimed at 2-to-4-bit where small-RAM Macs live. The docs are candid: DWQ takes longer than dynamic quantization but typically produces better results, and distilling to 6- or 8-bit often doesn't work well. MLX Swift is the sane route for native Mac or iOS apps.

They're not fully walled off. mlx.core.load reads .gguf alongside .npy, .npz and .safetensors. But read the caveat: unsupported GGUF quantization formats are cast to float16 on load. A k-quant doesn't execute natively under MLX: it decompresses, and the memory saving evaporates.

The benchmark problem

The most-cited head-to-head numbers come from Ollama, which moved its Apple silicon path onto MLX in preview on 2026-03-30. Their published figures, tested on M5 hardware: prefill 1,154 tok/s on 0.18 versus 1,810 on 0.19; decode 58 versus 112 tok/s. That looks like a decisive format win. It isn't. Version 0.18 ran a Q4_K_M GGUF and 0.19 ran NVFP4 through MLX: engine, version and quantization scheme all changed at once.

The preview also carried hard limits: a Mac with more than 32 GB of unified memory, and only one accelerated model at first. Ollama has kept investing: a 2026-06-11 update added NVIDIA's NVFP4 4-bit format, up to 20% faster output through fused Metal kernels, and a snapshot system for agent workloads, with the claim that NVFP4 roughly halves the quality loss of 4-bit quantization relative to unquantized BF16, measured on Gemma 4 12B. Still not a clean A/B. I couldn't find any first-party benchmark isolating GGUF against MLX at matched bit-width on identical hardware, so treat every "MLX is X% faster" number as confounded. So I ran one.

I measured it: the same model, both ways, on one Mac

On 3 November 2026 I ran Qwen3-8B at 4-bit through both, on my M1 Ultra (64 GB, macOS 26.3.1). Each side used its publisher's own 4-bit build: Qwen's Q4_K_M GGUF, read by llama.cpp, and the mlx-community 4-bit MLX weights, run by MLX LM at the exact versions Outlier 1.11.990 ships. Both got the same 515 prompt tokens, greedy decoding, exactly 128 new tokens, a fresh cache every run and a 4,096-token context. One warm-up, then five measured runs each, taking turns going first, with nothing else generating on the Mac.

Figures are medians of the five runs. What it means in use: llama.cpp starts answering sooner, and MLX writes faster once it starts. On this prompt the two cross at about 130 tokens, so a one-line answer arrives first from llama.cpp and a paragraph or longer arrives first from MLX. Long prompts, like a pasted document or a long chat, lean on llama.cpp’s faster prompt reading; long answers lean on MLX’s faster writing.

What it doesn’t show is which format is better in general. The two 4-bit schemes differ: the GGUF keeps some tensors at 6-bit and is 9% bigger, and on a Mac writing speed tracks how many bytes are read per token. The engines and their kernels differ too, and this is one model on one Mac. I measured speed, first token, memory and size, not answer quality. New llama.cpp and MLX releases will move these numbers, which is why the date is on them.

Which one belongs in your setup

Pick GGUF if you run models on more than one kind of machine, if you want whatever came out this week, if you have an Intel Mac or a pre-Sonoma macOS install, or if you want the extreme low-bit end of the curve with imatrix calibration. It's also the safer cross-platform default: llama.cpp has bindings for nearly every language you'd write in.

Pick MLX if you're on Apple silicon and staying there, if you want to fine-tune rather than only infer, if you're building a native app in Swift, or if you want the learned-quantization methods clawing back quality at 2–4 bit.

Pick neither, directly, if you just want to use a model. Neither GGUF nor MLX is a product a normal person can use: no installer, no GUI, no model picker, no update mechanism. Reaching a working chat window means assembling a runtime like Ollama or LM Studio, or writing code. Outlier is one example of the other approach: an Apple-silicon-only Mac app with six fixed tiers where the format question is decided for you rather than configured. You pick Nano or Core, not a quant type. That's a genuine trade: the 212,991-model catalog and every knob above, given up in exchange for not touching any of it. If turning those knobs is the fun part, run the frameworks directly. It costs nothing.