Streaming engine benchmarks: Plus + Vision tok/s on Apple Silicon

Corrected 2026-09-29: the quick answer and FAQ said ~2.1 tok/s; that is this benchmark’s protocol, and the shipping figure is 1.59.

All three of Outlier's MoE engines run on the same trick. Keep the active experts in unified memory, pull cold ones off disk. V9 paged, V10 streaming, V11 hybrid. Where they split is how they cache, and that split is what decided which one shipped. The streaming variants lost to V9 paged because their per-token streaming pattern committed far more Metal-wired memory. The raw comparison is below.

Methodology

Plus 397B-A17B: engine comparison

Look at the Wired Memory column. That's the one that mattered. V10 and V11 streamed experts per token, which committed 57–58 GB of Metal command-buffer memory during prefill, enough to take down a 32 GB Mac on the very first generation. The payoff was a ~30–40% tok/s bump over V9. V9 paged holds 3.6 GB Wired at 2.1 tok/s (this page's per-token protocol), 15× less. This tier exists to fit a 209 GB model on a small Mac, so footprint beats speed and V9 ships. On the 2.1 figure: that is this benchmark's own per-token RSS protocol. The pre-ship K-override sweep, run at the shipped K=20 setting over 5 prompts of 80 tokens with model load timed separately, measures 1.59 tok/s (per-prompt 1.35–1.65) and 14.04 GB peak. Where the two disagree the rest of this site publishes the lower, shipping-config number; the comparison below is kept on one protocol so the engines stay comparable to each other.

Vision 35B-A3B: engine comparison

This is Vision 3.6, the Qwen3.6-35B-A3B tier that was retired in v1.11.796. The shipping Vision tier is Vision 3.8 (Qwen3.8-27B).

Vision 3.6 was a little different. There was a V10 opt-in for people on tight RAM, lower footprint at the cost of speed (~6.6 tok/s), but the default was V9 at 16.31 tok/s on 24 GB+ Macs. That old 16 GB MacBook Air result leaned on V11's LRU specifically. With V11 gone, the practical floor for Vision 3.6 was 24 GB, and Vision 3.8 needs 24 GB too.

Soak verdict: what passed and what didn't

Here's what the 2026-05-09 soak verdict on V11 turned up over 9 cycles. The quality numbers were fine. The memory number wasn't.

• 135/135 turns coherent. No output collapse, no repetition loops.
• 0 crashes, 0 segfaults across both Plus N=4 (100 turns) and Vision N=2 (35 turns).
• Engine-swap V11→V9→V11 round-trip held up: post-swap tok/s 1.51 vs pre-swap 1.36 in the V11 soak protocol (not the shipping-config figure), no regression.
• Idle-resume across a 5-minute sleep came back clean. No orphan threads, no leaked tensors.
• But memory pressure on 50-turn Plus hit 14.34 GB peak vs smoke 7.34 GB. That's the regression that got it retired.

How to reproduce

1. Install Outlier. You'll want Pro (the $49 licence), which includes Plus 397B: outlier.host
2. Open Settings → Engine Mode. Plus runs on Capacity (V9 paged), and any old V10/V11 preference auto-migrates to Capacity for you.
3. Run a multi-prompt set and grab tok/s from the response footer.
4. While it's generating, sample OS RSS with ps -o pid,rss,command -p $(pgrep -f outlier-cli).

Want to reproduce V10/V11 themselves? You can't from a current DMG, since neither ships anymore. The source lives in repo history before commit 525298f (v1.11.71) and the v1.11.73 V10 removal.

Questions about the streaming-engine numbers