Gemma 4 vs Qwen 3.6 on a Mac: which one to run

I ship both of these in Outlier. Gemma 4 26B-A4B is the Quick tier and Qwen 3.6 27B is the Core tier, so I have run the same tests on both, on one Mac Studio. I already wrote a page for each, how to run Gemma 4 on a Mac and how to run Qwen3.6 27B on a Mac. This one is for choosing between them.

One thing to know before the numbers. This is a mixture-of-experts model against a dense one. Gemma stores 26 billion parameters and uses about 4 billion for each word. Qwen uses all 27 billion for each word. A lot of what follows, the speed most of all, comes from that design, and you would see it in any pair built that way. It is not a ruling on Google against Alibaba. Gemma 4 also comes in a 31B, and the Qwen 3.6 family has a 35B-A3B. I have not measured either, and they would pair up differently.

The two side by side

Two notes on the memory rows. Outlier lists 16 GB as Quick's minimum, and the site's RAM table puts it in the 24 GB row. The file is 15.61 GB, so on a 16 GB Mac it loads with nothing to spare, and I would not plan around that. Second, a long chat costs memory on top of the file. The app's own readout for the 32K default estimates about 2.5 GB of context memory with Quick loaded and 2.0 GB with Core. Those come from a formula, not a measurement, and Quick's formula assumes 10 of its 30 layers keep a full-length cache, which is a cautious assumption I did not check.

How fast each one is

Table 2 is decode speed, how fast the words arrive once it has started. M1 Ultra, 4-bit MLX, one request at a time.

So Gemma is 43.6 tokens a second in one run and 69.0 in another, and Qwen landed between 16.6 and 21.6 in the three runs I compare, with older figures I never reconciled. The builds and the methods differ, and I have not worked out which one accounts for the gap on Gemma, so I am not going to pick one for you. Both put Gemma at two to three times Qwen. That fits the design: on a Mac, each word is mostly limited by how fast the chip can read the model's weights from memory, and Gemma reads about 4 billion parameters' worth per word where Qwen reads all 27 billion.

The catch on short replies

Run B also measured the fixed wait before the first counted token: 3.72 seconds for Gemma and 1.03 seconds for Qwen. Gemma reasons before it answers, which may be part of that. I did not split it out. A reply takes the fixed wait plus its length divided by the speed, and Table 3 does that sum for both.

The two cross at about 85 tokens. A one-line answer comes back sooner from Qwen. A long draft comes back much sooner from Gemma: 1,000 tokens in about 18 seconds against about 47.

Treat Table 3 as a sketch. I computed it from the two measured numbers for each model, I did not time each length. The fixed wait came from a short prompt sent straight to the engine on build 1.11.669, and I have not re-measured it on 1.11.912. A new chat in the app adds Outlier's own system prompt, and I did not time Gemma's first word there. Time to first token on a Mac has the Qwen side of that wait.

How good each one is

Same Mac, same app, the same questions where I could manage it. Where I could not, the sample column says so.

General knowledge

On 200 identical questions, Qwen got 179 right and Gemma got 162. The 95% intervals, 84.5 to 93.0 and 75.0 to 85.8, touch, and a plain two-proportion test on the two counts gives p of about 0.02. I read that as Qwen ahead by several points, not by exactly 8.5. Gemma's separate 300-question run scored 79.3%, which lands in the same place, and the earlier build, 1.11.757, gave 81.5% for Gemma and 89.5% for Qwen. The tier benchmarks page has the method and every tier.

Short code and math

On 23 short coding problems asked in chat, Gemma got 21 and Qwen got 20, and Gemma finished in 250 seconds against 394. Their misses did not overlap, so between them they solved all 23. That is a small set. One problem either way is noise. July's smaller checks agree: functions run for real, 19 of 20 and 20 of 20, and 30 grade-school math problems, 26 and 27.

One number disagrees with all of that. An April run of the full 164-problem HumanEval scored Gemma at 21, or 12.8%, with a harness that pulls the code out of a fenced block, while Qwen scored 156 of 164 in August through the app. I do not believe a model that gets 19 of 20 and 21 of 23 is really at 13% on the full set, and I have not worked out which test is wrong. I am showing it because it is already on my Gemma page and you should see both. HumanEval is also from 2021, so a 2026 model has very likely seen it.

Real repositories

This is the big gap. Given a real bug in a real project and left to find the fix, Qwen fixed 23 of 50 in a blind SWE-bench Verified run. Gemma fixed none. Its patches were not wrong. In all 50 tries its agent never wrote one, so nothing was graded. An earlier run on a different set of 30 tasks gave 0 of 30, with 29 of those stalling without taking any action. I wrote up the pattern in chat coding vs agentic coding.

Read that as a result about Gemma inside Outlier's agent loop. It says the pairing does not work for agent coding in my harness. It does not say Gemma cannot do it in someone else's, and I have not tried another. The short-code scores above say the model can write code. That is why Core is the default for agent work. These runs were on builds 1.11.757 and 1.11.788, and I have not re-run them on 1.11.912.

Which one I would run

• One model for everything, on a 24 GB or larger Mac: Qwen 3.6 27B.
• Code in a project folder, with the agent: Qwen 3.6 27B. Gemma got nothing in my harness.
• Long drafts, summaries and explanations, where the wait is what bothers you: Gemma 4. At 500 tokens my July sum has it at about 11 seconds against 24.
• Short questions and one-line answers: Qwen, for the shorter wait.
• A 16 GB Mac: neither, with room to spare. Start with Lite, which is free.

Both are Pro tiers in Outlier. You switch between them from the model menu at the top of the window, and the free tiers are Nano and Lite. If you would rather run them without Outlier, the commands are on the Gemma 4 and Qwen3.6 27B pages. For Gemma against Llama, see Gemma vs Llama, and for Qwen against Llama, Qwen vs Llama.

What I did not test

• Gemma 4 31B and Qwen 3.6 35B-A3B, which would pair up differently.
• Any chip other than an M1 Ultra, and any Mac with 16 or 24 GB.
• Either model in Ollama, LM Studio or llama.cpp.
• A paired test of thinking on against thinking off for these two. The MMLU runs had it off.
• Images, audio, and chats longer than the 32K default.
• Gemma's first-word wait on a new chat in the app.
• Gemma as an agent in any harness other than Outlier's.

Frequently asked questions