What a local AI coding agent can actually build

"What can offline AI actually do for coding?" is usually answered with a demo or a vibe. This page answers it with receipts. Everything below was built by a local 27B coding agent — the "compact" tier running on-device via MLX on an M1 Ultra, no cloud call — during one real engineering session, and every capability claim is verified by the agent running the tests, not by anyone eyeballing the output.

The apps it built and tested on its own

These are not prompts I hand-fixed afterward. Each was produced end-to-end by the agent — write the files, run the tests, fix what failed, re-run — offline. What makes them count is the last step: a passing test suite the agent generated and executed itself.

What it built	Scope	Verified how
Bank Account class	Deposit / withdraw with overdraft protection	13/13 tests passing
Library management system	5 files — borrow / return with edge cases	Correct behavior across edge cases, tests green
Contacts app	sqlite-backed CRUD	Runs against a real sqlite database
Arithmetic expression evaluator	Operator precedence + parentheses	2+3*4 = 14, (2+3)*4 = 20
Multi-file build	33 files	Every file compiles

Source: measured autonomous runs in an on-device Outlier session, compact 27B tier (MLX 4-bit) on an M1 Ultra. "Verified" means the agent's own generated test suite ran and passed, offline.

Two of these deserve a closer look, because they're the kind of thing people assume a small local model can't do reliably.

The precedence evaluator

An arithmetic expression evaluator is a classic tripwire. Get the parser wrong and 2+3*4 comes out 20 instead of 14 because you evaluated left-to-right and ignored precedence. The agent got precedence right and got parentheses right — (2+3)*4 correctly returns 20. That is a small program, but it's exactly the kind of bounded, checkable logic where a local model can hit the correct answer every time, because a test can tell it when it's wrong.

The 33-file build

Thirty-three files that all compile is a different kind of achievement than one clever function. It's about coherence across a project — imports resolving, names matching, indentation surviving from file to file. This is the frontier where local agents historically fell apart (an earlier ceiling in Outlier's own testing was around 18 files before indentation drift crept in). Getting to 33 compiling files is the payoff of whole-file edits and self-checking rather than blind diff-patching.

It checks its own work — and won't cheat to pass

The reason these builds land isn't a smarter model. It's that the agent verifies itself. When a test fails, it doesn't just re-roll the code and hope. It computes the true expected answer itself, then decides which side is actually wrong:

• If the code is wrong, it fixes the code.
• If a test asserts a wrong value, it fixes the test.

That second case is dangerous, and there's a guard for it: the agent is blocked from weakening a good test just to force a green run. It can correct a test that genuinely encodes the wrong expected value, but it can't quietly delete an assertion to fake a pass. This is the difference between an agent that tells you the truth and one that games its own scoreboard.

Which points at the single biggest lever in local coding: the execution oracle. Select the agent's work on run-the-tests, not on "it compiles" and not on "the model approved itself." Compiling is necessary and nowhere near sufficient; a model grading its own homework is worth even less. Running the tests is the only signal that survives contact with reality, and it's why the local agent can be trusted on the tasks above.

The honest number: ~45% on a blind SWE-bench slice

Bounded toy programs are one thing; real repository bugs are another. On a blind subset of SWE-bench Verified — the standard "fix this real GitHub issue in a real codebase" benchmark — the local 27B measured about 45% (18 of 40).

Read that honestly. Forty tasks is a slice, not the full 500-task Verified set, so 45% is not a leaderboard number and I won't present it as one. It's a measured, blind, offline data point: a local 27B, no cloud, resolves close to half of a real-bug sample. That's a genuinely useful floor, and it's the number the marketing usually won't give you.

Where a local 27B already matches a cloud flagship

On a whole class of work, the gap to a frontier cloud model is small or gone. The common thread is that the task is bounded, in-distribution, and testable:

• Implementing a well-specified class or module with clear behavior (the bank account, the library system).
• Parsers, evaluators, and algorithmic logic where a test can confirm correctness (the precedence evaluator).
• CRUD apps against a real datastore (the sqlite contacts app).
• Multi-file projects where the goal is coherence and everything compiling (the 33-file build).

When there's a test to check against and the problem lives in familiar territory, the local agent's self-verification loop closes the gap that raw model quality would otherwise leave open.

Where it doesn't match — the honest ceiling

I'm not going to pretend the gap closes everywhere. There are tasks where a local 27B does not match a cloud flagship locally, and they cluster into three hard walls:

Wall	Why it's hard for a local 27B
Per-token slip floor	Exact enumeration, counting, and arithmetic have a residual error rate that's a structural property of a 4-bit 27B — small, but nonzero, and it shows up on tasks with no test to catch it.
Breadth / world & library knowledge	The long tail of API and library facts is a parameter-count wall. Distillation transfers style, not the full fact long-tail, so obscure-library tasks favor the bigger model.
Exact recall from far-back context	Pulling an exact byte-span out of deep context is where a cloud model with a huge context window and stronger recall pulls ahead.

Concretely, the tasks a local 27B won't match locally are: breadth-bound tasks (they need knowledge the parameters don't hold), flawless multi-step novel reasoning with no test to verify against (nothing catches a subtle slip), and 40+-step fully autonomous runs (errors compound over a long unattended horizon). For those, the right move is escalation, not pretending.

How far the local ceiling can move

Today's blind ~45% is a floor, not a wall, and the path up is engineering rather than a bigger model:

• Local engineering — execution-oracle selection, self-verification, verbatim memory, whole-file edits, and grammar-constrained tool emission — can push the same 27B into the mid-50s to low-60s.
• Training — LoRA and distillation on-distribution can reach roughly 70–80% in-distribution.
• The last ~15–20 points to a cloud flagship's ~65–72% is a genuine ceiling, gated by the three walls above. That final stretch doesn't close by trying harder locally — it closes by routing the hard call up.

The honest framing: a local coding agent isn't a smaller Claude. It's a tool that owns the bounded, testable, private, offline half of the work outright — and knows when to hand off the rest.

Frequently asked questions