Can a small open model run a design tool?

Axolo Design Manager is an AI-driven prototyping tool plugged into a team's real design system. Before anyone iterates on a screen, the tool has to import that design system: understand what is a component, who a prop is for, which token a raw value should become.

I set one rule for that first step. Importing a design system must work without any external AI API. No bill per call, and no client code sent to a third party. Large models come later, during screen iterations, where they really earn their place.

So the question was simple: can a small open model, running on an ordinary CPU, make those decisions well enough?

How we tested

• A dozen small open-source models, all on CPU, no GPU: Qwen3 and Qwen3.5, Gemma 4, Granite 4.2, Phi-4-mini, SmolLM3, LFM2.5 and Qwen2.5-Coder, from 1.2 to 4 billion parameters.
• Laya, a tiny decision model. It is not a chat model: it picks an answer from a list, very fast.
• Claude Opus 5.5 as the ceiling: what a large model gets on the same questions.
• Axolo's import scripts as the reference for everything mechanical.
• About 2,500 real design system questions, drawn from public systems such as shadcn, MUI, Mantine, Ant Design, Carbon, Chakra, Fluent, Spectrum and Radix.
• Ambiguous cases settled by hand. The ranking uses the 290 cases every engine answered, 135 of them reviewed by a human.

The podium

Ranked on the human-reviewed cases. Three 4-billion-parameter models finish within three points of each other.

1. 2Gemma 4 E4B63 %7.8 GB · Apache 2.0
2. 1Qwen3.5 4B65 %6.3 GB · Apache 2.0
3. 3Qwen3 4B62 %5.3 GB · Apache 2.0

Off the podium · ceiling Claude Opus 5.5 scores 87 % on the same reviewed cases. It runs behind an external API, so it is the yardstick, not a candidate.

The full ranking

Accuracy on the 135 human-reviewed cases, then on all 290 shared cases. Memory is peak RAM on the machine.

Import without an external API: validated

On the tasks that make up an import, the leading small model, Qwen3.5 4B, against Claude Opus 5.5.

On securing the import, the small model beats Claude: 90 % against 87 %. Elsewhere it stays within a few points, except on compositions. A design system can be imported without any external API.

Use case × engine

Accuracy per use case, on all cases. The highlighted cell is the best engine on that row, Claude aside. A dash means the engine does not handle that task.

Speeds are provisional: the clean speed runs are still going, so this article ranks on accuracy only. As an order of magnitude, the 4B models take one to two seconds per decision on CPU, Laya under a fifth of a second.

What we take away

• Below 4 billion parameters, it does not hold. Every smaller model ends at 54 % or less on the reviewed cases.
• On import, a small model is almost as good as a large one. Within a few points on most tasks, and ahead on securing the import.
• Scripts remain unbeatable on mechanical work. Instant, free, and 96 % on screen import: anything a rule can decide should stay a rule.
• The gap widens on judgement calls. Edit or create, pattern or component, risk of a change set: 40 to 53 points behind. Those decisions happen during iterations, and that is where a large model belongs, or where the tool offers an explained choice instead of an imposed answer.
• Laya is fast but limited. Good for a quick first pass, not for the final word.

Scripts first, a small model as backup, a large model only where it really helps.

Next step: skills per task. Instead of one generic prompt, each decision gets its own short brief and examples, and we measure how far that pushes the small models.