The best small and on-device AI models in 2026
The best small and on-device AI models right now are Gemma 4, Nanbeige4.2-3B and Ministral 3 (3B, 8B and 14B), out of 19 we have reviewed. Each is ranked by our rating, with its price, our verdict and who it suits.
By ReviewRamble editors · Updated Nov 11, 2026 · How we review
How we ranked: by our rating out of 5, from reviews built on each product's own pricing, documentation and policies plus independent benchmarks where they exist, each checked on the date shown in the review. We take no affiliate commission.
- 1
Gemma 4 is now the default open family below 32B parameters, because its licence finally matches its quality.
Best for: Developers building on-device, local or single-GPU apps who want a permissive licence and multimodal input.
Price: Free weights under Apache 2.0. Hosted on Google Cloud and third parties; OpenRouter lists 31B at $0.20/$0.40 and 26B A4B at about $0.08/$0.26 per 1M tokens.
Read our Gemma 4 review4.4/5 - 2
Nanbeige4.2-3B does agent work that used to need a 9B model, provided you give it time to think.
Best for: Developers running a local coding or office agent on a laptop or single GPU who can tolerate long reasoning times.
Price: Free Apache 2.0 weights. No first-party hosted API published.
Read our Nanbeige4.2-3B review4.2/5 - 3
A complete small-model family under a clean licence, with reasoning and vision in every size; Qwen still edges it on several of Mistral's own comparisons.
Best for: Teams that need a small, permissively licensed model with vision for laptops, edge devices or cheap hosted inference.
Price: Open weights under Apache 2.0. Mistral API: 14B $0.20, 8B $0.15, 3B $0.10 per 1M tokens, the same for input and output; batch half price.
Read our Ministral 3 (3B, 8B and 14B) review4.1/5 - 4
Six capable 2B models that fit in about a gigabyte each, from agents to speech to typed decisions, under a licence that still stops at $10 million in revenue.
Best for: Developers and companies under $10 million in revenue building on-device agents, voice features or fast classifiers that must run on a phone, a laptop or one GPU.
Price: Free weights under the LFM Open License (commercial use only below $10M annual revenue). Liquid API, as reported by VentureBeat from the signed-in console: LFM3-1.7B $0.08 input / $0.20 output per 1M tokens; VL $0.10 / $0.50; Audio and Omni $0.10 / $1.00 plus audio tokens. Hosted d1 decisions $0.02 per 1M input, output free.
Read our LFM3 review4.0/5 - 5
A 2B model that calls tools and patches code like a 4B one, and knows noticeably less than one.
Best for: Developers putting tool-calling or coding helpers on phones, laptops and edge hardware, and researchers who want fully documented small-model training.
Price: Free weights and training data under Apache 2.0.
Read our MiniCPM5-2B review4.0/5 - 6
Bonsai 2 9B puts a capable nine-billion-parameter vision model in the memory of a phone, and gives back most of what it costs on reasoning-heavy work.
Best for: Developers shipping a local assistant or coding helper on phones, laptops or 6 GB GPUs where memory matters more than the last points of reasoning.
Price: Free under Apache 2.0. Hosted on the Darkbloom console; prices not published.
Read our Ternary Bonsai 2 9B review4.0/5 - 7
The 74B puts a top-five sub-100B score on one workstation GPU, but the 24GB in its name is a weight budget, not the card you need.
Best for: Engineers with a 32 GB or larger NVIDIA GPU, or an RTX or DGX Spark, who want a capable local multimodal agent model, and device makers building voice assistants on Jetson or RTX.
Price: Free open weights under OpenMDW-1.1. No metered API price published for either model; E4B has a build.nvidia.com endpoint.
Read our Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB) review3.9/5 - 8
Fourteen months on, gpt-oss-20b is still the cheapest competent reasoning model to host, but Gemma 4 now does more in the same 16GB.
Best for: Developers who want a free, private and fast reasoning model on a laptop or single consumer GPU, and do not need image input.
Price: Free weights under Apache 2.0. Hosted from $0.018 input / $0.09 output per 1M tokens on OpenRouter; Artificial Analysis lists $0.07 / $0.25.
Read our gpt-oss-20b review3.8/5 - 9
A laptop-sized agent model that out-calls Qwen3.6-35B-A3B on tool tests, held back by a licence that stops at $10 million in revenue.
Best for: Developers and smaller companies running tool-calling agents on laptops or single GPUs, who fall under the $10 million revenue line or use a hosted API.
Price: Free weights under the LFM Open License (commercial use only below $10M annual revenue). Liquid Console: $0.10 input / $1.00 output per 1M tokens; OpenRouter: $0.05 / $0.70.
Read our LFM2.5-24B-A2B review3.8/5 - 10
Unusually well documented and better at keeping facts than a prompt, but one English rewrite in ten still changes something, and its headline score rests on a detector that may change tomorrow.
Best for: People who draft with AI and want the result to sound like their own writing, on their own machine, and who will proofread numbers and names before sending.
Price: Free under Apache 2.0: weights, GGUF files, desktop app and CLI. No hosted service.
Read our humanizer (12B) review3.7/5 - 11
On iFlytek's own chart Spark-X2.6-4B is the strongest 4B agent model yet, but none of it has been checked from outside, and two of its biggest wins come from benchmarks only iFlytek has seen.
Best for: Developers building local agents on laptops or edge devices who can verify the claims on their own tasks and are free to use a model from iFlytek.
Price: Free under Apache 2.0; runs on your own hardware. No hosted price for the 4B model published.
Read our Spark-X2.6-4B review3.7/5 - 12
VibeThinker-3B solves olympiad maths and contest code like a model hundreds of times larger, and should be kept well away from anything else.
Best for: Teams that need a cheap, self-hosted solver or checker for competition maths, science questions or algorithm problems.
Price: Free MIT-licensed weights. No first-party hosted API published.
Read our VibeThinker-3B review3.7/5 - 13
Remarkable maths per parameter, useless outside maths, and so far graded only by the people who made it.
Best for: Researchers and education-tech teams that need many competition-maths solutions cheaply and can verify answers automatically.
Price: Free open weights under Apache 2.0. No hosted API; you pay only for your own GPU.
Read our Limite 1B - Violetto review3.6/5 - 14
MobileMoE-L beats Qwen3.5 2B on most of Meta's table while running under a billion parameters per token, but the non-commercial licence keeps it out of any phone app you could sell.
Best for: Researchers and on-device ML engineers studying how sparse models behave on phone hardware, where commercial use is not the goal.
Price: Free to download after a gated sign-up, under the FAIR Noncommercial Research License: research and non-commercial use only. No hosted API or inference provider offers it.
Read our MobileMoE (S, M, L) review3.6/5 - 15
A credible phone-command model at a size nothing else reaches, sold with a broader claim than its own charts support.
Best for: Device and app makers with a fixed, small set of actions who will fine-tune a model to run offline on very constrained hardware.
Price: Free open weights under Apache 2.0; prebuilt engines free to download. The Cactus Platform sells 2-bit post-training, datasets and full fine-tunes; prices are not published.
Read our Needle 3 review3.6/5 - 16
The 2-bit Qwen3.8-27B to run when tool calling matters more than maths, as long as you treat Underdog's scores as a promise and not a measurement.
Best for: Developers running a local tool-calling agent on a 16 GB laptop or a single consumer GPU who accept a maths and reasoning penalty.
Price: Free Apache 2.0 weights. No hosted API price; you pay for your own hardware, or use it inside the Underdog app.
Read our Underdog Saluki 27B 1.0 review3.6/5 - 17
BitNet b1.58 2B4T proves a native 1-bit model can match 2B full-precision rivals on a fraction of the memory, but it is a research artefact, not a model to ship.
Best for: Researchers and embedded engineers evaluating whether native 1-bit models are viable on CPU-only hardware.
Price: Free MIT-licensed weights and bitnet.cpp runtime. No hosted API.
Read our BitNet b1.58 2B4T review3.5/5 - 18
SmolLM3 is a glass-walled training run more than a product: unmatched for learning how a 3B model is made, outclassed for shipping one.
Best for: Researchers and educators who want a small, fully reproducible model to study or fine-tune.
Price: Free weights under Apache 2.0, with the training data recipe and checkpoints published. No hosted API on Hugging Face's own Inference Providers.
Read our SmolLM3-3B review3.5/5 - 19
A small model that does one kind of logic well enough to match models twice its size, under a licence that keeps it in the lab.
Best for: Researchers and students who want a local model for entailment checks, first-order-logic translation and Lean critique, and do not need commercial rights.
Price: Free weights for non-commercial research and education under the webAI Non-Commercial License 1.0. No hosted API or commercial licence price published.
Read our TwIL-LM3-Pro review3.5/5
Sources
The main official source behind each ranked review; every review lists its full set.
- Gemma 4: Google: Gemma 4, byte for byte the most capable open models
- Nanbeige4.2-3B: Hugging Face: Nanbeige4.2-3B model card
- Ministral 3 (3B, 8B and 14B): Mistral AI: Introducing Mistral 3
- LFM3: Liquid AI blog: Introducing LFM3
- MiniCPM5-2B: Hugging Face: openbmb/MiniCPM5-2B model card
- Ternary Bonsai 2 9B: Hugging Face: prism-ml/Ternary-Bonsai-2-9B-gguf model card
- Nemotron 3.5 Bolt (74B-A9B 24GB and E4B 4GB): Hugging Face: NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF model card
- gpt-oss-20b: Hugging Face: openai/gpt-oss-20b
- LFM2.5-24B-A2B: Hugging Face: LFM2.5-24B-A2B model card
- humanizer (12B): Hugging Face: jialinyyzz/humanizer model card
- Spark-X2.6-4B: Hugging Face: XHToken/Spark-X2.6-4B model card
- VibeThinker-3B: Hugging Face: VibeThinker-3B model card
- Limite 1B - Violetto: Paradigma: Limite 1B - Violetto release blog
- MobileMoE (S, M, L): Hugging Face: facebook/MobileMoE-L-SFT model card
- Needle 3: Cactus Compute: Needle 3 release page
- Underdog Saluki 27B 1.0: Hugging Face: Underdog Saluki 27B 1.0 model card
- BitNet b1.58 2B4T: Hugging Face: BitNet b1.58 2B4T model card
- SmolLM3-3B: Hugging Face: HuggingFaceTB/SmolLM3-3B model card
- TwIL-LM3-Pro: Hugging Face: webAI-Official/TwIL-LM3-Pro model card
More guides
- Best large language models (LLMs)
- Best AI reasoning models
- Best AI coding models
- Best vision and multimodal AI models
- Best OCR and document AI models
- Best AI image models
- Best AI video models
- Best AI decision models
- Best text-to-speech models
- Best speech-to-text models
- Best AI music and audio models
- Best embedding models
- Best open-source LLMs
- Best American open-source LLMs
- Best Chinese open-source LLMs
- Best European open-source LLMs