Google just flipped the open-weight AI model race. On April 2, 2026, Google DeepMind released Gemma 4 — four models spanning edge devices to workstation GPUs, now under a clean Apache 2.0 license. The benchmark jump from Gemma 3 is not incremental. On math, code, and reasoning, it is a different era. But the community found something the launch blog did not mention: the MoE flagship runs at a fraction of Qwen 3.5's inference speed on the same hardware. Here is the full picture.
The Bottom Line
Gemma 4 is the biggest single-generation performance leap in the open-model space. The 31B dense model ranks #3 on Arena AI's text leaderboard. The 26B MoE model delivers nearly the same quality with 4B active parameters — an efficiency win few expected. The Apache 2.0 license removes every commercial blocker that killed enterprise adoption of Gemma 3.
- Switch to Gemma 4 31B if you need best-in-class math, reasoning, or coding on a self-hosted server and can tolerate lower inference speed.
- Choose the 26B MoE if you're building agentic pipelines or document-processing tools and need to balance quality against VRAM cost.
- Stay with Qwen 3.5 if multilingual support (especially CJK scripts), raw inference throughput, or battle-tested community tooling is your priority.
- Choose the E2B or E4B if you are building offline mobile apps — they are the only open multimodal models that run under 1.5 GB RAM on Android.
What's New
Four Models, Two Architectures, One License
Gemma 4 ships in four sizes: E2B, E4B, 26B, and 31B — covering a range from smartphones to GPU servers. The architecture split between the tiers is the key design decision.
The E2B and E4B are edge models that use Per-Layer Embeddings (PLE), a technique that feeds a secondary embedding signal into every decoder layer. A 2.3B-active model carries the representational depth of the full 5.1B parameter count while fitting in under 1.5 GB of memory with quantization. That means a smartphone-class model with reasoning depth that was impossible in previous generations.
The 26B A4B is a Mixture-of-Experts model. Only 3.8 billion of its 25.2 billion total parameters activate during inference, delivering roughly 26B-class intelligence with compute costs comparable to a 4B model.
Context windows reach 256K tokens on the workstation variants — enough to process a full codebase, an entire legal contract stack, or a book-length document in a single pass. Edge models support 128K tokens and include native multimodal processing of text, images, and audio.
The Apache 2.0 Shift
For two years, enterprises evaluating open-weight models faced an awkward trade-off. Gemma's custom license — with usage restrictions and terms Google could update at will — pushed many teams toward Mistral or Alibaba's Qwen instead. Gemma 4 eliminates that friction entirely, shipping under standard Apache 2.0 with no custom clauses, no restrictions on redistribution, and no restrictions on commercial deployment.
The standard of better licenses for strong open-weight models was set mostly by Chinese open model labs in the last one to two years. U.S. companies are now following suit. For enterprise legal and compliance teams, the difference is immediate and practical.
Agentic-First Design
Gemma 4 is purpose-built for advanced reasoning and agentic workflows, with multi-step planning, function calling, structured JSON output, and native system prompts built in. These are not post-training patches — they are part of the base architecture, which matters for reliability in production agent pipelines.
Benchmark Performance
The numbers below are drawn from Google's official model card, Arena AI's leaderboard, and independent evaluations from ai.rs and Lushbinary as of April 2026.
| Benchmark | Gemma 3 27B | Gemma 4 E4B | Gemma 4 26B MoE | Gemma 4 31B | Qwen 3.5 27B |
|---|---|---|---|---|---|
| Arena AI ELO (text) | 1,365 | — | 1,441 | 1,452 | ~1,440 |
| AIME 2026 (math) | 20.8% | 42.5% | 88.3% | 89.2% | ~85% |
| LiveCodeBench v6 | 29.1% | 52.0% | 77.1% | 80.0% | — |
| GPQA Diamond (science) | 42.4% | — | 82.3% | 84.3% | 85.5% |
| MMLU Pro (multilingual) | 67.6% | — | — | 85.2% | 86.1% |
| τ²-bench (agentic tool use) | 6.6% | — | — | 86.4% | — |
| Active params at inference | 27B | 4.5B | 3.8B | 30.7B | 27B |
| Context window | 128K | 128K | 256K | 256K | 128K |
Sources: Google DeepMind model card; Arena AI leaderboard as of April 2, 2026; ai.rs and Lushbinary independent analysis.
The Arena AI gap between Gemma 3 and Gemma 4 is not incremental. Gemma 3 27B scored 1,365. Gemma 4 31B scores 1,452 and the 26B MoE scores 1,441. That is a step change, not an iteration.
The τ²-bench number deserves special attention. Gemma 3 27B scored 6.6% on agentic tool use. Gemma 4 31B scores 86.4%. For developers building autonomous agents, this single figure matters more than any raw reasoning benchmark.
The Speed Problem Nobody Announced
Here is the non-obvious finding that every benchmark table obscures: Gemma 4's MoE model is significantly slower than competitors on identical hardware, despite the efficiency promise of MoE architecture.
Community testers measured the 26B MoE generating text at 11 tokens per second on GPUs where Qwen 3.5 produced 60+ tokens per second. The dense 31B model fared better but still landed at 18–25 tokens per second on dual GPUs.
Qwen 3.5 27B hits approximately 35 tokens per second on an RTX 4090 with Q4 quantization. Gemma 4 31B Dense achieves approximately 25 tokens per second. The 26B MoE lags further at approximately 11 tokens per second due to routing overhead.
MoE architectures activate fewer parameters per token in theory. In practice, the routing mechanism — moving data between expert layers — adds latency that offsets the compute savings on consumer and prosumer GPUs. Gemma 4's MoE differs from DeepSeek and Qwen's implementations: Gemma uses MoE blocks as separate layers added alongside normal MLP blocks rather than replacing them, which partially explains the routing overhead.
For batch inference on high-end server hardware this gap may close. For developers running local assistants or real-time pipelines, it is a real deployment constraint right now.
How It Compares to Llama 4 and Qwen 3.5
| Factor | Gemma 4 31B / 26B MoE | Llama 4 Scout | Qwen 3.5 27B |
|---|---|---|---|
| License | Apache 2.0 | Llama Community (700M MAU cap) | Apache 2.0 |
| Context window | 256K tokens | 10M tokens | 128K tokens |
| Arena AI rank | #3 / #6 | Lower | ~#4–5 |
| Inference speed | 11–25 tok/s | Slower (MoE overhead) | 35–60 tok/s |
| VRAM (flagship) | 58GB BF16 / 17GB Q4 | 32GB+ (Q6_K) | Comfortable on 32GB |
| Multilingual | Strong (140 languages) | Moderate | Best in class (250K vocab) |
| Edge / mobile | Yes (E2B/E4B, <1.5GB) | No small model | Limited |
| Agentic tool use | 86.4% (τ²-bench) | — | — |
| Best for | Reasoning, code, agents | Ultra-long context | Multilingual, speed, fine-tuning |
Sources: Lushbinary, ai.rs, VentureBeat, official model cards; April 2026.
For overall quality — especially on hard reasoning and coding tasks — Gemma 4 is the model to beat. Qwen 3.5 still holds the multilingual crown, and Llama 4 Scout still has the unmatched 10M context window.
Per active parameter, Gemma 4's MoE model is the most efficient reasoning engine available in the open model space. That efficiency claim is accurate on benchmark performance per active parameter — just not on wall-clock inference speed, which is a different thing.
What Changed from Gemma 3
The jump is stark enough that comparing the two feels unfair to Gemma 3.
| Metric | Gemma 3 27B | Gemma 4 31B | Change |
|---|---|---|---|
| AIME 2026 math | 20.8% | 89.2% | +329% |
| LiveCodeBench v6 | 29.1% | 80.0% | +175% |
| GPQA Diamond | 42.4% | 84.3% | +99% |
| τ²-bench (agentic) | 6.6% | 86.4% | +1,209% |
| Arena AI ELO | 1,365 | 1,452 | +87 pts |
| License | Custom (restricted) | Apache 2.0 | Full open |
Some researchers argue Gemma 4 31B is architecturally close to Gemma 3 27B — still using a hybrid sliding-window plus global attention pattern — suggesting the leap is likely training recipe and data quality rather than a fundamental architecture overhaul. If true, this has a practical implication: the gains are real, but they may be more replicable by competitors than a deep architectural change would be.
Who Should Switch
Switch to Gemma 4 now if:
- You were blocked by Gemma 3's licensing terms. Apache 2.0 eliminates every legal blocker. The model that was off-limits is now fully deployable.
- You are building agentic pipelines or coding assistants. The τ²-bench and LiveCodeBench gains are large enough to matter in production.
- You need on-device multimodal AI. The E2B running under 1.5 GB RAM with image and audio understanding has no real competitor in the open-weight space.
- You deploy on Google Cloud infrastructure. Serverless deployment via Cloud Run with NVIDIA RTX Pro 6000 GPUs, scaling to zero when idle, is available now on Vertex AI.
Stay with Qwen 3.5 if:
- Your application is latency-sensitive or handles high query volume. At 5x the inference speed, Qwen 3.5's throughput advantage is significant for live user-facing products.
- You serve multilingual users, especially CJK languages. Qwen 3.5 carries a 250K vocabulary advantage for non-Latin scripts where the gap remains decisive.
- You are mid-way through fine-tuning work. Community tooling — RL research frameworks, LoRA pipelines, datasets — took over a month to mature for Qwen 3.5 after release. Gemma 4 is at day five. QLoRA tooling had issues at launch.
Stick with Llama 4 Scout if:
- You need context windows beyond 256K. Llama 4 Scout's 10M token context window is irreplaceable for truly massive codebases or multi-document reasoning.
Ignore Gemma 4 for now if:
- You need inference speed above 40 tokens per second on consumer GPUs. Wait for vLLM and inference framework optimizations, which are actively in progress.
What to Watch Next
A larger Gemma 4 model with over 100B total parameters is rumored but not yet released. That would compete directly with Qwen 3.5's 397B flagship. Inference speed on the 26B MoE is the immediate bottleneck — framework teams at vLLM, llama.cpp, and Unsloth are actively working on optimizations. Alibaba released Qwen 3.6-Plus on the same day with a 1M token context window, keeping the competitive pressure immediate. Expect the Arena AI leaderboard rankings to shift within weeks as community fine-tunes and framework patches arrive.
Conclusion
Gemma 4 is a genuine leap — not a marketing rebrand. The math, code, and agentic benchmarks vs Gemma 3 are not close. The Apache 2.0 license removes the single biggest enterprise adoption barrier the Gemma line has had since launch. For self-hosted reasoning, coding, and agentic use cases, Gemma 4 31B and the 26B MoE are now the models to beat at their parameter tier.
The caveat is real: inference speed on the MoE variant lags Qwen 3.5 significantly on current hardware, and tooling is still maturing five days after launch. Watch the vLLM optimization track and the rumored larger model. If you have been waiting for a clean Apache 2.0 model from Google that can handle agentic workflows, the wait is over — pull the weights and test your pipeline now.
