In early 2024 the conversation about open-weight LLMs went something like this: “Yes, they’re cheaper, but they’re not as good. You’ll have to compromise.” By mid-2025 it was: “They’re comparable for most tasks, but for the hard stuff stick with the API.” By early 2026 it’s: “For our enterprise workloads, the open weights are now better.”
That last sentence still surprises people who haven’t looked recently. Let’s break down what changed.
The four families that actually matter
The open-weights universe is enormous, but in production we use four families almost exclusively. The rest are research toys, fine-tunes, or vendor lock-in dressed up as “open.”
LLaMA 4 — the workhorse
Meta’s LLaMA family is the safe default. Predictable behaviour, mature tooling, an enormous fine-tune ecosystem. The LLaMA 4 70B model is what we deploy when a customer says “we just need it to work, ours, on-prem, today.” The 8B variant is shockingly capable for narrow tasks — on a single L40S it handles 90% of internal-tool intents at sub-300ms latency.
Mistral Large 3 — the European answer
Mistral’s mid-sized models are our preferred recommendation when GDPR or data-residency teams want a European-trained provenance. The current Mixtral mixture-of-experts variants give you 70B-tier quality at roughly 35B-tier inference cost. For multilingual European customers (CZ/DE/PL/HU mixed in one document) Mistral Large 3 is consistently the strongest.
Qwen3 — the dark horse that became a frontrunner
Alibaba’s Qwen3 line, especially the long-context and reasoning-tuned variants, has quietly become the best in class for code generation, structured-output workflows, and very-long-document RAG. Three of our manufacturing customers run Qwen3 exclusively for CAD-adjacent agents. The political conversation is real — we deploy it air-gapped, weights downloaded once, no telemetry — and the model is genuinely excellent.
DeepSeek V3 and Gemma 4 — niche, but excellent
DeepSeek V3’s reasoning-tuned models broke a lot of assumptions about what was possible at consumer-GPU scale. Gemma 4 is the small-deployment champion for customers running edge devices or single-RTX-4090 deployments. Both are genuine open weights and both punch well above their parameter count.
What changed in 18 months: it’s not just the models
If you froze in late 2024 and looked again now, the model improvements are real but not the biggest story. Three other shifts mattered more:
- Inference servers caught up. vLLM, TGI, SGLang, and TensorRT-LLM made open weights serveable at managed-API economics. Two years ago you needed a research team to get good throughput. Today a competent platform engineer pulls a Docker image and gets 4000 req/s out of a 4×H100 box.
- Quantisation stopped being lossy. AWQ, GPTQ, and the newer FP8 paths let you run 70B-class models on a single 80GB card without measurable quality loss for most tasks. The economics shifted under everyone’s feet.
- Context windows stopped being a moat. 128k is now table-stakes; 1M-token open weights exist. The closed APIs no longer own “throw the whole repo into the prompt.”
Where closed models still win
To be fair: there are genuine cases where managed APIs are still the right call.
- Frontier reasoning on truly novel problems. The biggest closed models still have an edge on extreme out-of-distribution tasks. If your workload is “solve PhD-level physics problems we made up yesterday,” pay for the API.
- Zero ops appetite. If your team has zero capacity to run GPUs, the API is the right answer until that changes. We tell customers this honestly.
- Bursty, low-volume usage. Below ~10k completions/day, the per-token cost of an API can beat the amortised cost of even a single GPU. Above that threshold the math inverts hard.
For 80% of enterprise workloads we see — document processing, structured extraction, RAG over internal knowledge, code-adjacent automation, multilingual customer support, internal copilots — open weights running on customer infrastructure now win on quality, latency, cost, and compliance. That’s a corner of the matrix that didn’t exist 18 months ago.
The compliance argument is no longer secondary
For our European customers the conversation usually starts with quality and ends with sovereignty. By the time procurement, legal, and security have weighed in, “the prompt never leaves our network” is worth more than 5% of model quality. Open weights on your hardware are the only architecture where that statement is technically true. Everything else — including “private deployment” managed offerings — involves trusting a third party with your control plane.
The cheapest model is the one you don’t have to convince your DPO to approve.
Practical advice for 2026
- Start with LLaMA 4 70B-class for the general case. Mistral Large 3 or Qwen3 if you have specific multilingual or code needs.
- Quantise aggressively from day one. Don’t pay for FP16 weights you can run at FP8 with no measurable quality drop on your eval.
- Build a real eval harness. Vibes-based model selection is fine for a hackathon. For production, write 200 representative examples and grade them with a stronger model.
- Don’t marry the model. Treat the LLM as a swappable component. Our entire orchestration layer is model-agnostic on purpose — we’ve already swapped underneath three customer deployments.
- Reserve the API for the 5% it’s actually best at. Hybrid is fine. Hybrid by default is just expensive.
Where we go from here
The trend line is clear: every six months the open-weight frontier moves up, and the closed-model premium for general enterprise tasks shrinks. By the next time we write a post like this, we expect the conversation to have shifted again — from “is open as good as closed?” to “why would you pay for closed at all, outside of three narrow use cases?”
That’s a strange place to land for an industry that, two years ago, treated open-source LLMs as the cheap-and-cheerful option. But it’s where the data points.