Open-source LLM benchmarks 2026: Kimi K3 closes in

For three years the shape of the frontier model market was stable: the best proprietary models led, open-weight models trailed by roughly a generation, and anyone who needed the top of the leaderboard paid an API bill for it. That gap is now smaller than the gap between the top two proprietary models.

On the LLM-Stats composite as of early August 2026, the best open-weight model, Moonshot AI's Kimi K3, scores 55.4 against 57.2 for OpenAI's GPT-5.6 Sol and 56.5 for Anthropic's Claude Opus 5. It ranks fifth overall, ahead of GPT-5.6 Terra, Claude Opus 4.8, and Meta's Muse Spark 1.1.

That is a difference of 1.8 points between the best model you can download and the best model you can only rent.

Where the open models actually lead

The composite flattens some large differences, and the per-benchmark picture is more interesting than the ranking.

On Humanity's Last Exam, the broad expert-knowledge benchmark designed specifically to resist saturation, Kimi K3 posts 56 and Zhipu's GLM-5.2 posts 54.7. On SWE-bench Verified, the closest thing the field has to a realistic software engineering test, DeepSeek's V4-Pro-Max leads all open models at 80.6%. On GPQA Diamond, a graduate-level science question set, Kimi K3 reaches 93.5%, with GLM-5.2 at 91.2% and Kimi K2.6 at 90.5%.

Those are not "good for an open model" numbers. On the coding and science benchmarks that enterprises actually run their procurement bake-offs against, the open field is inside the proprietary field's error bars.

Scale explains part of it. Kimi K3 is a 2.8-trillion-parameter model with a one-million-token context window. DeepSeek V4-Pro-Max is 1.6 trillion, also with a million-token window. GLM-5.2 is 753 billion. These are not small models that punch above their weight; they are very large models released under permissive licenses by labs that decided distribution matters more than API margin.

Which raises the practical question that the leaderboard does not answer.

"Open weights" is not "runnable"

A 2.8-trillion-parameter model is downloadable in the same sense that a container ship is purchasable. The weights are public, the license permits commercial use, and essentially no Canadian company outside a handful of banks and telcos has the hardware to serve it at production latency without renting the same cloud GPUs they were trying to avoid renting.

This is the gap between the benchmark story and the deployment story, and it is where the interesting engineering competition has moved. Three things now matter more to a buyer than a two-point composite difference:

Active parameters, not total. Mixture-of-experts architectures mean a model can have 200 billion parameters and activate 25 billion per token. The memory footprint is set by the total; the inference cost is set by the active count. A model that fits your GPUs is worth more than a better model that does not.

Quantization quality. Every open model is served quantized in practice. How much capability survives compression to 8-bit or 4-bit determines what hardware it runs on, and vendors have historically been vague about it.

License, actually read. "Open" spans Apache 2.0, which permits nearly everything, through licenses with non-commercial clauses or user-count thresholds that a procurement lawyer will flag. The leaderboards label all of it "Open."

Why the leaderboards disagree with each other

Anyone comparing two of these rankings side by side will notice they do not agree, and the reason is worth understanding before you cite any of them in a procurement document.

Composite scores are weighted averages over a benchmark basket, and every leaderboard picks its own basket and its own weights. LLM-Stats, Artificial Analysis, and the various arena-style rankings weight reasoning, coding, math, and knowledge differently, so a model tuned hard on competition math can rank several places higher on one board than another without anything about the model changing. Some boards also normalize raw benchmark percentages onto a 0-100 scale, which is why the same model can appear with a GPQA figure in the 90s on one page and the 50s on another. Those are different quantities with the same label.

Contamination is the second problem. Benchmark questions leak into training corpora, and a model that has effectively memorized part of a test set will score well without generalizing. This is the main reason Humanity's Last Exam and SWE-bench Verified are weighted heavily by serious evaluators: HLE was built to be hard to saturate, and SWE-bench Verified was human-filtered specifically to remove tasks that were unsolvable or gameable.

The practical rule: use composites to decide which three models to try, and never to decide which one to ship.

The Canadian entry competes on the second list

Cohere released Command A+ on May 20, 2026 under Apache 2.0, and it is a clear illustration of the split between the two scoreboards.

On raw capability it is not close to the top of the open field. Artificial Analysis places it at 37 on its Intelligence Index, roughly alongside Claude 4.5 Haiku, with GPQA Diamond around 76%, HLE around 11%, and Terminal-Bench Hard around 25%. Against Kimi K3 and DeepSeek V4-Pro-Max those are not competitive numbers, and Cohere has not pretended otherwise.

On the deployment list it looks different. Command A+ is a mixture-of-experts model with 218 billion total parameters and roughly 25 billion active, and it runs on two Nvidia H100s or a single Blackwell GPU. That is a machine a mid-sized Canadian insurer, hospital network, or provincial agency can actually buy, put in a room it controls, and run under its own data residency rules. The release emphasizes lossless quantization and native citation support, both of which are deployment features rather than benchmark features. Its AA-Omniscience non-hallucination score of 86% is a similar kind of claim: not "smarter," but "wrong less often in ways an enterprise cares about."

Cohere is also the only major lab in this comparison headquartered in Toronto, which matters more in 2026 than it would have in 2023. Sovereign AI procurement, where a government or regulated industry requires the model to run inside national borders under domestic legal jurisdiction, is a real budget line now, and it is a category where a downloadable Apache-2.0 model from a domestic vendor beats a better model behind a foreign API on criteria that have nothing to do with GPQA.

What this means for buyers

Three practical conclusions follow from the current numbers.

Stop paying a frontier premium for non-frontier work. If your workload is summarization, classification, extraction, routing, or retrieval-augmented question answering over your own documents, the open field passed the threshold where the proprietary models were meaningfully better at those tasks some time ago. The remaining premium buys reliability of service and a support contract, which are legitimate purchases, but they should be priced as such rather than as capability.

Benchmark on your own data before switching. SWE-bench Verified at 80.6% tells you that a model is good at the specific distribution of GitHub issues in that set. It does not tell you how it handles your codebase, your ticket format, or your internal libraries. Every one of these numbers is a proxy, and the correlation between leaderboard rank and performance on a particular corporate workload is much weaker than the rankings imply.

Read the license before the leaderboard. The single most expensive mistake available in this market right now is building a product on an open-weight model whose license does not permit the use you have in mind, then discovering it during a diligence process.

Budget for the serving, not the model. The weights are free and the inference is not. A trillion-parameter open model served at production latency needs multi-GPU nodes running continuously, plus the engineering time to keep them fed, and that total cost of ownership frequently exceeds the API bill it was meant to replace. Self-hosting wins on data residency, on predictable unit economics at high volume, and on not having a vendor deprecate your model. It rarely wins on a spreadsheet at low volume, and vendors of open models have no incentive to tell you where that crossover point is.

The frontier gap is no longer the number that decides anything. For most production workloads the binding constraint is now hardware you can afford, a license you can defend, and latency your users tolerate, and on those three axes the ranking looks nothing like the leaderboard.

What to watch next

The composite gap has been closing at a rate of a few points per release cycle. If it closes entirely, the proprietary labs' pricing power depends on service quality and tooling rather than capability, which is a different and much more competitive business.

Watch also whether the open releases keep getting larger. The current open leaders are enormous, and their openness is partly safe for their releasers precisely because so few organizations can serve them. If a 70-to-200-billion-parameter model lands within a few points of Kimi K3, the practical picture changes far more than another trillion-parameter release would.

And watch the licenses. Cohere moving Command A+ to full Apache 2.0 was a deliberate positioning choice against models labeled open with strings attached. If that becomes the expectation rather than the exception, the word "open" on these leaderboards will finally mean one thing.

Benchmark figures cited are from the LLM-Stats open-model and overall leaderboards and Artificial Analysis, retrieved August 5, 2026. Leaderboard positions in this field change monthly; check the source before relying on a specific number.