Kimi K3: Moonshot AI’s 2.8T Open Model Is Here
Moonshot AI just dropped Kimi K3, a 2.8-trillion-parameter open model with a 1M-token context window that benchmarks competitively with Claude Opus 4.8. Here’s what developers need to know about the architecture, benchmarks, and pricing.
The World’s First Open 3T-Class Model Takes on Claude and GPT on Their Own Turf
Yesterday, Moonshot AI dropped Kimi K3, and the open-source AI landscape shifted. Again. A Beijing-based startup that most Western developers still haven’t heard of just released a 2.8-trillion-parameter model with a 1-million-token context window, native vision, and benchmark scores that compete with Claude Opus 4.8. The weights were published on Hugging Face around July 26-27, 2026. The license is the custom Kimi K3 License, not MIT. And this is not the first time Moonshot has done this.
The Kimi series has moved fast. K2 brought 1 trillion parameters to the open world. K2.6 and K2.7 Code pushed into multi-hour autonomous coding runs. Now K3 roughly triples the parameter count and reaches a 1M-token context window.
Let’s break down what K3 actually delivers, where the benchmarks are solid and where they need salt, and what this means for those of us building with LLMs day to day.
The Numbers at a Glance
| Spec | Kimi K3 |
|---|---|
| Total Parameters | 2.8 trillion |
| Active Parameters (per token) | 104B (16 of 896 experts, plus 2 shared) |
| Architecture | Kimi Delta Attention + Attention Residuals + Stable LatentMoE |
| Context Window | 1,048,576 tokens (1M) |
| Modalities | Text and image (the launch blog also cites video) |
| Training Precision | MXFP4 weights, MXFP8 activations |
| License | Kimi K3 License (custom) |
| Weights Release | Published around July 26-27, 2026 (Hugging Face) |
| API Input (cache hit) | $0.30/MTok |
| API Input (cache miss) | $3.00/MTok |
| API Output | $15.00/MTok |
A few things jump out. First, 2.8T total with only 104B active per token. That’s the MoE story: massive capacity, but you only pay compute for a small slice of it per forward pass. Second, the pricing is aggressive for a model this size. Third, and this is worth repeating: the weights are open.
Architecture: KDA, AttnRes, and the Quest for Stable Sparse Training
Moonshot published a technical report alongside the weight release, but the launch blog and API documentation give us enough to understand the key decisions.
Kimi Delta Attention (KDA) replaces standard multi-head attention with a delta-based formulation. Instead of computing full attention over all previous tokens, KDA computes attention over the differences between consecutive key/value representations. This slashes the KV-cache footprint, which is how K3 manages to serve a 1M-token context window at competitive per-token prices. They’ve submitted a corresponding vLLM implementation to the open-source community alongside the model.
Attention Residuals (AttnRes) is the companion innovation. Standard transformer layers accumulate representations uniformly across depth. AttnRes selectively retrieves representations from earlier layers rather than passing everything through.
Stable LatentMoE is the MoE routing layer. K3 deploys 896 experts and activates 16 per token. At that sparsity ratio (16/896 = 1.8%), routing stability becomes a first-order training challenge. The load balancing mechanism here is called Quantile Balancing: instead of heuristic updates with a sensitive hyperparameter, it derives expert allocation directly from router-score quantiles. This is a departure from the auxiliary-loss approach used in most MoE implementations (Mixtral, DeepSeek, etc.) and reportedly eliminates a major source of training instability.
The training stack includes two more notable components. Per-Head Muon extends the Muon optimizer by optimizing each attention head independently, which Moonshot says gives more adaptive learning at scale. SiTU (Sigmoid Tanh Unit) replaces standard activation functions for improved gradient flow. The whole model trains with quantization-aware training from the SFT stage onward, using MXFP4 weights and MXFP8 activations. This isn’t post-training quantization bolted on at the end; the model learns to operate in low precision from the fine-tuning phase.
Moonshot recommends deploying K3 on at least 64 accelerators for inference efficiency. You’re not running this on a single H100.
Benchmark Performance: Impressive, with Caveats
Here are the headline numbers from Moonshot’s evaluation suite. All K3 scores use max reasoning effort, temperature 1.0, and top_p 1.0.
Coding Benchmarks
| Benchmark | Kimi K3 | Claude Opus 4.8 | Claude Fable 5 | GPT 5.6 Sol |
|---|---|---|---|---|
| DeepSWE | 67.5 | 59.0 | 70.0 | 73.0 |
| Terminal-Bench 2.1 | 88.3 | 84.6 | 88.0 | 88.8 |
| Program Bench | 77.8 | 71.9 | 76.8 | 77.6 |
| FrontierSWE | 81.2 | – | 86.6 | 71.3 |
| SWE Marathon | 42.0 | 40.0 | 35.0 | 39.0 |
General Benchmarks
| Benchmark | Kimi K3 | Claude Opus 4.8 |
|---|---|---|
| GPQA-Diamond | 93.5 | 91.0 |
| BrowseComp | 91.2 | – |
| Humanity’s Last Exam (w/ tools) | 56.0 | – |
| MCP Atlas | 84.2 | – |
On Artificial Analysis’s Intelligence Index, K3 ranks #3 with a score of 57, behind Claude Fable 5 (60) and GPT-5.6 Sol (59) and one point ahead of Opus 4.8 (56).
Now the caveats, and they matter.
These are primarily vendor-reported benchmarks. Moonshot evaluates K3 with its own KimiCode harness while comparison models run through Claude Code, Codex, or Terminus 2 depending on the benchmark. Different harnesses mean different tool setups, different retry policies, and different grading. DeepSWE provides a useful sanity check: K3 scores 67.5 with KimiCode and 67.3 with the standard mini-SWE-agent harness, so at least on that benchmark, the harness choice doesn’t swing the result much. But the broader comparison isn’t that clean.
The Fable 5 results are also flagged: requests rejected under Anthropic’s usage policy fall back to Opus 4.8, which means some of the “Fable 5” scores are actually Opus 4.8 scores. FrontierSWE shows the starkest example: K3 at 81.2 vs Fable 5 at 86.6 is a 5.4-point gap, which is substantial.
Artificial Analysis provides the first independent data point. Their Intelligence Index combines nine evaluations across coding, terminal work, scientific reasoning, long-context retrieval, and general knowledge. The blended effective price comes to $2.31 per million tokens, and $0.94 per Intelligence Index task. That’s cheaper per task than GPT-5.6 Sol ($1.04).
The honest reading: K3 is a genuine frontier model that consistently beats Opus 4.8 on coding and agentic tasks, trades blows with GPT-5.6 Sol on several benchmarks, but does not clearly surpass Fable 5 on the hardest software engineering benchmarks. The gaps are often small enough that methodology and harness choices could flip the result.
Coding and Agentic Capabilities: The Case Studies Are the Real Story
The benchmark tables are fine for headlines, but the case studies in the launch post are what actually caught my attention. Moonshot let K3 loose on real engineering problems for extended periods, and the results are hard to dismiss.
GPU Kernel Optimization. Moonshot set up a 24-hour sandbox where models independently profiled, rewrote, and benchmarked four GPU kernel tasks across NVIDIA H200 and alternative-vendor hardware. K3 performed competitively with Claude Fable 5 and substantially outperformed Opus 4.8, GPT-5.6 Sol, and GPT-5.5. This is the kind of task where you need sustained reasoning over hours, not minutes.
Building a Compiler from Scratch. This is the one that made me sit up. K3 built MiniTriton, a compact Triton-like compiler with its own tile-level IR over MLIR, optimization passes, and a PTX code generation pipeline. Not just isolated kernels: a full compiler stack. On supported roofline benchmarks, MiniTriton matches or beats Triton and torch.compile. It sustained end-to-end nanoGPT training with stable convergence, loss curves closely tracking the reference implementation. A model built a compiler from scratch that rivals the extensively optimized Triton stack. Let that sink in.
Chip Design. In a 48-hour autonomous run, K3 designed a chip using open-source EDA tools on the Nangate 45nm library. The result: 4mm², timing closure at 100MHz, 1.46M standard cells, 0.277MB SRAM, and an INT4 MAC array with fused dequantization. Simulated decode throughput: over 8,700 tokens/second. A model designed a chip built to run a model.
Research Automation. K3 reproduced the I-Love-Q universal relations in computational astrophysics in about two hours. That’s work that would typically take an experienced researcher one to two weeks. It cross-validated 20+ papers, evaluated 300+ equations of state, identified inconsistencies in published formulas, and generated 3,000+ lines of Python.
These aren’t toy demos. They’re extended, open-ended engineering and research tasks where the model sustained coherent work over hours or days. This is the long-horizon agentic capability that Moonshot has been building toward since K2.6.
API Pricing and Availability
K3 is available through multiple channels right now:
- Kimi.com (web, iOS, Android, HarmonyOS)
- Kimi Work desktop app (v3.1.0+, Windows and Apple Silicon Mac)
- Kimi Code CLI (select K3 via
/modelcommand) - Kimi API (platform.kimi.ai)
- OpenRouter (for those without a Moonshot account)
The API pricing is straightforward: $0.30/MTok cached input, $3.00/MTok uncached input, $15.00/MTok output.
A few launch constraints worth knowing: Streaming separates reasoning content from the final answer. Multi-turn agents must return the complete assistant message including reasoning and tool-call fields to the next request. These are the kind of details that matter when you’re actually building on top of the API.
K3 Max vs K3 Swarm Max
Moonshot shipped two variants at launch. K3 Max is the general-purpose model for chat, coding, and agent tasks. K3 Swarm Max is built for large-scale parallel processing, coordinating hundreds of sub-agents across massive search and batch operations. The swarm system was already running in Kimi Work; K3 Swarm Max extends it to the API and coding CLI.
Swarm Max is designed for tasks like the 42-year ASIC industry analysis K3 produced: 120+ recursive improvement rounds, 2,800+ web searches, 1,100+ terminal data pulls, 87 quarterly reports, and 99 original PDFs processed across 11,000+ pages. That kind of throughput requires parallel agent coordination, not a single monolithic reasoning chain.
What This Means for Open-Source AI
Here’s the bigger picture. A Chinese startup backed by Alibaba and Tencent just released a model that competes with the best proprietary systems from Anthropic and OpenAI. The weights were published on Hugging Face around July 26-27, 2026, so you could, in theory, download and self-host a 2.8T-parameter frontier model.
Moonshot raised $500M in Series C funding in January 2026 at a $4.3B valuation. They have the capital and the compute to keep scaling.
The open-source AI race is no longer about catching up to proprietary models. K3 isn’t trying to match where Claude and GPT were six months ago. It’s competing with what they are right now. The gap between open and proprietary frontier models has effectively closed on most coding and agentic benchmarks.
That said, “open weights” doesn’t mean “run it on your laptop.” You need 64+ accelerators for efficient inference. The Kimi K3 License is a custom license, not MIT, with attribution requirements for very large commercial products (100M monthly active users or $20M monthly revenue), so read the license file before commercial deployment. Independent benchmarking is still thin: most of what we have is vendor-reported. Artificial Analysis provides the first broad independent evaluation, and it largely confirms Moonshot’s claims, but we need more third-party data, especially on SWE-Bench and real-world coding tasks.
My Take
K3 is the real deal. The architecture innovations (KDA, AttnRes, Quantile Balancing) are genuinely interesting engineering. The case studies demonstrate long-horizon agentic capability that goes beyond anything else I’ve seen in the open-weight world. The pricing makes it practical for production use today through the API.
The caveats are real but manageable. Vendor-reported benchmarks need independent confirmation. Self-hosting needs serious hardware.
The weights and technical report were published around July 26-27, 2026. If the community can reproduce these benchmark numbers with the open weights, and if the vLLM integration works as advertised, K3 becomes a serious option for teams that want frontier-level capability without being locked into Anthropic or OpenAI’s pricing and usage policies.
In the meantime, if you want to try K3 today, Kimi Code in your terminal is probably the fastest path. Give it a real engineering task, not a toy prompt. That’s where this model is supposed to shine.
Corrected on 29 October 2026: active parameters (104B, not ~50B), licence (Kimi K3 License, not Modified MIT), weights now published, modalities, Fable 5 and GPT-5.6 Sol benchmark scores, Artificial Analysis rank (#3, not #4); unsourced figures removed (decoding speedup, AttnRes efficiency, cache-hit rate, RMB pricing, promotion, kernel speedups, token and speed stats, Fable 5 cost per task, funding target, K2 context length, “nine of twelve months”, max-only reasoning).
Found a mistake?
Write to info@howpcrules.com with the post and what is wrong. Corrections are dated at the end of the post.