How do you actually train 2.8 trillion parameters?

Stacking parameters up to 2.8 trillion brings its own headaches: a model this large needs information to flow cleanly to every corner during training, and the compute math has to work out at inference time — otherwise you end up with a giant that's both untrainable and unaffordable to run. K3 makes three structural changes to solve this.

The first is KDA (Kimi Delta Attention). Attention is the mechanism by which a model reads text, and with ordinary attention, the longer the text, the faster the compute cost grows. KDA is a more efficient attention backbone — it's what makes reading and processing 1 million tokens of content actually feasible. The second is Attention Residuals (AttnRes). Models like K3 have many internal layers, and the old approach passed information from all prior layers upward as a uniform average; with enough layers, critical early signal gets diluted by the time it reaches the top. Attention residuals switch this to on-demand retrieval — whichever layer's information a higher layer needs, it pulls directly from that layer, with no relay in between.

The third change is sparse experts. Think of K3's core as 896 smaller specialist models — in the jargon, "experts" — under a framework called Stable LatentMoE. For every token it processes, K3 picks only the 16 best-matched experts to do the work, while the other 880 sit idle. That's the reason 2.8 trillion parameters can still serve requests at all: the total parameter count is huge, but only a small slice is actually computing at any given moment.

Coding marathons from 24 to 48 hours straight

K3's headline strength is sustained, largely unsupervised coding: reading large codebases, operating terminal tools, and grinding through hours of work on its own. Kimi backs this up with four case studies, each running longer than the last.

24 hours optimizing a GPU kernel

The first case study is GPU kernel optimization — the core compute routines that run on the graphics card, whose speed directly determines training and inference cost. Each competing model was sealed into an identical sandbox, given up to 24 hours, and left to profile performance, rewrite code, and run benchmarks on its own across four tasks, on NVIDIA H200 hardware plus a general-purpose GPU from another vendor. K3 tied with Claude Fable 5 and beat Opus 4.8, GPT 5.6 Sol, and GPT 5.5 by a clear margin. Two testing conditions are worth flagging: Fable 5's score was run by a third party and may include cases where another model stepped in after Fable 5 declined to answer; and several models took small precision shortcuts in parts of their approach, though all stayed within the allowed error margin.

One detail from development: by the later stages, the team was handing most of its own kernel-optimization work to earlier versions of K3.

Writing a GPU compiler from scratch

The second case study raises the difficulty another notch: K3 was asked to build a GPU programming system from scratch. What it produced is MiniTriton, a small compiler benchmarked against the industry-standard Triton. It built the entire pipeline itself — front-end language, intermediate representation, optimization passes, all the way through to generating PTX code the GPU can execute. On supported benchmarks, MiniTriton matches or beats Triton and torch.compile, and even outperforms Triton on a few tasks. Used to run a full training pass on a nanoGPT model, its loss curve tracks the reference implementation closely, showing this from-scratch compiler chain holds up on real work.

MiniTriton's roofline benchmark on NVIDIA L20: a compiler built from scratch already holding its own against Triton's mature stack. Source: Kimi's official blog

48 hours designing a chip for itself

The third case study is designing a chip for an AI model. K3 ran autonomously for 48 straight hours, using open-source chip design (EDA) tools and the Nangate 45nm process library to take a chip from design through optimization to verification. The chip serves a mini model that itself runs on K3's own architecture — meaning the model designed hardware for its own kind. This is currently just a proof of concept, but the key numbers were all genuinely measured:

2 hours of work that takes a researcher 1 to 2 weeks

The fourth case study is a piece of research work: reproducing a classic result in computational astrophysics called the I-Love-Q universal relations, a set of formulas describing fixed relationships between several properties of neutron stars — verifying it requires reading a large body of literature and writing a full numerical computation program. K3 read and cross-checked more than 20 papers on its own, wrote the complete numerical pipeline, computed over 300 equations of state, caught several inconsistencies in already-published formulas along the way, wrote more than 3,000 lines of Python in total, and shipped a clickable web results dashboard as the final deliverable (see the finished result online). Work like this typically takes an experienced researcher one to two weeks; K3 took about two hours.

Nine playable 3D demos

Games and 3D are where K3's capabilities are most viscerally obvious. Starting from an idea, an image, or a video, it can produce a playable 3D piece outright. It gets there by using screenshots as its own eyes: it writes code, runs it, takes a screenshot to check the result, spots problems in the image, goes back and fixes the code, and repeats the loop until it's satisfied with what it sees.

All nine demos below are K3's own work, each with a playable version online — links are in the captions.

From industry research reports to a self-edited trailer

Beyond coding, K3's other headline strength is knowledge work — research, report writing, slide decks, video editing, and other everyday tasks. Alongside public benchmarks, Kimi also uses an internal evaluation set drawn from tasks real users actually ask AI to do. Keep in mind this suite is both written and graded in-house, so treat it as a reference point rather than gospel.

Internal knowledge-work benchmark: K3 (max) leads consistently across production-style tasks. Vendor-reported. Source: Kimi's official blog

Turning 42 years of industry history into a drill-down website

The heaviest case study here is a research report on 42 years of ASIC (chip industry) history. The final product is an interactive website you can drill down into layer by layer (see the finished site online), with charts, animated diagrams, and narrative pages all built by K3 directly from the evidence it gathered, refined over 120-plus rounds of self-revision. Here's how much research went into it:

Using the same approach, K3 produced two more deliverables. One is a consulting-style research report on the fusion energy industry, complete with timelines, funnel charts, range bar charts, and Gantt charts. The other is an astronomy data analysis: it ran more than 20 sub-agents in parallel to work through all 391 gravitational-wave events, producing 7 scientific charts, 2 data tables, and a literature review synthesizing more than 10 papers. Both deliverables are embedded below — you can page through them directly:

The cover page of the fusion energy industry report (16 pages total). Click through to the official viewer to page through the whole thing. Source: Kimi's official blog

The cover page of the gravitational-wave analysis report (11 pages total). Click through to the official viewer to page through the whole thing. Source: Kimi's official blog

Infographic-style presentations are another strength, with two examples given: a fully editable heatmap deck, and an annual reading-habits report.

Widgets and Dashboard: components born from chat, pinned to a page

Alongside K3, Kimi Work is also shipping two new features. One is Widgets: ask it in chat to build a small component — a data card, a progress tracker, whatever — and that component stays live, refreshing continuously by hooking into local data or external plugins. The other is Dashboard: it collects a bunch of widgets you care about onto a fixed page, pinned around a given project or goal — essentially, you build your own dashboard just by chatting.

Video editing: cutting 56 clips into a trailer on its own

Video editing turns out to be a somewhat unexpected strength this time around. The reason is that K3's multimodality is native: text, images, and video are all understood inside the same model, with no separate model bolted on to help it "see." In the first example, K3 made an explainer video about its own architecture in the style of 3Blue1Brown (the math-explainer channel known for its animated visuals):

In the second example, K3 took 56 raw clips and edited its own trailer from them: choosing which segments to use, where to cut on action, syncing frames to the beat of the music, and handling the audio — the whole workflow was its own, revised across several rounds. For reference: an experienced editor typically needs one to two working days to produce a short film this dense; a novice would need three to five.

31 benchmarks: where it leads, where it still falls short

This round of benchmarking covers 31 tests against Claude Fable 5, GPT 5.6 Sol, Claude Opus 4.8, GPT 5.5, and GLM-5.2. Reading through the full table, the pattern is clear: on anything testing sustained, many-step work, K3 frequently ranks first; on deep reasoning and fine-grained visual understanding, there's a visible gap between it and the strongest closed models.

Overview of the official benchmark comparison. Source: Kimi's official blog

Before reading this table, three testing conditions are worth spelling out. First, all of K3's scores were run at its highest reasoning tier (max). Second, different models ran in different environments — one of KimiCode, Claude Code, or Codex, assigned per benchmark — and for some competitors, the score reflects whichever environment it performed best in. Third, on the PostTrain Bench test specifically, when Claude Fable 5 hits a request its usage policy won't allow, it automatically hands off to Opus 4.8 to finish the job, and the table's score includes that handoff. The full 31 results and 15 footnotes are in the original table.

What third-party testing found

On launch day, independent evaluator Artificial Analysis released its first round of test results, followed quickly by coverage from tech outlets. The overall impression roughly matches Kimi's own numbers: it genuinely made it into the top tier, but the independent testing also surfaced an issue that didn't show up in the vendor's own evaluation.

This round of independent testing corroborates the vendor's own numbers: the Elo score from AA's GDPval retest matches the 1668 in the official table. The commentary lines up with the official narrative too — K3 is well-rounded, its analytical quality approaches Claude Fable 5's level, and GPT 5.6 Sol still leads on presentation polish.

Press coverage clustered around two angles. The first is open source catching up to the frontier itself: Axios quoted Mozilla CTO Raffi Krikorian's view that U.S. AI labs are "clearly worried" about open-weight models like this one — otherwise they wouldn't bother lobbying Washington to restrict them; VentureBeat called it the largest open-source model yet, closing in on top U.S. systems. The second is price: The Decoder ran with a headline flatly declaring the end of the era of super-cheap Chinese AI, since K3's pricing runs more than three times what its own predecessor charged.

Pricing, access points, and the open-source timeline

Want to use K3 right now? All five access points are open today. Want the weights to self-host? You'll need to wait for the open-source release by July 27. API pricing is already fully public:

• App / web: update to the latest Kimi from the app store (iOS, Android, HarmonyOS), or just head to kimi.com
• Kimi Work: available from desktop version 3.1.0 onward (Windows and Apple silicon Macs)
• Kimi Code: run Kimi Code in your terminal and select Kimi K3 with the /model command
• API: select kimi-k3 at platform.kimi.ai
• Enterprise: enterprise-grade data isolation and member management, with personal and organization accounts fully separated

Is this price a hike or a bargain?

Compared with its own predecessor, this is a real price hike. K2.6's official rate was $0.95 for input misses and $4.00 for output; K3 raises those to $3.00 and $15.00 respectively — a two-to-threefold jump. It does use fewer tokens for the same work (21% less output than K2.6 across the same evaluation set), but the savings fall well short of the increase, so the cost per task still ends up higher.

Against closed-source rivals, though, it's not expensive at all: it matches Claude Sonnet 5 exactly, runs at 60% of GPT 5.6 Sol's input price and half its output price, and is far cheaper than Claude Opus 4.8 or Claude Fable 5. Artificial Analysis ran the numbers on actual task execution: K3 averages $0.94 per task, roughly on par with GPT 5.6 Sol's $1.04, and about half of Opus 4.8's $1.80 — though it costs considerably more than open-source peers like GLM-5.2 ($0.32) and DeepSeek V4 Pro ($0.04).

Units: USD per million tokens. Figures come from each vendor's official pricing page (compiled by The Decoder); the two Anthropic models' cache-hit prices are converted using Anthropic's 0.1x cache-read rule.

For anyone planning to self-host once the weights are out, here are a few practical details. The model has been trained at low precision since the fine-tuning stage (MXFP4 weights paired with MXFP8 activations), so the released weights are natively compatible with more hardware without an extra round of quantization. On deployment scale, Moonshot recommends a superpod built from 64 or more accelerators — you need that much interconnect bandwidth to actually exploit the sparse architecture. There's also an ecosystem move worth noting: KDA, being a new attention structure, didn't play well with the caching mechanism in vLLM, the open-source inference framework most people use to deploy models — so Kimi contributed the adaptation directly to the vLLM community, and it'll ship alongside the model. This also explains, in passing, how 2.8 trillion parameters can still sell at the price quoted above: KDA plus prefill caching is what keeps serving costs down.

Three known limitations worth knowing before you use it

K3 has three known limitations, all of which directly affect the experience of using it, and are worth noting exactly as reported.

This time, Moonshot pushed open-model scale to the 3-trillion tier for the first time: Kimi K3 has 2.8 trillion parameters (parameters are the internal "dials" of a model — more of them usually means stronger but harder to train), plus native image understanding and the ability to read over a million words in one pass. It topped a batch of tests measuring "sustained work over long stretches," closing in on the strongest closed models overall, though not quite catching them yet. The app, web, and API (the interface other programs call into) are all live today, with weights out by July 27.

But here's a serious problem: 2.8 trillion parameters is an astronomical figure. What makes a model this big trainable at all — let alone servable to millions of people at an ordinary price?

Three structural changes are what make 2.8 trillion parameters trainable and affordable: KDA (which keeps text spanning over a million words computationally feasible), attention residuals (which stop critical information from being diluted layer after layer in a very deep network), and the easiest one to grasp — splitting the model into a room full of experts and waking only a handful each time. The diagram below shows exactly that:

"N times faster" doesn't mean much on its own — translate it into how long the same job actually takes, and it clicks. One case study from Kimi: reproducing a classic astrophysics study (reading 20-plus papers, computing 300-plus equations, writing 3,000-plus lines of code) usually takes an experienced researcher one to two weeks. K3 did it in about two hours.