Anthropic Analyzed 400,000 Claude Code Sessions: Domain Expertise Matters More Than Coding Skill
- Anthropic analyzed roughly 400,000 Claude Code interaction sessions from October 2025 to April 2026, covering about 235,000 users
- In a typical session, users make about 70% of the "what to do" decisions (planning), while Claude makes about 80% of the "how to do it" decisions (execution)
- The higher a user's expertise on a given task (not their coding background), the higher the session success rate: verified success runs 28% to 33% for experts, versus just 15% for novices
- Among sessions that produce code, the top 10 professional categories differ from each other by no more than 7 percentage points — all landing close to software engineers
- Over 7 months, the share of sessions spent fixing bugs dropped from 33% to 19%, sessions for ops, documentation, and data analysis roughly doubled, and average task valuation rose about 25% to 27%
Across 400,000 conversations, who's really calling the shots — human or AI?
Anthropic recently published a study analyzing roughly 400,000 Claude Code interaction sessions from October 2025 to April 2026, aiming to answer one question: can people with no coding background direct an AI agent to complete complex technical work?
This is the largest public analysis to date of real-world Claude Code usage — around 400,000 interaction sessions, roughly 235,000 users, spanning 7 months. The research zeroes in on one thing: what actually determines whether usage goes well.
Why it's worth reading: two findings echo each other. Among sessions that produce code, the 10 largest professions all land within 7 percentage points of software engineers in success rate; meanwhile, the share of expert-level users who reach "verified success" (28% to 33%) is nearly double that of novices (15%). Going by this data, what decides the outcome is how well a person understands the task at hand — not whether they can code.
How do you analyze this many real conversations without violating privacy? The study uses privacy-preserving analysis: a Claude model automatically reads each de-identified session transcript and applies labels, which are then cross-checked against telemetry the system already logs automatically (for instance, whether the session actually added or removed lines of code). Researchers never read any individual's specific chat history — they only look at aggregate patterns across a large population of users. The two data sources line up well: over 90% of sessions the AI classified as having "created or modified code" did in fact show code changes in the telemetry.
Scope note: this study covers only interaction sessions via the CLI, Claude.ai, and the Claude Code desktop app. It excludes headless mode via single claude -p commands, as well as bulk usage through third-party dev environments or SDKs.
Users set direction, Claude does the legwork
The study splits every decision in a session into two categories. Planning decisions cover "what to do, which path to take, what counts as done." Execution decisions cover "which file to edit, what code to write, which language to use, which command to run." A classifier then judges whether each decision was made by the human or by Claude, producing two figures per session.
In plain terms: humans set direction, AI does the detail work. This division of labor shows up consistently across sessions.
Looking at it another way — swap "decisions" for "actions." A Claude Code session is a back-and-forth between human and Claude: the user writes an instruction, Claude goes off and does a string of work, then the user writes the next instruction. A typical session runs about 4 rounds of back-and-forth; historically, each user instruction triggers about 10 Claude actions on average, sometimes over 100. Each round, Claude reads files, edits code, runs commands, and writes roughly 2,400 words of output on average.