Twelve Rules of Harness Engineering: A Google Principal Engineer's Year of Practice, Distilled Into Processes and Templates You Can Use Today
- Ryan Lopopolo is now a Principal Engineer at Google Cloud; before that he was at OpenAI. He distilled a year of work into twelve rules and published them as a public repo that can be fed directly to your own Agent as reference material; most of the evidence behind the twelve rules comes from his time at OpenAI.
- The core move is counterintuitive: treat the chosen model and coding Agent as an immutable black box — don't tune a single parameter — and only change the two things around it: context and tools.
- The twelve rules split into three layers. Three set the boundaries: freeze the worker, feed in private process data, measure by real results. Six are the actual levers: hand one Agent the whole job, route context just in time, make capabilities usable, let the repo teach itself, spell out authority, prove it in a real environment. Three govern compounding: turn feedback into infrastructure, protect long-term coherence, run settled work as a standing loop.
- The evidence behind all this: an internal product that grew from an empty repo to roughly a million lines of code in five months, across roughly 1,500 merged PRs, all written by Codex — no human wrote a single line by hand.
- What you can take away directly: a six-step improvement process, three fill-in-the-blank templates, an eleven-row table of "whatever you claim, that's where you need to gather evidence," and a nine-item checklist of "when your A/B test doesn't count."
Twelve rules, and a word that gets mistranslated
Let's get the word right first. Harness here means tack — bridle, reins, horseshoes, straps, the whole rig — and the repo's description opens with a horse emoji. It's easy to read as "restraint," which points exactly the wrong way.
The horse is already capable enough. What's missing is the bridle, the reins, the track — the rig that lets its power actually reach the ground and go where it needs to go. Harness engineering is the work of building that rig.
The person behind this is Ryan Lopopolo. He's now a Principal Engineer at Google Cloud working on agentic GCP. Before that he was at OpenAI, where he helped build out the Seattle office and OpenAI Frontier for enterprise — work that meant equipping Agents with context, tools, authority, evals, observability, orchestration, and feedback loops. Most of the evidence in the twelve rules comes from that stretch. Earlier still, he was tech lead for the data marketplace at Snowflake, led a 40-person team supporting a 350-person engineering org at Brex, and was tech lead for infrastructure and payments at Stripe as the company grew from 600 to 6,000 people. In his spare time he wrote a Ruby implementation in Rust called Artichoke.
When he published the twelve rules, he described them as a distillation of the past year of his work — practice, and technique.
He condensed his own practice into twelve rules, packaged with evidence, case studies, and executable processes, in a public repository; the original content he authored is licensed CC BY 4.0. There are two ways to use it: read it yourself, or hand it — along with the system you're trying to improve — to a coding Agent and let the Agent use the repo's own routing index to find the right rule to consult.
Why this is worth your time: every one of the twelve rules is backed by verifiable evidence. An internal product grew from an empty repo to roughly a million lines of code in five months, across roughly 1,500 merged PRs, all written by Codex with no human hand-writing code; a 60-hour overhaul that burned through 300+ million tokens ran on nothing but the initial prompt plus two follow-ups; and after turning on one lint rule (a tool that automatically flags code that violates a convention), roughly 600 violations got fixed and given test coverage in a single PR.
Here's the full picture of the twelve rules. They're grouped into three layers to make the relationships clear: the top three set the boundaries, the middle six are the actual levers, and the bottom three determine whether any of this compounds over time. Each gets its own section below.
Rule one: freeze whoever's actually doing the work
Within a single eval or deployment cycle, the model, the coding Agent's runtime, and its native tool interface all stay frozen. If any one of the three changes — even a generational bump — the whole environment needs re-validating from scratch.
The payoff of freezing the black box is that failure becomes attributable. When a genuinely capable Agent can't get a job done, the reason usually lands in one of five places: missing relevant institutional context, a missing capability that actually works, a missing feedback loop, missing authority, or missing evidence at the point of delivery. All five are properties of the deployment environment — all five can be inspected and fixed. Without a frozen black box, you're stuck endlessly relitigating "is the model bad, or did I just configure this wrong."
Your own accumulated instincts are also up for re-validation. Habits an old model taught you — how small to slice a task, how much orchestration is needed, how long the inner loop can wait, where a human absolutely has to watch — carried over unchanged to a new generation, and whatever the new model can newly do goes to waste. The opposite assumption — that every point release gets stronger on every dimension — is an error in the other direction.
One observation he noted himself: GPT-5.3 was less willing than GPT-5.2 to grind through hard work. That's a single usage observation, not a claimed trend.
Two numbers show how much a generational jump can matter. GPT-5.2 shipped while he was on vacation; he came back to find the team producing one to two extra PRs per engineer per day, with zero additional investment in the environment. A separate Symphony environment overhaul later reported a tenfold increase in PRs per engineer per week. The first number came from a model upgrade, the second from an environment overhaul — the two need to be booked separately, which is exactly the point of freezing the black box.
Not a single line of the environment changed during the GPT-5.2 rollout.
One to two extra PRs per engineer per day.
The Symphony intervention — the worker didn't change.
Reported tenfold growth in PRs per engineer per week.
Inner-loop latency is also part of what gets re-validated. How far an Agent can reason depends on how much feedback it gets within a single trajectory (one trajectory being an Agent's full run at a task, start to finish) before that run falls apart. The GPT-5.2-era Codex environment had no background shell, so the Agent would patiently wait out blocking build scripts; once GPT-5.3 added a background shell, it stopped being willing to wait. The team spent a week migrating the repo's build off homegrown scripts and onto a proper build tool (trying Bazel, then Turbo, then Nx, in that order), and stopped once a full build came in under one minute. That one-minute mark was set as a hard ceiling — cross it, and the team drops what it's doing to fix the build graph.
Validation also cuts things. To let the Agent see the interface, the team had built a whole rig — a virtual display plus screen recording — from scratch; once the model gained native computer use (the ability to see the screen and click for itself), the entire rig was deleted, and end-to-end verification didn't lose anything in the process.
Rule two: deploy the submerged part of your private process data
What a general-purpose model's weights hold is only the tip of an organization's knowledge. Below the waterline sits what the work actually runs on — and that part is private, and it keeps changing.
This rule requires spelling out two things clearly. One is the data ontology: what entities, metrics, and identifiers exist, and how they relate. The other is the work ontology: what outcomes, workflows, roles, tools, exceptions, evidence, and authorization relationships exist.
One example makes the point well. An internal data Agent, given only access to the data warehouse, still can't resolve the fact that "revenue" and "active users" mean different things to different teams inside the company. It also needs to know what product lines the company has, how customers are segmented, what teams exist, which customers are in pilots, what the current plan is, and how shared metric definitions are written. Only with those relationships in hand can it give an answer that's actually specific to the company.
Current records stay in the data warehouse, the ticketing system, the docs system, logs, the codebase, and customer systems. Use search and connectors to pull an authorized slice — don't create a second copy that will go stale.
There's an organizational pattern here worth copying directly. Basis runs two repos: Arnold is the production monorepo, and Atlas is a second repo dedicated to company context, kept outside the production code tree. Codex can combine personal notes, company operating knowledge, and production code together, while each set of information keeps its own independent update path. The implementation can be as plain as it sounds — in his words, "it's just Codex plus a Git repo full of Markdown."
One more thing belongs in this layer: settled decisions need to be promoted up to whichever layer actually governs them. One team, together with security, picked an approved encryption implementation — and the decision stayed in Slack. Later, a new engineer working with Codex had Codex install a different npm crypto package. The team pulled the earlier decision back out, wrote it into the repo as a guard rule, and re-ran that change.
Rule three: give one Agent the whole job
What gets delegated is the outcome, handed entirely to a single primary trajectory that owns decomposition, execution, integration, evidence, and safe wrap-up. The Agent decides its own steps. A human only steps in for three things: direction, judgment, and authorization with real consequences.
He often writes prompts with close to zero constraints — he calls these "lazy prompts," lazy in the sense that method is left blank and the Agent decides how. A sparse delegation doubles as a test: it tests whether your environment lets the real requirements be something the Agent can find its way back to on its own. Whenever the Agent misreads the intent, that exposes exactly which part of the environment isn't built out yet. The matching loop is "propose, observe, improve, rerun": read the trajectory, fix the environment, throw away this run's result, and run it again from scratch.