NVIDIA Research releases HORIZON: an unattended agent takes the entire RTL chip design benchmark suite to 100% pass rate
- NVIDIA Research proposes HORIZON, which manages hardware design as repository-level code evolution: the user writes a single Markdown spec, the system automatically compiles it into a "project pack," and an unattended agent loop repeatedly edits code and runs simulations inside an isolated Git workspace, committing only once verification passes.
- Across ChipBench, RTLLM-2.0, Verilog-Eval, and 9 CVDP verification categories, HORIZON uses the same unattended loop to push every benchmark to a 100% pass rate.
- The agent's aggregate pass rate on the first iteration is only 47.8%, and the hardest CVDP checker-generation category starts at just 3.8% on the first pass; the same loop eventually grinds every category up to 100%, but some tasks clear in 1–2 rounds while others take 82 rounds of iteration.
- Once correctness saturates, token cost becomes the real differentiating signal — the 9 CVDP categories consume 97.1% of all tokens (about 203.9M), roughly 91% of which are reusable cached input tokens.
- The authors explicitly do not claim that "agentic hardware design is solved," and point out that current feedback mechanisms carry an "overfitting/reward hacking" risk, recommending that future benchmarks separate debugging feedback from the final hidden scoring.
What this NVIDIA paper says
NVIDIA Research recently published a paper proposing HORIZON, an agentic hardware design framework that treats chip design as repository-scale code evolution.
Give the system one Markdown spec, and a fully unsupervised agent repeatedly edits code and runs simulations inside an isolated Git workspace, committing only once verification passes — eventually pushing the entire RTL (register-transfer level hardware design, the layer of code that describes exactly what a chip does on every clock cycle) benchmark suite to a 100% pass rate.
Why having an AI write Verilog once falls far short
Verilog is the hardware code that describes chip logic. RTL design is a sharp touchstone: whether a module is correct depends on the behavior of every clock cycle, the reset convention, how the interface handshakes, the bit width of every signal, and what the simulator's feedback actually shows. So "Verilog that looks plausible" is nowhere near enough.
The model has to infer the data path's bit widths on its own, how the state machine transitions, the reset convention, ready-valid handshake protocols, how memory gets read and written, plus all the edge cases that natural language often fails to spell out. Put simply, a syntactically correct module is only the starting point. Real, useful automation has to wire "writing it" into the full chain of "compile, simulate, inspect waveforms and failure traces, then fix" — running the loop, spotting what's wrong, going back to fix it, and running again.
HORIZON's core idea is to lock this entire chain inside a loop that can turn itself over.
How a spec turns into a self-running project pack
The only input the user has to provide is a structured Markdown spec, not code. It spells out high-level intent, repository context, expected deliverables, evaluation criteria, and domain knowledge. Domain knowledge is especially useful, because it lets you make explicit the invariants, tooling conventions, and failure modes that "just looking at the files" can't reveal.
A "bootstrap agent" compiles this spec into a "project pack," which serves as the control plane for the whole system. The spec has four components, which get compiled into a five-element runtime configuration that drives the unattended loop.
- Goal / task
- Domain-knowledge directions
- Evaluator specification
- Acceptance criteria
π_agentpolicy prompt and tool contractE_pexecutable evaluator (compile / simulate / coverage)A_pacceptance predicate (can it be committed)Γ_pversion control and artifact policyΩ_pdomain skills and repository operating instructions
- Read state
- Edit workspace
- Call tools to evaluate
- Fix or commit
Think of a project pack like a complete onboarding packet for a new hire: it doesn't just spell out what to do, it also attaches the evaluation criteria, tool usage instructions, and handover procedures. Once the employee reads through it, they can start working independently, with no one standing over their shoulder.
The key is that the problem is defined over a generic Git workspace, not some fixed chip-repository format. So this organizing scheme isn't exclusive to RTL or EDA: any task with "a persistent Git workspace + machine-checkable feedback + versioned artifacts" can be organized the same way. Swap domains, and the evaluator slot can be replaced with unit tests, a theorem prover, a profiler, a security scanner, a synthesis tool, or even a human review gate.