PPABench: Verifying Agentic Chip Design from Prompt to Hardened Silicon

PPABench contributors · 2026 · Preprint

Abstract

Large language models can now write RTL and drive chip-design flows, but existing benchmarks score either the source (RTL that lints and simulates) or a single axis — none answer the question a tape-out actually asks: is this a real, manufacturable chip, and how good is it? We present PPABench, a benchmark that grades agentic chip design end to end. A natural-language prompt becomes a core hardened into a fixed OpenFrame/chipignite chassis on the sky130 process, and is scored on both halves of a real chip: a deterministic harden flow (synthesis → place-and-route → DRC → LVS) yields power, performance, and area, while a hidden testbench verifies functional correctness and measures energy. Two design choices make the benchmark hard to game: cycles are counted externally at the wrapper boundary rather than self-reported, and inputs are randomized per run so outputs cannot be precomputed or hardcoded. We define three benchmarks — a fixed-RV32I microcontroller (graded on IPC and energy), an open-ISA core verified by result over random cryptographic input, and a streaming JPEG codec — and report reference baselines on a sky130-hardened picosoc, tracking a full energy-delay-area figure-of-merit set.

Read it

The full preprint is available as a PDF:

For the methodology in brief, see Details; for the live numbers, the leaderboard.