Grafy News
Try Grafy free

Razavi‑Bench Shows Near‑Perfect AI Mastery of Analog Circuit Reasoning

A new benchmark based on Razavi's analog design questions tracks AI models improving from 28% to 98% accuracy.

Illustration for the news story: Razavi‑Bench Shows Near‑Perfect AI Mastery of Analog Circuit ReasoningSource: Grafy Research Lab

What is Razavi‑Bench and why does it matter?

Razavi‑Bench is a newly released benchmark that measures how well large language models (LLMs) can reason about analog circuit design. It compiles the 50 questions that appear in the two "Analog Design Experiments With AI" papers by Razavi and evaluates model answers against reference solutions and detailed rubrics. According to the Grafy Research Lab pre‑print, the benchmark is intended to give a textbook‑style yardstick for AI progress in a domain that has traditionally resisted pure‑text reasoning.

How was the benchmark built and evaluated?

The authors extracted every problem statement, schematic hint, and performance target from Razavi’s experiments, turning them into a structured prompt set. Each prompt was fed to GPT‑6 Luna, which acted as an automated grader, comparing model outputs to the reference answers and scoring them on a rubric that covers correctness of calculations, identification of key design trade‑offs, and fidelity of the final schematic. The process was applied uniformly across 73 multimodal LLM releases that appeared between May 2024 and November 2026, allowing a longitudinal view of improvement.

What trends does the benchmark reveal about AI performance?

The most striking trend is the speed of improvement. The top‑scoring model in May 2024 achieved just 28.83 % of the possible points, while the best model in November 2026 reached 98.33 % – a jump of nearly 70 percentage points in just 30 months. Notably, the bulk of this gain occurred before the reference answers and rubrics were publicly released, suggesting that model developers were already making breakthroughs in analog reasoning without explicit supervision from the benchmark itself.

Another observation is that the majority of models now answer almost every textbook‑style question correctly. Only the hardest items – those that require constructing a circuit that resembles a textbook example but includes subtle parameter tweaks – still expose occasional gaps. The paper notes that these outliers “show a circuit that resembles a textbook one but …” (the sentence is truncated in the abstract, but the implication is that the models are close to human‑level design intuition).

Which models are now leading in analog circuit reasoning?

While the authors do not name individual commercial products, they report that the leading entries are multimodal LLMs that combine large language capabilities with visual reasoning modules, allowing them to interpret schematic diagrams alongside textual prompts. These top performers consistently surpassed the 95 % threshold on the benchmark and demonstrated the ability to generate complete, simulation‑ready netlists from a single design brief.

The rapid climb in scores aligns with broader trends in the AI industry, where newer model families released after mid‑2025 have incorporated higher‑resolution visual encoders and more sophisticated chain‑of‑thought prompting techniques. As a result, the gap between textbook‑style problem solving and real‑world circuit layout is narrowing.

How can Grafy users explore or leverage these findings?

Grafy provides several tools that let creators, developers, and researchers put Razavi‑Bench concepts into practice:

  • Grafy Chat lets you run multiple LLMs side‑by‑side, making it easy to replicate the benchmark’s comparative setup. You can feed a Razavi question to a selection of models and watch the scores evolve in real time.
  • Grafy Reader enables you to upload the original Razavi papers or the Razavi‑Bench PDF, then ask natural‑language questions about the methodology, rubrics, or specific circuit examples. The AI will surface the relevant sections without you having to scroll manually.
  • Grafy Graph is ideal for tracking prompt variations. You can create a branching graph where each node represents a different prompting strategy (e.g., chain‑of‑thought, few‑shot examples, diagram‑first) and compare the resulting scores on the benchmark questions.

By combining these apps, a researcher can quickly prototype new prompting techniques, evaluate them against the benchmark, and document the workflow for later sharing in the Grafy Community gallery.

What challenges remain for AI in analog design?

Even with near‑perfect scores on textbook problems, the benchmark highlights two open challenges. First, real‑world design often involves constraints that are not captured in the academic questions, such as component tolerances, layout parasitics, and manufacturability limits. Second, the hardest benchmark items still produce circuits that are only similar to the expected answer, indicating that models sometimes miss nuanced trade‑offs that seasoned engineers would catch.

Addressing these gaps will likely require tighter integration between LLMs and circuit simulators, as well as richer multimodal training data that includes layout images, SPICE netlists, and fabrication design rules.

Frequently asked questions

How many questions are in Razavi‑Bench and where do they come from?

Razavi‑Bench contains 50 questions drawn directly from the two Analog Design Experiments With AI papers authored by Razavi. They cover a range of topics from basic transistor biasing to multi‑stage amplifier design.

What does a 98.33 % score represent in practical terms?

A 98.33 % score means the model answered virtually every rubric criterion correctly, producing a circuit that matches the reference solution in topology, component values, and expected performance metrics.

Can I use Grafy to create my own analog‑circuit benchmark?

Yes. Using Grafy Reader you can import any set of design problems, and with Grafy Graph you can construct a branching evaluation workflow that grades model outputs against custom rubrics you define.


Source: Grafy Research Lab — Razavi-Bench: Tracking AI Progress in Analog Circuit Reasoning