Coding agents often solve early but keep running, study finds
A new arXiv study shows 38‑82% of AI coding agents produce a correct patch on their first edit but then waste up to 60% of steps before stopping.
Source: Grafy Research LabWhat is termination lag and why it matters?
The term termination lag describes the gap between an autonomous coding agent’s first correct repository‑level patch and the moment it finally stops running. According to the arXiv paper "Termination Lag: Coding Agents Solve Early but Stop Late" (Cim et al., 2026), this lag can account for a sizable share of an agent’s total steps, inflating token usage, latency, and cloud cost while also risking the loss of a previously correct fix.
How did the researchers measure early correct patches?
The authors instrumented two agent frameworks – OpenHands and mini‑SWE‑agent – to record the full repository state after every action. Each distinct patch was evaluated offline against the official SWE‑bench tests, which the agents never see during execution. This approach let the team identify the time to first correct patch (TTFCP) and the subsequent termination lag (TL) without influencing the agents’ behavior.
Key findings across models and datasets
- Early success is common. Between 38 % and 82 % of successful runs produced a passing patch on the very first source‑code edit, with the highest rate observed for Qwen3‑Coder on SWE‑bench Verified (82 %).
- Agents keep working after solving. In the same successful runs, post‑solution steps made up a median of 35‑37 % of the total steps for Qwen3‑Coder and 54‑60 % for Laguna‑XS.2.
- Verification dominates the lag. The extra steps were largely spent on self‑verification (48‑63 %) and repository exploration (15‑29 %). Direct source‑code edits after the first correct patch were rare.
- Occasional regression. In 1‑8 % of cases, agents overwrote a correct patch later in the run, ending with an incorrect final submission.
Figure 1 shows how Qwen3‑Coder’s first correct patch often appears early, yet the agent continues for many more steps before halting. Figure 5 illustrates a similar pattern for Laguna‑XS.2, confirming that termination lag is not limited to a single model.
Why do agents continue after a correct fix?
Coding agents lack direct access to the benchmark test suite; they must infer correctness from their own evidence, such as running local tests, inspecting call graphs, or re‑executing commands. The study found that agents devote a large share of their post‑solution activity to these verification tasks, trying to increase confidence before deciding to stop. While some verification is prudent, the data suggest that agents often over‑verify, leading to diminishing returns and, in a minority of runs, to accidental regressions.
Implications for developers and researchers
- Cost awareness. Each extra step incurs model calls, token consumption, and compute time. Understanding TL helps teams estimate hidden costs that final‑patch metrics hide.
- Designing better stopping criteria. A practical early‑termination rule could examine signals such as passing local tests, stable diffs, or lack of further repository changes. The TL bounds the maximum savings achievable by such a rule.
- Safety considerations. The small but real risk of later edits breaking a correct patch argues for mechanisms that lock in a known‑good state once confidence is high.
- Benchmark reporting. Future SWE‑bench evaluations might report both final‑patch success and TTFCP to give a fuller picture of agent efficiency.
How to explore termination lag with Grafy
Grafy Code lets you run autonomous coding agents in a controlled environment, inspect each intermediate repository state, and compare patches against custom test suites. By replicating the paper’s methodology—recording the repository after every step—you can measure your own agent’s termination lag. For deeper analysis of the recorded patches, Grafy Reader can ingest the run logs (PDF or JSON) and let you query when the first passing patch appeared. Together, these apps give creators a hands‑on way to experiment with early‑stop heuristics and cost‑saving strategies.
Source: https://arxiv.org/abs/2611.14960
To explore this yourself, open Grafy Code: an AI coding studio.
Frequently asked questions
What is the typical percentage of steps that occur after the first correct patch?
In the core experiments, the median termination lag ranged from 35 % to 60 % of total steps, depending on the model and benchmark.
Do all coding agents eventually overwrite their correct patches?
Only a small minority—between 1 % and 8 % of successful runs—ended with the correct patch undone, according to the study’s analysis.
How can Grafy help me reduce termination lag in my own coding agents?
Grafy Code lets you capture each intermediate patch, while Grafy Reader can query those logs to pinpoint the first passing edit. By analyzing these signals you can design custom stopping rules that halt the agent once sufficient verification is achieved, cutting unnecessary steps and cost.
Source: Grafy Research Lab — Termination Lag: Coding Agents Solve Early but Stop Late
More news
WordInferBench tests AI agents on inferring Word formatting
A new benchmark, WordInferBench, challenges AI agents to infer formatting changes from Word document context.
guardscope Framework Quantifies Runtime Control Coverage for LLM Agents
A new arXiv study introduces guardscope, a two‑stage framework that measures how many of an LLM agent's action paths are inspected by runtime controls.
Study Reveals LLM Agents Mostly Browse Instead of Acting in Public‑Service Tasks
Researchers analyzed 2,400 runs of ten LLMs on four public‑service scenarios and found agents read files 97% of the time but only produce useful output.