Grafy News
Try Grafy free

AI Coding Adds Code, Delays Review

Harvard researchers found that AI coding assistants raise code output by 30 % but increase code‑review time by 49 %, leaving finished work unchanged.

Illustration for the news story: AI Coding Adds Code, Delays ReviewSource: Hacker News — front page

Engineers using AI‑powered coding assistants are writing more code, but the extra output is getting stuck in review, a new Harvard working paper shows. The study, authored by Ph.D. candidates Fiona Chen and James Stratton, measured a 30 % rise in code added per worker‑month while code‑review latency grew by roughly 49 %, with no measurable increase in finished work.


What did the Harvard study find about AI coding agents?

The researchers examined activity from 718 firms that use Jellyfish’s engineering‑analytics platform, linking it with LinkedIn employment data collected by Revelio Labs. The sample spans January 2021 to March 2026 and includes 68.3 million Jira issues, 200.9 million commits, 49.1 million pull requests and 21.3 million calendar events, though only 213 firms supplied calendar data. Using a staggered difference‑in‑differences design, they compared firms that had adopted AI coding agents with those that had not yet done so.

According to the paper, engineers at adopting firms added about 1,494 lines of code per worker‑month – a 30 % lift over the 5,017‑line baseline. Commits rose 20 % and pull‑request submissions grew 23 %. These figures capture the incremental effect of agents in environments that already used some form of AI assistance.

How did the increased code output affect finished work?

The study defines “finished work” as Jira issues resolved and deployed to production. Despite the surge in coding activity, the number of resolved issues moved only 0.12 per worker‑month on a 3.67‑issue baseline, a change that is statistically indistinguishable from zero. The 95 % confidence interval excludes any increase larger than about 12 % of the baseline, far below the 30 % rise in code written.

To rule out the possibility that engineers simply tackled larger tasks, the authors sampled 0.5 % of issues, labeled their size with GPT‑4o, and extrapolated using a text‑embedding model. The analysis showed no shift in average issue size, and pull requests did not become larger either. In short, the extra code did not translate into more completed features or products.

Why did code review become the new bottleneck?

The paper pinpoints the slowdown to the review stage. The average time from pull‑request submission to merge increased by 3.45 days on a 7.03‑day baseline – a 49 % rise. Formal change requests jumped about 12 percentage points from a 13 % baseline, nearly doubling. Review comments per pull request rose 35 %, and the share of workers doing any code review grew 14 % from a base of roughly 29 %.

Even though nearly 80 % of the firms had deployed at least one AI‑assisted review tool by March 2026, those tools generated only 23.3 % of review comments, and just 10.8 % of pull requests received an AI comment. A junior engineer interviewed for the paper described a workflow where a single senior engineer could approve changes only once a week, and new rules requiring senior approval for AI‑generated commits further congested the pipeline.

The authors model two channels that explain the bottleneck. First, faster drafting sends more code to a fixed pool of reviewers, overwhelming capacity. Second, the rise in change requests and comments suggests lower‑quality drafts, a problem that faster drafting alone cannot solve. Distinguishing which channel dominates is crucial for firms deciding where to invest next.

What implications does this have for legal AI tools?

The timing of the study coincides with legal‑tech announcements from Harvey and Cooley. Both products automate an initial draft—Harvey for motions to dismiss, Cooley for redaction recommendations—then place a lawyer at a final checkpoint before filing. The Harvard findings imply that, just as in software, accelerating the drafting phase may shift the bottleneck to the human review step.

In the legal context, the checkpoint carries a statutory duty under Federal Rule of Civil Procedure 11(b). Even if an AI produces a draft in hours, the attorney must still conduct a reasonable inquiry, meaning the review stage could absorb the time saved in drafting. Moreover, the paper notes that AI authorship leaves metadata (e.g., “Co‑authored‑by” lines) in collaboration systems, which information‑governance teams may need to capture for retention schedules and legal holds.

While the study focused on software engineering, the pattern suggests that any domain that inserts a human‑centric verification stage after AI‑generated output may encounter a similar slowdown. Legal teams should therefore measure not only drafting throughput but also reviewer time, revision rates and approved work product when evaluating AI tools.

How can Grafy users explore or address these findings?

Grafy provides several ways to investigate the productivity dynamics highlighted by the Harvard paper.

  • Grafy Code lets developers experiment with AI coding assistants in a controlled environment. By tracking commit frequency, pull‑request volume and merge latency within a project, teams can surface the same metrics the study used and see whether review time is becoming a constraint.
  • Grafy Reader can ingest the full working paper (or any PDF of a related study) and let users ask natural‑language questions about methodology, results or confidence intervals, without manually scrolling through tables.
  • Grafy Graph enables users to map the workflow of code creation, review and deployment as a branching graph of prompts, model outputs and timestamps. Visualizing the pipeline can reveal where bottlenecks appear and help teams prototype process changes, such as adding an AI‑assisted reviewer.

By combining these apps, a development team can run a small‑scale A/B test: one branch uses an AI coding assistant alone, another adds an AI‑driven review bot. The Graph view will expose differences in cycle time, while Reader can summarize statistical significance. The insights can then inform budgeting decisions—whether to invest in faster drafting tools or in richer review‑automation.


Frequently asked questions

Did the Harvard study show that AI coding agents reduce overall engineering headcount?

No. The authors found no evidence of a significant drop in engineering employment. Their estimates rule out a decline larger than 2.9 % in overall employment and 13.8 % in engineering‑specific roles, suggesting the productivity gains did not translate into immediate layoffs.

Are the bottlenecks observed in software development expected to appear in legal AI workflows?

The study itself examined software only, but the authors note that legal tools like Harvey and Cooley follow a similar “draft‑then‑human‑checkpoint” design. Because the human checkpoint remains essential for judgment and compliance, a comparable slowdown in review is plausible, though it has not yet been measured in legal contexts.

Which Grafy app should I use to analyze my team’s code‑review latency?

Start with Grafy Code (/code) to collect commit and pull‑request data, then visualize the workflow in Grafy Graph (/graph). For deeper document‑level queries, you can load your project’s logs into Grafy Reader (/reader) and ask specific questions about cycle times or reviewer workload.


Source: Hacker News — front page — Harvard researchers find AI coding agents shifted the bottleneck to code review