Grafy News
Try Grafy free

Mechanistic Interpretability Finds Partial Clues to Web Agent Hallucinations

A new arXiv paper shows that linear probes can predict and steer away web‑agent hallucinations.

Illustration for the news story: Mechanistic Interpretability Finds Partial Clues to Web Agent HallucinationsSource: Grafy Research Lab

Mechanistic interpretability (MI) tools were applied to a live web‑navigation agent and managed to predict and partially correct hallucinated actions, but the study reveals why standard MI methods still fall short for agentic failures.

What failure did the researchers focus on?

The Grafy Research Lab paper investigates a concrete failure mode of LLM‑based web agents: the model sometimes generates actions that do not exist on the target website, a phenomenon the authors call web element hallucination. This error is especially problematic because it can lead agents to click nonexistent buttons, submit invalid forms, or waste time exploring dead ends. Understanding why the model invents such actions is essential for improving reliability and safety of autonomous agents that interact with the open web.

How was mechanistic interpretability applied?

The authors used Qwen2.5‑7B‑Instruct as the backbone LLM and let it interact with live sites from the Online‑Mind2Web benchmark. They then deployed a suite of MI techniques that are common in static‑task research:

  • Linear probing – a binary classifier trained on the final token of the prompt to predict whether the upcoming action would be hallucinated.
  • Circuit discovery – attempts to locate specific feed‑forward network (FFN) components whose activation patterns correlate with hallucination decisions.
  • Feature identification – using pretrained sparse autoencoders (SAEs) to find interpretable directions in activation space that might encode the “hallucination concept.”

Each method was evaluated both for predictive power and for the ability to intervene (steer) the model away from hallucination.

What quantitative results were reported?

According to the paper, the linear probe achieved an area‑under‑the‑curve (AUC) of 0.80 when predicting hallucinations before the model emitted any action token. When the model’s hidden state was nudged along the probe’s direction, 85 % of hallucinated actions were corrected, but 7 % of previously valid actions were mistakenly broken.

Circuit‑level interventions were far less effective: targeting the most strongly implicated FFN components fixed at most 8.9 % of cases, indicating that the responsible computation is spread across many parts of the network. Feature‑level analysis using off‑the‑shelf SAEs did not reveal any cleanly interpretable concepts that aligned with hallucination behavior.

Why do standard MI methods struggle in this setting?

The paper outlines several structural challenges that differentiate agentic contexts from the static tasks where MI has historically succeeded:

  1. Distributed computation – Unlike single‑step generation, the decision to hallucinate is a product of many components acting together, breaking the localizability assumption that a single neuron or layer can be held responsible.
  2. Variable input relevance – The specific tokens or observations that trigger hallucination differ across examples, so a probe trained on one prompt format does not generalize when the action list is reformatted.
  3. Non‑stationarity – Counterfactual interventions cause cascading changes in the agent’s state, making it hard to isolate a single causal factor without destabilising the whole trajectory.
  4. Temporal leakage – The paper finds that the signal for hallucination can be present before the action token is generated, meaning the model may have already committed to a faulty plan in earlier reasoning steps.
  5. Feature mismatch – SAEs trained on generic language data do not capture the specialized representations that arise when a model is coupled to a web‑interaction loop, leading to a distributional mismatch between the features the SAE can express and the features the agent actually uses.

These observations suggest that MI tools need to be adapted to handle multi‑step, stateful environments, and to incorporate richer notions of causality that span time and multiple model components.

What does this mean for creators, developers, and researchers using Grafy?

While Grafy does not yet offer a dedicated “circuit‑discovery” UI, the findings can still be explored with existing tools. The Reader app lets you upload the paper PDF, ask targeted questions, and retrieve the exact sections on probe performance or intervention outcomes (see /reader). You can also use Chat to experiment with prompting strategies that the study identified as sensitive – for example, reformatting action lists and observing whether hallucination rates change. By iterating in Chat with the same model (Qwen2.5‑7B‑Instruct or a comparable open‑router model), developers can reproduce the probe‑steering experiment in a sandboxed setting and measure how often valid actions are disrupted.

For teams building their own web agents, the paper’s roadmap offers concrete checkpoints:

  • Benchmark MI on agentic tasks – create a small dataset of hallucination vs. correct actions and train linear probes to gauge predictability.
  • Track belief states across timesteps – use Grafy’s Operator agent to log model activations at each reasoning step, then feed those logs into Reader for analysis.
  • Design prompt‑robust representations – the sensitivity to formatting highlighted in the study suggests that a stable JSON‑like schema for action lists may reduce hallucination risk.

By combining the analytical power of Reader with the rapid prototyping capabilities of Chat and Operator, practitioners can begin to close the gap between diagnostic MI research and practical reliability improvements for their own agents.

A roadmap for future MI work on agents

The authors conclude with several research directions that align with Grafy’s product roadmap:

  • Agent‑centric benchmarks – standardized test suites that capture multi‑step decision making, similar to the Online‑Mind2Web benchmark used in the paper.
  • Beyond single‑component attribution – methods that consider clusters of neurons or whole‑layer dynamics rather than isolated units.
  • Stateful counterfactual construction – tools that can modify an agent’s internal state without causing uncontrolled divergence, a capability that could be built into future versions of Grafy’s Code or Operator studios.
  • Feature discovery for agentic distributions – training SAEs on activation data collected from agents operating in environments, rather than on static text corpora.

These suggestions point toward a future where mechanistic insights not only explain failures but also directly inform safer, more reliable agent designs.

Frequently asked questions

Can linear probes reliably predict web‑agent hallucinations?

The study reports an AUC of 0.80 for a probe trained on the last prompt token, indicating good predictive power but not perfect certainty. The probe works before the action token is generated, showing that hallucination signals are present early in the reasoning process.

Does intervening on the probe direction fix most hallucinations?

Steering the model along the probe’s direction corrected about 85 % of hallucinated actions, but it also broke roughly 7 % of actions that were originally correct, highlighting a trade‑off between fixing errors and preserving valid behavior.

Why didn’t pretrained sparse autoencoders reveal useful features?

The paper finds a distributional mismatch: SAEs trained on generic language data do not capture the specialized representations that emerge when a model is coupled to a web‑interaction loop, so they failed to surface interpretable “hallucination” features.


Source: Grafy Research Lab — Can Mechanistic Interpretability Help Explain LLM Agent Failures? A Case Study on Web Element Hallucination