MicroLens Lets Researchers Read and Rewire AI

MicroLens is an interactive viewer that lets researchers read, trace and edit hidden states inside a language model. According to its authors, the tool matters because a decoded-word heatmap alone cannot show whether a model actually uses the information the vector carries.

What MicroLens Does

The project comes from a research paper and an interactive viewer described in it. The focus is the residual stream, the continuing vector state that a transformer updates as layers process input. The authors note that Gemma-3-4B-it has 34 layers and 2,560-dimensional vectors. Recording each token vector after each layer creates a grid where tokens form columns and layers form rows, from input embeddings at the bottom to the final layer at the top.

A lens applies a readout to a cell and produces a distribution over words. The logit lens uses the model final norm and unembedding. The Jacobian lens, or J-lens, uses an average Jacobian from a layer toward the top and can read an idea several layers earlier. The viewer allows a user to choose a lens, inspect a cell, trace its sources and edit the cell, then rerun the rest of the model.

The authors describe an open-source, local viewer for Hugging Face decoder models that combines two lenses, attention-flow tracing, four kinds of cell edits and chain-of-thought tracking on one grid. That combination is meant to make a reading directly testable. The viewer is narrower than some broader transparency tools, but it is built around the residual-stream grid and makes every cell both readable with two lenses and editable, so a reading can be tested and traced.

Why Two Lenses and Editing Matter

The source describes a common problem. Lenses that decode hidden states into words are popular, but a heatmap built from decoded words only shows what a vector reads as. It cannot tell whether the model uses the information or how much it uses it. The authors say causal methods can test this, citing activation patching, steering and attribution graphs, though those methods are often run as scripts separate from lens intuition.

The authors say the heatmap invites two mistakes. A word that is readable may play no part in the model computation, and a word that matters may not be readable. MicroLens tries to put reading, tracing and editing on the same layers-by-tokens grid. The paper frames this as bridging the gap between lens intuition and causal tests that confirm it.

Different lenses provide edit directions that are useful for different edits. To set what the final token outputs, the logit lens direction is used. To change what later tokens read, the J-lens direction is used, and it works only where the J-lens can read the word.

What the Gemma Experiments Found

The team demonstrated three usage scenarios on Gemma-3-4B-it, backed by sweeps over a few hundred prompts. In two-hop trivia questions, the intermediate entity was readable at the city token from as early as layer 7, and swapping it there could redirect the answer 92 to 98 percent of the time. At the answer position, the source says the state or country information was not used.

In step-by-step reasoning, the correct answer was often already the top word in the residual stream before reasoning tokens were written. The paper reports top answers for 89 to 92 percent of retrieval questions and 39 to 63 percent of multi-hop and serial questions, far above distractors at or below 5 percent. Ordering puzzles were an exception. For answers visible in the residual stream, the J-lens read them a layer or two earlier than the logit lens across different question types.

The authors say the findings guide edit design. They also say that in two-hop recall the intermediate entity was readable at two tokens, but editing showed the model used it at only one. A lens reading alone, they argue, cannot reveal that.

How the Tool Works

The viewer records attention and MLP writes, per-head attention weights and token-level vectors. Clicking a cell can list both lenses top words and split the block update into what was kept from below, what attention wrote and what the MLP wrote. It can show which earlier tokens and heads the attention write came from, weighting sources by vector contribution rather than raw attention weight alone.

Edits include push, swap and copy from another prompt. Push moves a cell toward a word direction. Swap replaces an old word with a new one. Copy from another prompt replaces a cell with the same layer and token position from a different prompt. The rest of the model can be rerun and compared. The paper notes that the what changed shading shows how far an edit spreads, due to causality, only cells above and to the right can move.

The source describes four edits. The examples include push, swap and copy from another prompt. The what changed shading shows how far the edit spreads because of causality, with only cells above and to the right able to move.

MicroLens and Grafy

For creators and developers in the Grafy ecosystem, the most relevant existing surfaces are Reader and Chat. Reader is the AI research tool for PDFs and papers, so it could be used to study the MicroLens paper or related literature. Chat is the multi-model workspace, so it could support experiments comparing how different models handle traced edits. The source does not describe a native MicroLens feature inside Grafy.

Frequently asked questions

Does MicroLens teach a smaller language model?

The paper does not claim that MicroLens teaches a smaller model. Its viewer is described for decoder models, and its examples use Gemma-3-4B-it. The experiments focus on reading, tracing and editing residual streams. A claim about teaching smaller models would require a different source.

Does the J-lens read an answer earlier than the logit lens?

According to the paper, when an answer was visible in the residual stream, the J-lens read it about a layer or two earlier than the logit lens across the tested question types. The J-lens uses an average Jacobian through higher layers, while the logit lens applies the final norm and unembedding. The source reports this timing difference.

Can swapping the intermediate token redirect a two-hop answer?

The paper reports that in two-hop trivia questions, the intermediate state or country was readable at the city token from layer 7, and swapping it there redirected the answer 92 to 98 percent of the time. At the answer position, the source says the information was not used. The experiments used Gemma-3-4B-it.

---

Source: Grafy Research Lab — MicroLens: Read, Trace, and Rewire the Residual Stream with Two Lenses