StitchRAG enables compact, evidence‑preserving contexts for RAG
Researchers introduce StitchRAG, a framework that dynamically compresses retrieved passages for Retrieval‑Augmented Generation while preserving evidence.
Source: Grafy Research LabRetrieval‑augmented generation (RAG) improves factuality by grounding large language models (LLMs) in external evidence, but long retrieved contexts increase latency and can introduce irrelevant information. A new paper titled StitchRAG: Constructing Compact and Evidence‑Preserving Contexts for RAG with Diffusion‑Based Stitching proposes a systematic way to shrink those contexts while keeping the essential facts. The work, authored by Minsang Kim, Junpyo Lim, Heeje Cho and Seungjun Baek, appears on arXiv on 18 November 2026.
What is StitchRAG and why does it matter?
StitchRAG is a framework that builds a faithful, information‑dense context for RAG pipelines. Instead of feeding an LLM a raw list of retrieved passages, the system dynamically compresses each passage to a size that balances retained information against computational cost. The authors argue that this approach reduces the overhead of processing long contexts and mitigates the risk of noisy or irrelevant content degrading generation quality. By preserving the evidence needed for factual grounding, StitchRAG aims to keep the benefits of retrieval while streamlining inference.
How does StitchRAG compress retrieved passages?
The core of StitchRAG is a diffusion‑based stitching process. After an initial retrieval step, each passage is passed through a diffusion model that learns to generate a shorter representation of the text. The model is trained to retain key entities, relations and supporting statements, effectively “stitching” together the most salient fragments. The output is a compact passage that can be concatenated with other compressed passages to form the final context supplied to the LLM.
What utility model guides the compression ratio?
StitchRAG formulates the compression decision as a utility maximization problem. For each retrieved passage , the system estimates two quantities: (1) the information retained after compression, and (2) the computation cost of feeding the compressed passage to the LLM. The utility function trades off these factors, selecting a size‑reduction ratio that maximizes retained evidence per unit of compute. This adaptive strategy means longer, more complex passages may be compressed more aggressively than shorter, highly relevant ones.
How does the framework preserve evidence?
Preserving evidence is central to the design. During training, the diffusion model receives ground‑truth evidence tags that indicate which sentences or phrases directly support a downstream query. The loss function penalizes the removal of such evidence, encouraging the compressed output to keep all required citations. In practice, the system can produce a traceable mapping from the compact context back to the original passages, allowing downstream verification if needed.
What early evaluation results suggest about its impact?
The authors report that, on standard RAG benchmarks, StitchRAG reduces the average token count of the context by a noticeable margin while maintaining or slightly improving factual accuracy compared with uncompressed baselines. They also note a drop in inference latency proportional to the token reduction, confirming the intended computational savings. No absolute numbers are disclosed in the summary, but the qualitative findings indicate that compacting context does not have to come at the expense of grounding quality.
How can creators and developers experiment with StitchRAG ideas in Grafy?
While Grafy does not yet ship a dedicated StitchRAG module, the concepts map cleanly onto existing tools:
- Grafy Graph (/graph) lets you construct branching prompt pipelines. You can set up a retrieval step, feed the results into a custom compression prompt (e.g., “summarize this passage while keeping all citations”), and then pass the compressed output to a generation node.
- Grafy Code (/code) provides an environment to prototype the diffusion‑based compression model using Python libraries such as PyTorch or Diffusers. You can prototype the utility‑maximization logic and test it on your own datasets.
- Grafy Chat (/chat) offers a quick way to compare LLM responses with and without compressed context. By switching between the original and a shortened prompt, you can observe differences in answer length, latency, and factual grounding.
- For a deeper dive into the paper itself, Grafy Reader (/reader) enables you to upload the arXiv PDF, ask questions, and extract the sections that describe the diffusion architecture or utility formulation.
By chaining these apps, a developer can recreate a lightweight version of StitchRAG, evaluate its effect on a specific RAG use case (e.g., question answering over a corporate knowledge base), and iterate on the compression prompts without leaving the Grafy workspace.
Source: https://arxiv.org/abs/2611.22766
Frequently asked questions
How does StitchRAG differ from simple summarization?
StitchRAG uses a diffusion‑based model trained to keep evidence tags, whereas generic summarization often discards citation details. The utility maximization also adapts the compression ratio per passage, rather than applying a fixed length limit.
Can StitchRAG be used with any LLM?
Yes. The framework outputs plain text passages, so it can feed any downstream LLM that accepts a context window. The main requirement is that the LLM can handle the reduced token count without losing the ability to reference the retained evidence.
What are the main computational benefits of StitchRAG?
By cutting the number of tokens that travel through the LLM, StitchRAG lowers inference latency and reduces the amount of GPU memory needed per request. This can translate into lower operating costs for large‑scale RAG services.


