Grafy News
Try Grafy free

Hybrid RAG Framework Boosts Multi‑Hop QA on Long Documents

A new arXiv study introduces a hybrid Retrieval‑Augmented Generation system that combines knowledge graphs with dense vector search to improve multi‑hop.

Grafy News DeskNov 10, 20264 min read
Illustration for the news story: Hybrid RAG Framework Boosts Multi‑Hop QA on Long DocumentsSource: Arxiv IR Recent

What problem does the new hybrid RAG approach aim to solve?

Long, multi‑topic documents pose two persistent challenges for Retrieval‑Augmented Generation (RAG) pipelines. First, fixed‑size chunking often slices sentences in ways that break semantic continuity, a phenomenon the authors call context fragmentation. Second, standard dense‑vector retrieval struggles with multi‑hop queries that require evidence from several separate passages or documents. According to the arXiv paper, these issues lead to “transitive dependency failures” and “implicit linkage failures,” where the system cannot locate or connect the intermediate facts needed for a correct answer.

How does the proposed hybrid architecture work?

The authors describe a four‑stage pipeline (see Figure 1). 1) Semantic Chunking uses a transformer‑based summarizer to split each source document into meaning‑preserving units rather than arbitrary token windows. 2) Each chunk is indexed in a dense vector database for semantic similarity search. 3) In parallel, a dynamic knowledge graph is built from named entities and their extracted relations across all chunks, enabling symbolic reasoning over explicit connections. 4) During inference, a hierarchical selector first attempts a graph‑based path ranking; if no high‑confidence path is found, the system falls back to dense retrieval. The combined evidence is then fed to the LLM for answer generation. Figure 2 illustrates the graph‑reasoning component, showing how entity nodes and edges are traversed to form a logical chain.

Figure 1: The end-to-end architecture of the hybrid RAG framework.
Figure 1. The end-to-end architecture of the hybrid RAG framework. Figure 1 of “Improving Contextual Retrieval for Long Documents using Semantic Chunking and Hybrid Retrieval Strategies” by Sakshi Garg, William B. Andreopoulos (arXiv.org), CC BY 4.0.
Figure 2: Knowledge Graph Reasoning
Figure 2. Knowledge Graph Reasoning Figure 2 of “Improving Contextual Retrieval for Long Documents using Semantic Chunking and Hybrid Retrieval Strategies” by Sakshi Garg, William B. Andreopoulos (arXiv.org), CC BY 4.0.

How does the hybrid system perform against existing baselines?

Evaluation was carried out on the MultiHop‑RAG benchmark, which contains questions that require linking information across multiple documents. The hybrid model achieved higher Exact Match scores than both a pure dense‑retrieval RAG and a graph‑only baseline. The paper reports a noticeable lift in retrieval accuracy for complex queries, especially in domains where terminology is highly specialized (e.g., legal or scientific texts). An ablation study confirmed that both semantic chunking and the knowledge‑graph layer contributed meaningfully; removing either component reduced performance to near‑baseline levels.

Why does this matter for developers and researchers?

For anyone building AI‑powered search, QA, or summarisation tools, the study demonstrates that combining structured symbolic reasoning with the breadth of dense embeddings can mitigate the weaknesses of each approach when used alone. Developers can adopt the hybrid pattern to improve interpretability (graph paths are traceable) and reduce hallucination, while researchers gain a concrete benchmark for testing multi‑hop reasoning strategies. The methodology also aligns with emerging trends in “graph‑enhanced” language models, suggesting a viable path toward more reliable, evidence‑grounded AI systems.

How to experiment with hybrid retrieval in Grafy

Grafy does not yet offer a built‑in hybrid RAG engine, but the workflow can be assembled using existing apps:

  • /reader – Upload the long PDFs you want to index; the Reader can extract text, run semantic chunking via a transformer model, and store the chunks.
  • /graph – Use the Graph app to construct a knowledge graph from the extracted entities (Grafy supports custom prompt pipelines that can output triples for graph ingestion).
  • /chat – Combine the dense‑vector index (via an external vector DB you connect through the chat prompt) with the graph‑based reasoning by prompting the model to first request a graph path and then fall back to vector search if needed.

By chaining these apps, creators can prototype the same multi‑stage pipeline described in the paper without leaving the Grafy workspace. For deeper integration, developers could write a small script in /code to orchestrate the fallback logic and expose it as a custom Bot via /bots for interactive QA.

What are the limitations and next steps?

The authors acknowledge that building and maintaining a high‑quality knowledge graph remains resource‑intensive, especially for domains with noisy entity extraction. Additionally, the fallback hierarchy currently prefers graph traversal, which may overlook semantically relevant passages that lack explicit relations. Future work suggested includes dynamic weighting between graph and vector scores and extending the approach to multimodal documents (e.g., PDFs with figures). These directions align well with Grafy’s multimodal capabilities, hinting at possible extensions that incorporate image embeddings alongside text.

Source: "Improving Contextual Retrieval for Long Documents using Semantic Chunking and Hybrid Retrieval Strategies", arXiv, 2026‑11‑09.

To explore this yourself, open Graph: a branching graph of prompts, outputs and edits.

Source: Arxiv IR Recent — Improving Contextual Retrieval for Long Documents using Semantic Chunking and Hybrid Retrieval Strategies

More news