Grafy News
Try Grafy free

RAG-FGO Introduces Dynamic Optimization for Query‑Rewriting Agents in Generative Retrieval

A new arXiv paper presents RAG-FGO, a framework that treats query rewriting as a mixed discrete‑continuous black‑box optimization.

Illustration for the news story: RAG-FGO Introduces Dynamic Optimization for Query‑Rewriting Agents in Generative RetrievalSource: Grafy Research Lab

RAG‑FGO, a framework for dynamic optimization of query‑rewriting agents, was unveiled in a recent arXiv pre‑print (2026‑11‑16). The work matters because it tackles three persistent obstacles in retrieval‑augmented generation (RAG): noisy feedback, the joint tuning of discrete prompts and continuous decoding knobs, and the brittleness of static rewriting under complex semantics.

What is RAG‑FGO and why was it introduced?

According to the authors, RAG‑FGO re‑imagines generative retrieval as an optimization problem rather than a single‑shot rewrite step. In traditional RAG pipelines, a user query is fed directly to a retriever, which often suffers when the surface form of the query mismatches the corpus representation. Generative retrieval attempts to fix this by letting a large language model (LLM) rewrite the query, but the quality of the rewrite depends on both the prompt template and decoding parameters such as temperature and pp (top‑pp). The paper argues that static or heuristic rewrites cannot reliably navigate the noisy, non‑differentiable landscape of retrieval feedback, prompting the need for a more systematic search strategy.

How does RAG‑FGO treat query rewriting as optimization?

The authors formulate query rewriting as a black‑box optimization over a mixed discrete–continuous space. Discrete elements are prompt templates that dictate the structural form of the rewritten query (e.g., "Explain in simple terms:" versus "Provide a concise answer:"). Continuous elements are decoding‑control parameters like temperature, top‑pp, and repetition penalty. Because the retrieval step is non‑differentiable and the feedback signal (e.g., relevance scores) is stochastic, gradient‑based methods are unsuitable. Instead, RAG‑FGO employs a heuristic search inspired by the Fungal Growth Optimizer (FGO), alternating between global semantic jumps and local fine‑grained perturbations to explore the strategy space.

What are the main components of the framework?

  1. Agent Initialization – Each agent bundles a specific prompt template with a set of decoding parameters.
  2. Scoring and Selection – After generating a rewritten query, the system retrieves documents and evaluates semantic similarity to the target information need. The score guides selection.
  3. Exploration vs. Exploitation – Global operations clone agents and apply large‑scale template swaps, enabling jumps to distant regions of the search space. Local operations tweak parameters slightly for refinement.
  4. Elite‑Preserving Memory Pool – High‑scoring agents are stored in a memory pool that survives across iterations, reducing drift caused by noisy feedback.
  5. Termination – The process stops when a budget of retrieval calls is exhausted or when performance plateaus.

How does RAG‑FGO perform compared to baselines?

The paper reports experiments on several knowledge‑intensive benchmarks, including open‑domain question answering and multi‑hop reasoning datasets. Under identical model size and retrieval budget constraints, RAG‑FGO consistently outperformed standard retrieval‑based baselines, static query‑rewriting methods, and recent prompt‑optimization techniques. Ablation studies showed that removing the elite memory pool or disabling global jumps reduced both retrieval quality and robustness, confirming the importance of each component. Cross‑task generalization tests also indicated that agents discovered on one dataset transferred reasonably well to unseen tasks, suggesting the framework’s adaptability.

How can Grafy users experiment with similar ideas?

Grafy’s Graph app lets creators build branching workflows of prompts, model outputs, and parameter tweaks. By treating each branch as a candidate query‑rewriting agent, developers can manually emulate the exploration‑exploitation cycle described in RAG‑FGO. The memory‑pool concept can be mimicked by saving high‑scoring branches and re‑using them as seeds for new experiments. For a deeper dive into the paper itself, users can upload the PDF to Reader and ask the AI to summarize sections, extract the optimization algorithm, or compare results with other RAG techniques. While Grafy does not yet provide an out‑of‑the‑box FGO optimizer, the combination of Graph for systematic prompt tracking and Reader for on‑the‑fly analysis offers a practical sandbox for prototyping RAG‑FGO‑style workflows.


Source: arXiv pre‑print “RAG‑FGO: Dynamic Optimization of Query‑Rewriting Agents for Generative RAG” (https://arxiv.org/abs/2611.17788)

Frequently asked questions

Does RAG‑FGO optimize both prompt templates and decoding parameters together?

Yes. The framework treats a prompt template (discrete) and decoding knobs such as temperature and top‑pp (continuous) as a single agent that is jointly optimized through black‑box search.

What role does the elite‑preserving memory pool play?

It stores the highest‑scoring agents across iterations, allowing the optimizer to retain good strategies despite noisy retrieval feedback and to reuse them for further exploration.

Can Grafy’s Graph app replace the FGO algorithm in RAG‑FGO?

Graph does not implement the FGO algorithm automatically, but it can be used to manually construct and visualize a branching search over prompts and parameters, approximating the exploration‑exploitation dynamics described in the paper.


Source: Grafy Research Lab — RAG-FGO: Dynamic Optimization of Query-Rewriting Agents for Generative RAG