Grafy News
Try Grafy free

Bundle-Aware Agent Resource Composition Improves AI Tool Selection

A new study proposes Retrieval‑Grounded Bundle‑Aware Agent Generation, jointly picking backbone models and tool‑skill bundles to boost execution utility.

Illustration for the news story: Bundle-Aware Agent Resource Composition Improves AI Tool SelectionSource: Grafy Research Lab

What problem does bundle-aware agent composition address?

As catalogs of large language models, APIs, and reusable skills expand, developers face a combinatorial explosion when deciding which resources an autonomous agent should employ for a new task. The authors note that a resource’s true value is only observable after execution and depends heavily on the backbone model and the companion tools it is paired with. Existing pipelines either bind tools to fixed tokens in the training catalog or score each resource in isolation, missing the synergistic effects of bundles. Consequently, agents either ignore useful complementarities or reuse past configurations that were built for unrelated tasks, leading to sub‑optimal performance.

How does Retrieval‑Grounded Bundle‑Aware Agent Generation work?

The paper introduces Task‑Conditioned Agent Resource Composition (AgentComposition) and a three‑stage framework called Retrieval‑Grounded Bundle‑Aware Agent Generation.

  1. Retrieval Context – Given a task description, a dense retriever pulls candidate backbone models, tools, and skill APIs from the catalog, as well as historic configurations that solved similar tasks. This narrows the search space from millions of possible bundles to a manageable set.
  2. Structured Generator Fine‑Tuning – A language model fine‑tuned on resource‑description data reads each candidate’s documentation and recombines them through constrained beam search. At each step the generator selects the next tool conditioned on the already‑chosen backbone and previously selected tools, ensuring that the emerging bundle respects compatibility constraints. Because the model ingests raw descriptions, a newly added tool can be used immediately without retraining.
  3. Bundle‑Aware Critic – A separate ranking model evaluates the complete configurations. It is trained on two proxy signals: (a) judgments from a larger LLM acting as a teacher, and (b) execution outcomes of historically successful bundles. When real execution feedback becomes available, the critic updates its parameters to better align its scores with actual utility.

The overall pipeline therefore retrieves, composes, and ranks configurations in a single end‑to‑end loop, addressing the four challenges highlighted by the authors: combinatorial explosion, catalog dynamism, bundle interdependence, and delayed reward signals.

How does the new method perform against prior baselines?

The authors evaluate on several benchmarks, most notably AgentSelect‑Exec, a collection of 5,060 executed configurations, and SkillsBench, a suite of tool‑use tasks. Key findings include:

  • The system recovers the exact reference configuration within the top‑10 candidates for 67.73 % of AgentSelect queries – almost twice the success rate of the strongest baseline.
  • Using the proxy‑trained critic raises the predicted execution score of the top‑ranked agent from 7.58 to 8.37 (out of 10). After incorporating real execution feedback, the score climbs further to 8.78, surpassing all baselines.
  • In offline execution tests, agents recommended by the new method achieve higher native verifier scores than retrieval‑only, token‑based, and retrieval‑augmented LLM baselines.
  • Online recommendation experiments show that higher‑ranked suggestions consistently execute better, confirming that the critic’s ranking correlates with real‑world utility.

These results demonstrate that jointly optimizing backbone and bundle selection yields measurable gains over approaches that treat resources independently.

Can the approach adapt to new domains and evolving catalogs?

A central claim of the paper is that the composition model transfers across domains with minimal fine‑tuning. When adapting from the original AgentSelect setting to SkillsBench, the authors needed only four target‑domain queries to adjust the generator, after which performance matched that of a model trained from scratch on the new domain. Moreover, the bundle‑aware critic can be updated in a third stage (Stage 3) using execution feedback from newly generated candidates, allowing it to stay current as the catalog expands.

Because the generator reads raw resource descriptions, adding a new API or skill to the catalog does not require re‑training the entire model – the retriever will surface the new entry, and the generator can immediately incorporate it into future bundles.

What does this mean for developers and researchers using Grafy?

Grafy’s multimodal workspace already supports resource‑rich AI workflows. While the platform does not yet embed a full‑stack bundle‑aware composition engine, creators can experiment with the core ideas using existing apps:

  • Grafy Chat (ChatChat) lets you query multiple models side‑by‑side, making it a convenient sandbox for testing backbone‑selection heuristics on a given task.
  • Grafy Reader (ReaderReader) enables you to load the full paper, search for specific sections, and ask follow‑up questions about the retrieval or critic components without leaving the interface.
  • For teams interested in prototyping the retrieval‑grounded pipeline, Grafy Code provides an AI‑assisted coding environment where you can integrate a dense retriever (e.g., OpenRouter embeddings) with a fine‑tuned generator and a ranking model, all within a single notebook‑style workspace.

By combining these tools, developers can build lightweight versions of the proposed system, experiment with custom catalogs, and evaluate bundle performance on their own tasks. The study’s open‑source implementation (https://github.com/Ancientshi/AgentComposition) can be imported directly into Grafy Code, accelerating the path from research to production.


Source: https://arxiv.org/abs/2611.22030

Frequently asked questions

How does bundle‑aware generation differ from token‑based tool generators?

Token‑based generators assign a fixed token to each tool during training, limiting selection to the original catalog. Bundle‑aware generation reads each tool’s description at inference time and composes resources through constrained beam search, allowing brand‑new tools to be used without retraining.

What role does the bundle‑aware critic play in the pipeline?

The critic ranks complete backbone‑bundle configurations using proxy signals (LLM judgments and historical success) and adapts to real execution feedback. Its scores guide the final reranking step, ensuring that the top‑suggested agent is likely to achieve high utility when run.

Can Grafy users apply this research without building a custom pipeline?

Yes. Users can leverage Grafy Chat to experiment with backbone selection, Grafy Reader to explore the paper’s methodology, and Grafy Code to prototype the retrieval‑generator‑critic loop using the provided open‑source code. This modular approach lets creators test bundle‑aware ideas without waiting for a dedicated Grafy feature.


Source: Grafy Research Lab — What Does an Agent Need? Bundle-Aware Agent Resource Composition