Grafy News
Try Grafy free

Google launches Tool Retrieval in Gemini API to streamline large tool libraries

Google introduced Tool Retrieval in the Gemini API, letting agents load tool schemas on demand to cut token usage and latency.

Illustration for the news story: Google launches Tool Retrieval in Gemini API to streamline large tool librariesSource: Google AI Blog

Google announced a preview of Tool Retrieval for the Gemini Interactions API, a feature that lets AI agents keep a full catalog of tools on standby and pull in the detailed parameter schemas only when a specific tool is needed. By avoiding the transmission of every tool definition on each request, developers can reduce input token counts, lower latency, and maintain higher tool‑selection accuracy as their agents scale to dozens or hundreds of functions.


Introducing Tool Retrieval in the Gemini API
Embedded audio from the source story.

What is Tool Retrieval and why it matters?

Traditional agent workflows require the full list of tool definitions—including parameter schemas—to be sent with each prompt. When a system integrates a large number of APIs, this overhead quickly inflates token usage, adds latency, and can confuse the model as the candidate list grows. Tool Retrieval changes that pattern: the model sees only the names and short descriptions of all available tools, while the heavy schema payload is deferred until the model explicitly requests a particular function. According to the Google AI Blog, this design "reduces input token counts and latency by loading parameter schemas only when needed," which is especially valuable for complex, multi‑step tasks.


How does the feature work in the Gemini Interactions API?

To enable Tool Retrieval, developers add a special tool_search entry to their tool list and set defer_loading: true on any function they wish to keep hidden until required. The model still knows the existence of each deferred tool, so it can decide which one to call based on the conversation context. When the model decides on a tool, it issues a tool_search_call; the API then returns the full schema in a subsequent tool_search_result, after which the model emits the usual function_call. All three steps appear in the interaction.steps array, making debugging straightforward.

A minimal Python snippet from the announcement illustrates the flow:

from google import genai
client = genai.Client()

tools = [
    {"type": "tool_search"},
    {
        "type": "function",
        "name": "issue_refund",
        "description": "Refund an order to the original payment method.",
        "defer_loading": True,
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "amount": {"type": "number"}
            },
            "required": ["order_id", "amount"]
        }
    }
    # ... 50+ other deferred functions
]

interaction = client.interactions.create(
    model="gemini-4-hydrogen",
    input="Order A-1042 arrived damaged. Please refund 40 dollars.",
    tools=tools,
)

The response contains a tool_search_call for issue_refund, the API loads its schema, and the model proceeds with a function_call. No separate index or hosting infrastructure is required; Gemini handles the lookup on the server side.


Benefits for developers handling large tool libraries

  1. Token efficiency – By sending only the names and descriptions of hundreds of tools, the prompt size shrinks dramatically. Tokens are only spent on schemas that the model actually uses.
  2. Lower latency – Deferred schemas are fetched on demand, meaning the initial request is faster. Subsequent schema loads happen within the same API call, avoiding extra round trips.
  3. Improved selection accuracy – With fewer schemas crowding the prompt, the model can focus on the most relevant tool descriptions, reducing the chance of picking the wrong function.
  4. Seamless integration with prefix caching – Because the deferred schemas sit outside the static prompt prefix, cached prefixes remain reusable across turns, further speeding up multi‑turn conversations.
  5. Support for remote MCP servers – Setting defer_loading: true on an mcp_server tool automatically defers every function the server exposes, allowing developers to connect massive external tool collections without enumerating each one.

Pricing and model support

Tool Retrieval does not introduce a new pricing tier. Input and output tokens are billed at the standard rates for the chosen Gemini model. Enabling tool_search adds a small, fixed token overhead for the search request itself, and each loaded schema is billed like any other input token. In practice, the net savings appear when the number of never‑used schemas exceeds the few that are actually loaded—a typical scenario for long‑tail tool sets.

The feature is compatible with Gemini 3.6 and later, including all Gemini 4 models, when accessed through the Interactions API.


Getting started and practical steps

  1. Read the official documentation – Google provides a full setup guide with code examples.
  2. Choose the right model – Compare Gemini model capabilities to match your workload (the documentation includes a model comparison page).
  3. Add tool_search and defer flags – Update your tool definitions as shown in the code snippet above.
  4. Test with a simple request – Send a prompt that would trigger a deferred tool and inspect the interaction.steps array to verify the three‑step flow.
  5. Monitor token usage – Compare runs with and without deferred loading to quantify savings for your specific library.

Developers can also join the community discussion on Discord, where the Gemini team collects feedback and shares best practices.


Grafy perspective – how to experiment with Tool Retrieval

While Grafy does not host the Gemini API directly, the Grafy Code studio (/code) lets you prototype the same request‑flow using any HTTP client. You can write a small script that calls the Gemini Interactions endpoint, toggles defer_loading, and visualizes the returned steps. This hands‑on approach helps you understand how deferred schemas affect prompt length and latency.

If you prefer a no‑code exploration, the Grafy Chat interface (/chat) can be pointed at the Gemini endpoint (if you have API credentials) and used to converse with a model that has Tool Retrieval enabled. By observing the model’s tool suggestions and the subsequent schema loads, you get a live demonstration of the feature without writing code.

For deeper analysis of token economics, the Grafy Reader (/reader) can ingest the Google AI Blog article (PDF or HTML) and let you ask questions about specific sections, such as the exact token‑saving calculations or the interaction schema format.


Audio version

An audio narration of the announcement is available at https://storage.googleapis.com/gweb-uniblog-publish-prod/media/tts_audio_87052_umbriel_2026_11_19_18_31_23.mp3.


Frequently asked questions

How does Tool Retrieval differ from loading all tool schemas up front?

Tool Retrieval keeps only the tool names and short descriptions in the initial prompt, loading the full parameter schema only when the model explicitly requests a specific tool. This reduces token consumption and latency compared to sending every schema with each request.

Do I need to change my existing code to use Tool Retrieval?

You only need to add a tool_search entry to your tool list and set defer_loading: true on the functions you want to defer. The rest of your workflow—sending prompts and handling function calls—remains unchanged.

Is there any extra cost for using the tool_search feature?

No. Tool Retrieval is billed at the standard token rates for the Gemini model you use. The only extra cost is a small, fixed number of tokens for the search request itself; any loaded schemas are charged like regular input tokens.


Source: Google AI Blog


Source: Google AI Blog — Introducing Tool Retrieval in the Gemini API