Google has launched Tool Retrieval in the Gemini API, available in preview as of November 19, 2026. This feature allows AI agents to efficiently manage large tool libraries by loading tool parameter schemas only when needed, significantly reducing input token counts and latency, according to Google.
This update addresses a key challenge for developers building complex AI agents that integrate with dozens or hundreds of external tools. Previously, sending every full parameter schema with each request inflated token counts and increased processing time. The new approach streamlines this process, making agents more scalable and responsive.
How Does Tool Retrieval Work?
Tool Retrieval enables agents to keep their entire tool library on standby, fetching specific parameter schemas only when a task requires them. The model retains tool names and descriptions in context, knowing what's available, but defers loading the detailed schema. This process occurs within a single API request, from the model asking for a tool to the API loading its schema and the model executing the function call.Developers activate Tool Retrieval by adding a tool_search tool to their request and setting defer_loading: true on specific tools. This keeps detailed schemas out of the initial prompt context. When the model requests a deferred tool by name, the Gemini API dynamically loads its full schema, allowing the model to proceed with the function call immediately. This method works with Gemini 3.6 and later models, including all Gemini 4 models.
Benefits for Agent Development
The primary benefit of Tool Retrieval is a substantial reduction in input token usage and improved latency for agents utilizing extensive tool sets. By only loading necessary schemas, developers can build more complex and accurate agents without incurring the prohibitive costs and performance bottlenecks of sending full tool definitions in every prompt. It also enhances tool-selection accuracy as the candidate list remains manageable.This feature also supports both server-side and client-side retrieval, offering flexibility. For those maintaining their own tool registries or permission filters, declaring tool_search with "execution": "client" allows the application to handle the tool lookup. This can expose new tools mid-conversation when users connect integrations, making agent interactions more dynamic. Developers can also optimize costs, as tool retrieval has no separate charge beyond standard input and output token rates, which for the Hydrogen model are $1.50 per million input tokens and $7.50 per million output tokens.
| Feature | Traditional Tool Handling | Tool Retrieval (New) |
|---|---|---|
| Schema Loading | All tool schemas sent with every request | Schemas loaded on demand when needed |
| Input Token Count | Higher, especially with many tools | Lower, only active schemas count |
| Latency | Increased with larger tool libraries | Reduced due to smaller context window |
| Tool Selection Accuracy | Can decrease with large candidate lists | Improved by focused context |
| Cost Efficiency | Potentially higher token costs | Optimized for large tool libraries |




