You can use Google models through AI Gateway (recommended)
or connect directly to the Google APIs with @ai-sdk/google.
Using AI Gateway
The AI SDK uses AI Gateway by default for model strings.
AI Gateway provides access to Google and other providers through a single API key, with usage and cost tracking.
Install the ai package:
$ pnpm add ai
$ npm install ai
$ yarn add ai
$ bun add ai
Set an AI Gateway API key in your environment:
Vercel deployments can also use OIDC authentication. See the AI Gateway provider documentation for configuration and the model catalog for available Google models.
Google model options can be passed through providerOptions.google when using AI Gateway.
Support depends on the model and the provider serving the request; see Gateway provider options.
Using @ai-sdk/google
For direct access to the Google APIs, install ai and @ai-sdk/google:
$ pnpm add ai @ai-sdk/google
$ npm install ai @ai-sdk/google
$ yarn add ai @ai-sdk/google
$ bun add ai @ai-sdk/google
Set your Google API key in the GOOGLE_GENERATIVE_AI_API_KEY environment variable:
AI Gateway does not expose every API in @ai-sdk/google.
Use the direct provider for Google-specific APIs such as google.interactions(...),
or when you need to customize the Google connection with createGoogle.
The following sections document the direct @ai-sdk/google provider,
including its model factories, tools, and settings. Gateway model and feature
availability can differ from the direct provider.
Provider Instance
You can import the default provider instance google from @ai-sdk/google:
If you need a customized setup, you can import createGoogle from @ai-sdk/google and create a provider instance with your settings:
You can use the following optional settings to customize the Google provider instance:
-
batchResultDownloads object
Settings for downloading file-based JSON Lines batch results.
-
maxLineBytes number
Maximum UTF-8 bytes per row, excluding the LF delimiter. Defaults to 64 MiB (67,108,864 bytes). Must be a positive safe integer. Oversized rows throw a
DownloadErrorand cancel the download.
-
-
baseURL string
Use a different URL prefix for API calls, e.g. to use proxy servers. The default prefix is
https://generativelanguage.googleapis.com/v1beta. -
apiKey string | PromiseLike<string> | (() => string | PromiseLike<string>)
Pass a function to resolve credentials before each request, including when keys rotate. The function can return a string or a promise. Credential caching and refresh are managed by the function.
API key that is being sent using the
x-goog-api-keyheader. It defaults to theGOOGLE_GENERATIVE_AI_API_KEYenvironment variable. -
headers Record<string,string>
Custom headers to include in the requests.
-
fetch (input: RequestInfo, init?: RequestInit) => Promise<Response>
Custom fetch implementation. Defaults to the global
fetchfunction. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing. -
generateId () => string
Optional function to generate unique IDs for each request. Defaults to the SDK's built-in ID generator.
-
name string
Custom provider name. Defaults to
'google.generative-ai'.
Language Models
You can create models that call the Google Generative AI API using the provider instance.
The first argument is the model id, e.g. gemini-4-argon.
The models support tool calls and some have multi-modal capabilities.
You can use Google language models to generate text with the generateText function:
Google language models can also be used in the streamText function
and support structured data generation with Output
(see AI SDK Core).
To remain forward-compatible, the provider treats unrecognized gemini-*
model IDs and -latest aliases like the newest supported Gemini generation.
Currently, this means Gemini 3 request behavior for provider-defined and mixed
tools, thinkingLevel reasoning, multimodal function responses, and thought
signatures. Known legacy Gemini model IDs keep their generation-specific
request behavior. This SDK default controls request serialization; the
selected model must still support each feature in the Google API.
Google also supports some model specific settings that are not part of the standard call settings. You can pass them as an options argument:
The following optional provider options are available for Google models:
-
cachedContent string
Optional. The name of the cached content used as context to serve the prediction. Format: cachedContents/{cachedContent}
-
structuredOutputs boolean
Optional. Enable structured output. Default is true.
You can disable structured outputs when a schema is not supported or exceeds Google's schema complexity limits. The model will still be instructed to return JSON, but the schema will not be sent to Google.
See Troubleshooting: Schema Limitations for more details.
-
safetySettings Array<{ category: string; threshold: string }>
Optional. Safety settings for the model.
-
category string
The category of the safety setting. Can be one of the following:
HARM_CATEGORY_UNSPECIFIEDHARM_CATEGORY_HATE_SPEECHHARM_CATEGORY_DANGEROUS_CONTENTHARM_CATEGORY_HARASSMENTHARM_CATEGORY_SEXUALLY_EXPLICITHARM_CATEGORY_CIVIC_INTEGRITY
-
threshold string
The threshold of the safety setting. Can be one of the following:
HARM_BLOCK_THRESHOLD_UNSPECIFIEDBLOCK_LOW_AND_ABOVEBLOCK_MEDIUM_AND_ABOVEBLOCK_ONLY_HIGHBLOCK_NONEOFF
-
-
responseModalities string[] The modalities to use for the response. Valid values are
TEXT,IMAGE, andAUDIO, but support is model-dependent. When not defined or empty, the model defaults to returning only text. -
thinkingConfig { thinkingLevel?: 'minimal' | 'low' | 'medium' | 'high' | 'xhigh'; thinkingBudget?: number; includeThoughts?: boolean }
Optional. Configuration for the model's thinking process. Only supported by specific Google models.
-
thinkingLevel 'minimal' | 'low' | 'medium' | 'high' | 'xhigh'
Optional. Controls the thinking depth for Gemini 3 and later models. Gemini 3.1 Pro supports 'low', 'medium', and 'high', Gemini 3 Pro supports 'low' and 'high', while Gemini 3 Flash supports all four levels: 'minimal', 'low', 'medium', and 'high'. Gemini 4 Hydrogen supports 'low', 'medium', 'high', and 'xhigh'. The 'xhigh' level is only supported by Hydrogen.
-
thinkingBudget number
Optional. Gives the model guidance on the number of thinking tokens it can use when generating a response. Setting it to 0 disables thinking, if the model supports it. For more information about the possible value ranges for each model see Google thinking documentation.
This option is for Gemini 2.5 models. Gemini 3 and later models should use
thinkingLevelinstead. -
includeThoughts boolean
Optional. If set to true, thought summaries are returned, which are synthesized versions of the model's raw thoughts and offer insights into the model's internal reasoning process.
-
-
imageConfig { aspectRatio?: string, imageSize?: string }
Optional. Configuration for the models image generation. Only supported by specific Google models.
-
aspectRatio string
Model defaults to generate 1:1 squares, or to matching the output image size to that of your input image. Can be one of the following:
- 1:1
- 2:3
- 3:2
- 3:4
- 4:3
- 4:5
- 5:4
- 9:16
- 16:9
- 21:9
-
imageSize string
Controls the output image resolution. Defaults to 1K. Can be one of the following:
- 1K
- 2K
- 4K
-
-
audioTimestamp boolean
Optional. Enables timestamp understanding for audio-only files. See Google Cloud audio understanding documentation.
-
mediaResolution string
Optional. If specified, the media resolution specified will be used. Can be one of the following:
MEDIA_RESOLUTION_UNSPECIFIEDMEDIA_RESOLUTION_LOWMEDIA_RESOLUTION_MEDIUMMEDIA_RESOLUTION_HIGH
-
labels Record<string, string>
Optional. Defines labels used in billing reports. Available on Vertex AI only. See Google Cloud labels documentation.
-
serviceTier 'standard' | 'flex' | 'priority'
Optional. The service tier to use for the request. Set to
'flex'for 50% cheaper processing at the cost of increased latency. Set to'priority'for ultra-low latency at a 75-100% price premium over'standard'.Because Priority can be gracefully downgraded to Standard under load, the tier the request actually ran on is surfaced on
result.providerMetadata.google.serviceTier. See Priority inference and Flex inference. -
continuationToken string
Optional. Resumes a response that stopped because it used up its
maxOutputTokensbudget (Gemini 4 long decode). Send it with the original request and a highermaxOutputTokens. See Long decode. -
threshold string
Optional. Standalone threshold setting that can be used independently of
safetySettings. Uses the same values as thesafetySettingsthreshold.
Thinking
The Gemini 2.5 and Gemini 3-and-later series models use an internal "thinking process" that significantly improves their reasoning and multi-step planning abilities, making them highly effective for complex tasks such as coding, advanced mathematics, and data analysis. For more information see Google thinking documentation.
Gemini 3 and Later Models
For Gemini 3 and later models, use the thinkingLevel parameter to control the depth of reasoning:
For gemini-4-hydrogen (including gemini-4-hydrogen-eap), use
reasoning: 'xhigh' or thinkingConfig: { thinkingLevel: 'xhigh' } for maximum
reasoning depth. Hydrogen defaults to high when the thinking level is unset.
Following the November 11, 2026 update, xhigh matches the previous high,
high matches the previous medium, and medium sits between low and the
new high. The low level is unchanged. Re-run prompts tuned before that
update to check their reasoning quality.
Gemini 2.5 Models
For Gemini 2.5 models, use the thinkingBudget parameter to control the number of thinking tokens:
Long decode
Gemini 4 models such as gemini-4-argon can keep thinking and writing past a
single request's limits, up to the shared 1M token context window (input,
thinking, and output tokens together). Use it for hard math, science, proofs,
and long coding or agentic tasks.
- A single request generates at most 32,768 tokens without streaming and 65,536
tokens with streaming (
streamText). Thinking tokens count towards this cap. maxOutputTokensis a cumulative budget across requests, not a target length. When a request reaches the per-request cap while budget remains, the provider automatically sends the same request again with a continuation token and combines the results into one response. SetmaxOutputTokensabove the per-request cap (up to 1M minus the input tokens) to enable this. If you do not set it, the call ends after one request.- A lower
maxOutputTokensdoes not make the model think less, it ends the response early (possibly before any answer) and you are still charged for the tokens. UsethinkingConfig.thinkingLevelto control how much the model thinks.thinkingBudgetis not supported by these models. - Complex requests can run for several minutes. Make sure your HTTP client allows read timeouts of at least 15 to 20 minutes.
Grounding across requests. When a response is continued automatically,
providerMetadata.google.groundingMetadataPerRequest contains the grounding
metadata from each request, in order. An entry is null if that request did
not return grounding metadata. This preserves earlier Search results even if
the final request returns none, and keeps citation indices scoped to their
original request. The existing groundingMetadata field keeps its current
behavior; use the per-request array when inspecting a continued response.
Usage across requests. Each continued request re-sends the prompt plus all
output so far as input, so its prompt token count includes everything
generated before it. usage is the sum of all requests, which matches what
Google bills (re-sent tokens are billed at a discount through implicit
caching). Do not use the summed input tokens for context window checks or
compaction: they can be several times larger than the real context. When a
response spanned more than one request,
providerMetadata.google.initialPromptTokenCount holds the prompt token count
of the first request (the google-genai SDKs expose the same value as
initial_prompt_token_count):
Resuming after the budget is used up. When the cumulative generated tokens
reach maxOutputTokens, the finish reason is length and the provider returns
a continuation token on result.providerMetadata.google.continuationToken. To
continue, send the original request unchanged (same prompt, tools, and
settings; do not append the partial output as a message) together with the
token as providerOptions.google.continuationToken and a higher
maxOutputTokens. Resending with the same maxOutputTokens fails because the
budget is used up. Each response contains only the newly generated content.
Things to keep in mind:
- Dropped streams. Streamed parts are forwarded as they arrive and are not buffered between the server's checkpoints, so a stream that drops mid-request is not resumed automatically. Retry the request from the start.
- Provider metadata is per call. If you store the conversation history, combine the content of all calls into one model message only after the final call finishes. Keep thought signatures unchanged, including the ones on empty trailing text parts.
- Zero Data Retention. Continuation works with
store: falsebecause the state travels encrypted inside the token.
Long decode with the Interactions API
With the Interactions API, the AI SDK currently requires
background: true for long decode. The server runs every request itself, so no
client-side continuation is needed, and usage covers all requests. Set an
explicit maxOutputTokens above 65,536 and keep store enabled (the default).
The provider polls the interaction (or streams it, reconnecting when the
connection drops) until it finishes.
Foreground client-side continuation is not yet supported by the AI SDK.
If you do not set maxOutputTokens, non-streaming calls (generateText)
default to 32,768 tokens and streaming calls (streamText) to 65,536 tokens.
An interaction that reaches its limit finishes with length. Long decode is
not available with the Vertex AI Interactions API.
The token budget includes both thinking and output tokens. To retain the previous
64K budget for a non-streaming call, set maxOutputTokens: 65_536 explicitly.
File Inputs
The Google provider supports file inputs, e.g. PDF files.
You can also use YouTube URLs directly. Pass videoMetadata through
providerOptions.google on a video file part to clip a segment or control the
frame rate. This also works with inline video data and uploaded file references.
The metadata supports these optional fields:
startOffset(string): Start time as a duration in seconds, e.g.'60s'or'3.5s'.endOffset(string): End time in the same duration format.fps(number): Frames per second in the range(0, 24]. The default is1.
The AI SDK will automatically download URLs if you pass them as data, except
for https://generativelanguage.googleapis.com/v1beta/files/ and YouTube
URLs. You can use the Google Generative AI Files API to upload larger files to
that location. YouTube URLs (public or unlisted videos) are supported directly
- you can specify one YouTube video URL per request.
See File Parts for details on how to use files in prompts.
Cached Content
Google supports both explicit and implicit caching to help reduce costs on repetitive content.
Implicit Caching
Gemini models automatically provide cache cost savings without needing to create an explicit cache. When you send requests that share common prefixes with previous requests, you can receive discounted pricing on cached input tokens.
To maximize cache hits with implicit caching:
- Keep content at the beginning of requests consistent
- Add variable content (like user questions) at the end of prompts
- Ensure requests meet minimum token requirements:
- Gemini 3 series: 4096 tokens minimum
- Gemini 2.5 series: 2048 tokens minimum
Usage metadata was added to providerMetadata in @ai-sdk/google@1.2.23. If
you are using an older version, usage metadata is available in the raw HTTP
response body returned as part of the return value from generateText.
Explicit Caching
For guaranteed cost savings, you can still use explicit caching with Gemini 2.5 and 3 series models. Gemini 4 Argon supports implicit caching only, so this example uses gemini-3.8-flash. See the models page to check if caching is supported for the used model:
Code Execution
With Code Execution, certain models can generate and execute Python code to perform calculations, solve problems, or provide more accurate information.
You can enable code execution by adding the code_execution tool to your request.
The response will contain the tool calls and results from the code execution.
Google Search
With Google Search grounding, the model has access to the latest information using Google Search.
The googleSearch tool accepts the following optional configuration options:
-
searchTypes object
Enables specific search types. Both can be combined.
webSearch: Enable web search grounding (pass{}to enable). This is the default.imageSearch: Enable image search grounding (pass{}to enable).
-
timeRangeFilter object
Restricts search results to a specific time range. Both
startTimeandendTimeare required.startTime: Start time in ISO 8601 format (e.g.'2025-01-01T00:00:00Z').endTime: End time in ISO 8601 format (e.g.'2025-12-31T23:59:59Z').
When Google Search grounding is enabled, the model will include sources in the response.
Additionally, the grounding metadata includes detailed information about how search results were used to ground the model's response. Here are the available fields:
-
webSearchQueries(string[] | null)- Array of search queries used to retrieve information
- Example:
["What's the weather in Chicago this weekend?"]
-
searchEntryPoint({ renderedContent: string } | null)- Contains the main search result content used as an entry point
- The
renderedContentfield contains the formatted content
-
groundingSupports(Array of support objects | null)- Contains details about how specific response parts are supported by search results
- Each support object includes:
segment: Information about the grounded text segmenttext: The actual text segmentstartIndex: Starting position in the responseendIndex: Ending position in the response
groundingChunkIndices: References to supporting search result chunksconfidenceScores: Confidence scores (0-1) for each supporting chunk
Example response:
Enterprise Web Search
With Enterprise Web Search, the model has access to a compliance-focused web index designed for highly-regulated industries such as finance, healthcare, and public sector.
Enterprise Web Search is only available on Vertex AI. You must use the Google
Vertex provider (@ai-sdk/google-vertex) instead of the standard Google
provider (@ai-sdk/google) to use this feature. Requires Gemini 2.0 or newer
models.
Enterprise Web Search provides the following benefits:
- Does not log customer data
- Supports VPC service controls
- Compliance-focused web index for regulated industries
File Search
The File Search tool lets Gemini retrieve context from your own documents that you have indexed in File Search stores. The provider enables this tool for Gemini 2.5 and Gemini 3-and-later model IDs, including unrecognized future Gemini IDs.
File Search responses include citations via the normal sources field and expose raw grounding metadata in providerMetadata.google.groundingMetadata.
Custom metadata attached to the indexed documents is available in
groundingMetadata.groundingChunks[].retrievedContext.customMetadata.
URL Context
Google provides a provider-defined URL context tool.
The URL context tool allows you to provide specific URLs that you want the model to analyze directly in from the prompt.
The URL context metadata includes detailed information about how the model used the URL context to generate the response. Here are the available fields:
urlMetadata({ retrievedUrl: string; urlRetrievalStatus: string; }[] | null)- Array of URL context metadata
- Each object includes:
retrievedUrl: The URL of the contexturlRetrievalStatus: The status of the URL retrieval
Example response:
With the URL context tool, you will also get the groundingMetadata.
The URL context tool is only supported for Gemini 2.0 Flash models and above. Check the supported models for URL context tool.
Combine URL Context with Search Grounding
You can combine the URL context tool with search grounding to provide the model with the latest information from the web.
Google Maps Grounding
With Google Maps grounding, the model has access to Google Maps data for location-aware responses. This enables providing local data and geospatial context, such as finding nearby restaurants.
The optional retrievalConfig.latLng provider option provides location context for queries about nearby places. This configuration applies to any grounding tools that support location context, including Google Maps and Google Search.
When Google Maps grounding is enabled, the model's response will include sources pointing to Google Maps URLs. The grounding metadata includes maps chunks with place information:
RAG Engine Grounding
With RAG Engine Grounding, the model has access to your custom knowledge base using the Vertex RAG Engine. This enables the model to provide answers based on your specific data sources and documents.
RAG Engine Grounding is only supported with Vertex Gemini models. You must use
the Google Vertex provider (@ai-sdk/google-vertex) instead of the standard
Google provider (@ai-sdk/google) to use this feature.
When RAG Engine Grounding is enabled, the model will include sources from your RAG corpus in the response.
Additionally, the grounding metadata includes detailed information about how RAG results were used to ground the model's response. Here are the available fields:
-
groundingChunks(Array of chunk objects | null)- Contains the retrieved context chunks from your RAG corpus
- Each chunk includes:
retrievedContext: Information about the retrieved contexturi: The URI or identifier of the source documenttitle: The title of the source document (optional)text: The actual text content of the chunk
-
groundingSupports(Array of support objects | null)- Contains details about how specific response parts are supported by RAG results
- Each support object includes:
segment: Information about the grounded text segmenttext: The actual text segmentstartIndex: Starting position in the responseendIndex: Ending position in the response
groundingChunkIndices: References to supporting RAG result chunksconfidenceScores: Confidence scores (0-1) for each supporting chunk
Example response:
Configuration Options
The vertexRagStore tool accepts the following configuration options:
-
ragCorpus(string, required)- The RagCorpus resource name in the format:
projects/{project}/locations/{location}/ragCorpora/{rag_corpus} - This identifies your specific RAG corpus to search against
- The RagCorpus resource name in the format:
-
topK(number, optional)- The number of top contexts to retrieve from your RAG corpus
- Defaults to the corpus configuration if not specified
Image Outputs
Gemini models with image generation capabilities (e.g. gemini-3.1-flash-image-preview) support generating images as part of a multimodal response. Images are exposed as files in the response.
If you primarily want to generate images without text output, you can also use
Gemini image models with the generateImage() function. See Gemini Image
Models for details.
Safety Ratings
The safety ratings provide insight into the safety of the model's response. See Google AI documentation on safety settings.
Example response excerpt:
Troubleshooting
Schema Limitations
The Google Generative AI API supports a subset of JSON Schema. Google notes that very large or deeply nested schemas may be rejected. When a schema exceeds Google's complexity limit, the API may return the following generic error:
Request contains an invalid argument.
You can work around this by simplifying the schema sent to Google. For example,
remove unnecessary constraints or reduce nesting while preserving the fields
and types the model needs to generate. If you need stricter client-side
validation than the schema sent to Google, use jsonSchema
with a custom validation function or validate the generated output separately.
By default, structured outputs are enabled (and for tool calling they are required). You can also disable provider-enforced structured outputs for object generation:
When structuredOutputs is false, Google still receives the JSON response MIME type
but not the schema. Describe the required schema in your prompt. AI SDK
will still validate the response against the schema in Output.object.
Model Capabilities
| Model | Image Input | Object Generation | Tool Usage | Tool Streaming | Google Search | URL Context |
|---|---|---|---|---|---|---|
gemini-nano-banana-2.1 | ||||||
gemini-4-argon | ||||||
gemini-4-carbon | ||||||
gemini-4-hydrogen | ||||||
gemini-3.8-flash | ||||||
gemini-3.7-flash | ||||||
gemini-3.6-flash | ||||||
gemini-3.5-flash | ||||||
gemini-3.5-flash-lite | ||||||
gemini-3.1-pro-preview | ||||||
gemini-3.1-flash-image-preview | ||||||
gemini-3.1-flash-lite-preview | ||||||
gemini-3-pro-preview | ||||||
gemini-3-pro-image-preview | ||||||
gemini-3-flash-preview | ||||||
gemini-2.5-pro | ||||||
gemini-2.5-flash | ||||||
gemini-2.5-flash-lite | ||||||
gemini-2.5-flash-lite-preview-06-17 | ||||||
gemini-2.0-flash |
The table above lists popular models. Please see the Google Generative AI docs for a full list of available models. The table above lists popular models. You can also pass any available provider model ID as a string if needed.
Batch
Batch support is experimental and the API may change in patch releases.
The Google provider supports asynchronous text generation through the Gemini Batch API. Pass the Google provider to the AI SDK's Batch API for the complete workflow, including polling, persistence, and result handling.
Each request specifies its type and model. Google requires every text
request in a batch to use the same model and throws before submission when the
models differ.
Webhooks
You can pass a webhookUrl to receive a notification when the batch reaches a terminal state:
Google sends a thin event to the webhook URL. After receiving and verifying the event, use the persisted batch reference with getBatchStatus or getBatchResults. Your application is responsible for handling the webhook and verifying Google's dynamic webhook signature.
Large Batches
When the serialized batch creation body is under 20 MB, the provider sends the requests inline. At 20 MB or more, it uploads a JSONL input file through the Gemini Files API.
Realtime Models
For a trusted server-owned connection, getServerWebSocketConfig() uses the unconstrained Gemini Live endpoint with resolved API-key headers. Use createServerEventParser() once per socket. Completion-frame usage is attached to response-done; independently delivered usage uses custom events with rawType: 'usageMetadata' and typed usage, without inferring a response association. Gross totals never substitute for missing modality counts.
You can create models that call the Gemini Live API
using the .experimental_realtime() factory method.
Realtime sessions run in the browser and require a short-lived token created on
your server with google.experimental_realtime.getToken():
Google realtime models may require provider-specific audio formats, depending on the model and modality. See Realtime for the complete setup and tool calling pattern.
Gemini lifecycle signals that do not have provider-neutral equivalents are
emitted as custom events. Inspect rawType for goAway,
sessionResumptionUpdate, and generationComplete and use the event's raw
payload for provider-specific details.
With input audio transcription enabled, parseServerEvent emits
input-transcription-completed events with a stable itemId for each user
utterance. Each transcript contains the accumulated text so far. If you consume
these events directly, replace the text for that itemId with transcript on
each event. The realtime session handles this automatically.
Google's finished transcription marker ends an utterance even when the marker
contains no text. For payloads without this marker, response completion and
interruption provide fallback boundaries. These fallback boundaries cannot
unambiguously associate all delayed transcription fragments. Input item IDs use
their own sequence and need not match the model's response IDs.
Live Translation
Use gemini-3.5-live-translate-preview for low-latency speech-to-speech
translation. Configure the target language with
providerOptions.google.translationConfig in the realtime session config you
use when creating the token:
Set targetLanguageCode to a BCP-47 language code (defaults to en). When
echoTargetLanguage is true, input audio already in the target language is
echoed instead of producing silence.
Gemini Live Translation accepts audio input and produces translated audio output. Text input, tools, and custom instructions are not supported by this model.
Gemini 3.8 Live
gemini-3.8-live and gemini-3.8-live-extended-thinking are audio-only
realtime models: request outputModalities: ['audio'], and add
outputAudioTranscription: {} if you want the assistant's transcript as
audio-transcript-delta events. Server-side voice activity detection and
proactive audio are always on.
gemini-3.8-live-extended-thinking reasons in the background while streaming
audio. Google requires exactly one of thinkingLevel (low, medium, high)
or thinkingBudget for it; the provider sends thinkingLevel: 'low' when you
set neither. Override it through providerOptions.google.thinkingConfig:
Tools declared in the session config run asynchronously (NON_BLOCKING) by
default on these models, so the model keeps talking while a tool call is in
flight. Set providerOptions.google.defaultToolBehavior: 'BLOCKING' on
gemini-3.8-live to make it wait for the result instead.
On the extended thinking model, turnComplete (the response-done event) does
not mean the model is idle. Watch the interactionStatus custom event: its
raw.interactionStatus is IN_PROGRESS, IDLE, or WAITING_FOR_INPUT. A
waitingForInput custom event is emitted when the model has stopped and needs
user input to continue.
Translation Models
For server-side streaming translation, you can create models that call the
Gemini Live API using the
.translation() factory method. Translation models are streaming-only and are
used with
experimental_streamTranslate.
The first argument is the model id e.g. gemini-3.5-live-translate-preview.
The Gemini Live API auto-detects the source language (sourceLanguage is not
supported) and always outputs 24kHz 16-bit PCM audio (outputAudioFormat is
not supported). Unsupported settings surface as call warnings.
The following provider options are available:
- echoTargetLanguage boolean Whether input audio already in the target language should be echoed instead of producing silence. Optional.
Model Capabilities
| Model | Translated Audio | Translated Text | Source Transcript |
|---|---|---|---|
gemini-3.5-live-translate-preview |
Interactions API
The Gemini Interactions API
(POST /v1beta/interactions) is a separate Google endpoint with server-side
state, unified content blocks, first-class built-in tools, agent presets,
managed agents that run in a sandboxed Linux environment, and native
multimodal image and video output. It is reached via the
google.interactions(...) factory:
google.interactions(...) accepts a model ID string (e.g.
'gemini-4-argon', 'gemini-3.1-pro-preview'), { agent: <name> } to use
a Gemini agent preset, or { managedAgent: <name> } to
invoke a managed agent you created on Google's side.
The returned model can be passed to generateText and streamText like
any other AI SDK language model.
Use google(...) for the standard :generateContent /
:streamGenerateContent endpoints, and google.interactions(...) for the new
Interactions endpoint. Pick one per model instance — they target different
request bodies and SSE event vocabularies.
Reasoning
Use the top-level reasoning option with generateText or streamText to
control thinking depth on Interactions models that support thinking levels:
The provider maps reasoning to generation_config.thinking_level using the
model's supported levels. xhigh is supported only by Gemini 4 Hydrogen.
Unsupported levels fall back to a supported level with a compatibility warning;
none maps to the lowest supported level. Models without thinking-level
support and agent calls emit a warning and omit the setting. Omitting reasoning
or setting it to 'provider-default' preserves the API default.
An explicit providerOptions.google.thinkingLevel takes precedence.
Provider Options
The Interactions model reads its options from the shared
providerOptions.google.* namespace. Validate them with the
GoogleLanguageModelInteractionsOptions type:
The following optional provider options are available:
-
labels Record<string, string>
User-defined metadata sent in the top-level
labelsfield of the Interactions request. For example, set{ safety_identifier: 'opaque_user_123' }to attach an opaque end-user identifier. Other labels are preserved unchanged. Vertex AI Interactions models also read labels fromproviderOptions.google, notproviderOptions.googleVertexorproviderOptions.vertex. -
previousInteractionId string
Server-side interaction id from a prior turn. When set, the server pulls prior context from its own state and only the new user message is sent on the wire. Pair with the default
store: trueto chain stateful conversations. See Stateful chaining. -
store boolean
Whether the server should persist the interaction. Defaults to
true. Set tofalsefor stateless multi-turn conversations where the full message history is re-sent on every turn. -
agent string
Name of a Gemini agent preset (e.g.
'deep-research-pro-preview-12-2025').Prefer the factory form
google.interactions({ agent: '...' })over settingagentin provider options — the factory is type-checked against the supported agent names. -
agentConfig object
Per-agent configuration. Currently supports
{ type: 'dynamic' }and{ type: 'deep-research', thinkingSummaries?, visualization?, collaborativePlanning? }. -
thinkingLevel 'minimal' | 'low' | 'medium' | 'high' | 'xhigh'
Controls reasoning depth for thinking-enabled models. Mapped onto the Interactions request's
thinking_level. Overrides the top-levelreasoningoption when set. Thexhighlevel is only supported by Gemini 4 Hydrogen and requests its maximum reasoning depth. -
thinkingSummaries 'auto' | 'none'
Whether the model returns synthesized thought summaries on reasoning parts. Defaults to the API default.
-
responseFormat Array<ResponseFormatEntry>
Output-format entries that map directly to the API's
response_formatarray. Entries have the following shape:Use this for fine-grained control over image, audio, video, or non-JSON text outputs. Video entries can control aspect ratio, resolution, duration, and inline or URI delivery; set
gcsUriwhen delivering to Google Cloud Storage. Inline videos are returned inresult.filesbygenerateTextand emitted asfileparts bystreamText. The AI SDK call-levelresponseFormat: { type: 'json', schema }still drives JSON-mode automatically and prepends a matching text entry; entries listed here are appended.For image entries,
aspectRatioaccepts1:1,2:3,3:2,3:4,4:3,4:5,5:4,9:16,16:9,21:9,1:8,8:1,1:4,4:1. -
imageConfig { aspectRatio?: string; imageSize?: '1K' | '2K' | '4K' | '512' } (deprecated)
Use responseFormat with a
{ type: 'image', ... }entry instead. Retained for backwards compatibility; the SDK translatesimageConfiginto a matchingresponse_formatimage entry and emits a warning when set. Ignored whenresponseFormatalready supplies an image entry. -
mediaResolution 'low' | 'medium' | 'high' | 'ultra_high'
Media resolution applied to image inputs / outputs.
-
serviceTier 'flex' | 'standard' | 'priority'
Service tier for the request. Mirrored back on
result.providerMetadata.google.serviceTierfor observability. -
systemInstruction string
Alternative to the AI SDK
systemmessage. If both are set, the AI SDKsystemmessage wins and a warning is emitted. -
background boolean
Run the interaction in the background. Required for agents whose server-side workflow cannot complete within a single request/response; rejected by agents that only support synchronous calls. When
true, the POST returns a non-terminal status and the SDK pollsGET /interactions/{id}until the work completes. -
environment string | object
Sandbox environment configuration for managed agents. Pass
'remote'to provision a fresh sandbox, anenvironment_idstring to reuse an existing one, or an object of the form{ type: 'remote', sources?, network? }to preload files and/or constrain outbound traffic. Only applies to agent calls. -
pollingTimeoutMs number
Maximum time, in milliseconds, to poll a background interaction before giving up. Defaults to 30 minutes (1,800,000 ms). Long-running agents may need longer.
Provider Metadata
result.providerMetadata.google (typed via GoogleInteractionsProviderMetadata)
exposes:
-
interactionId string
Server-side interaction id. Pass this back as
previousInteractionIdon the next turn to chain. -
serviceTier string
Service tier the request actually ran on.
-
signature string
Per-block signature hash, set by the SDK on output reasoning and tool-call or agentic video processing parts. Round-tripped automatically on the next turn.
Agentic video custom parts additionally expose:
-
processingId string
ID of an agentic video
processing_call. Present ongoogle.processing_callcustom parts. -
processingCallId string
ID of the matching agentic video processing call. Present on
google.processing_resultcustom parts.
Stateful chaining
With the default store: true, the server retains the prior turn so the
next request only needs to send the new user message and the
previousInteractionId:
For stateless multi-turn conversations, set store: false and re-send the
full message history on every turn (no previousInteractionId):
Agentic video understanding
Gemini 4 Argon, 3.7 Flash, 3.6 Flash, and 3.5 Flash Lite support agentic video understanding through the Interactions API. Instead of sampling the entire video at a fixed frame rate, the model dynamically navigates the timeline and loads the transcripts, frames, or audio needed for the prompt.
Set processing: 'agentic' in the Google provider options of an individual
video file part. You can validate the per-file options with
GoogleInteractionsVideoOptions:
Agentic navigation is exposed as custom content parts:
google.processing_callmarks a request by the model to inspect part of the video. ItsproviderMetadata.google.processingIdcontains the processing ID.google.processing_resultmarks completion of that request. ItsproviderMetadata.google.processingCallIdlinks it to the processing call.
These parts are informational and do not require a tool response. They are
available from both generateText and streamText.
For static processing, use 'static' or provide clipping and frame-rate
options. startOffset and endOffset are measured in seconds:
With stateful conversations, pass the returned interactionId as
previousInteractionId; Gemini retains the video context server-side. With
store: false, append result.responseMessages and re-send the full history.
The processing custom parts and their signatures are then round-tripped as
native processing_call and processing_result steps so the video context is
preserved.
Built-in Tools
The Interactions API ships a built-in tool catalog. The provider-defined
tools under google.tools.* map onto Interactions tool descriptors:
| AI SDK tool | Interactions tool type | Notes |
|---|---|---|
google.tools.googleSearch | google_search | Web / image search grounding. |
google.tools.codeExecution | code_execution | Server-side Python execution. |
google.tools.urlContext | url_context | Fetch URLs referenced in the prompt. |
google.tools.fileSearch | file_search | Retrieval from File Search stores. |
google.tools.googleMaps | google_maps | Maps grounding for nearby-places queries. |
provider tool google.computer_use | computer_use | Computer use (browser environment). |
provider tool google.mcp_server | mcp_server | Remote MCP server passthrough. |
provider tool google.retrieval | retrieval | Vertex AI Search retrieval. |
Function tools (type: 'function') defined with the AI SDK tool(...)
helper are translated to Interactions function tool descriptors. Other
tool kinds emit a warning and are dropped.
Function tools work the same way as on the standard provider:
Image output via Interactions
Add a { type: 'image' } entry to responseFormat on a Gemini
image-capable model to get images as LanguageModelV4FilePart files in
the response. No tool wrapping is required, and the entry doubles as the
place to set aspectRatio, imageSize, and mimeType.
To control aspect ratio, image size, or output mime type, add those fields to the same image entry:
For multimodal output, list one entry per modality. The model returns
text in result.text and the accompanying image(s) in result.files:
Iterative image editing pairs naturally with stateful chaining — keep
previousInteractionId set across turns and the model edits its prior
output:
Agent presets
Pass { agent: <name> } to target a Gemini agent preset. The factory
type-checks the agent name against the supported set:
Whether an agent runs synchronously or in the background depends on the
agent. Long-running presets (such as the deep-research-* family)
require background: true — the POST returns a non-terminal status and
the SDK polls GET /interactions/{id} internally until the interaction
completes. Other agents accept synchronous calls only and will reject
background: true. Set the flag explicitly via
providerOptions.google.background.
The default polling timeout is 30 minutes; raise it via
pollingTimeoutMs for slower agents:
Agents also chain through previousInteractionId like model-id calls.
Managed Agents
Managed agents run inside a
sandboxed Linux environment provisioned per interaction. Pass the environment
provider option to control how the sandbox is set up; the option is only
accepted on agent calls.
The simplest form provisions a fresh sandbox:
environment accepts three shapes:
'remote'— provision a fresh sandbox for this call.- any other string — an
environment_idto reuse, forking the previous sandbox so its filesystem and installed packages persist. - an object — provision a fresh sandbox and optionally preload
sourcesand/or constrain outbound traffic vianetwork:
Three source types are supported: inline (write a string into the
sandbox at target), repository (clone a git repository — pass the
URL as source), and gcs (mount a Google Cloud Storage prefix).
The network field accepts the string 'disabled' to block all
outbound traffic, or an object with an allowlist array whose entries
each carry a domain plus an optional transform array of header
objects to inject into matching requests.
Custom managed agents
For user-defined agents that you created on Google's side via the
Gemini API's /v1beta/agents endpoint, pass the agent's name through the dedicated
managedAgent factory shape instead of agent (which only accepts
known preset names):
Streaming
streamText is supported. The stream's finish part exposes
interactionId on providerMetadata.google so callers can chain.
Gemma Models
You can use Gemma models with the Google Generative AI API. The following Gemma models are available:
gemma-3-27b-itgemma-3-12b-it
Gemma models don't natively support the systemInstruction parameter, but the provider automatically handles system instructions by prepending them to the first user message. This allows you to use system instructions with Gemma models seamlessly:
The system instruction is automatically formatted and included in the conversation, so Gemma models can follow the guidance without any additional configuration.
Embedding Models
You can create models that call the Google Generative AI embeddings API
using the .embedding() factory method.
The Google provider sends API calls to the right endpoint based on the type of embedding:
- Single embeddings: When embedding a single value with
embed(), the provider uses the single:embedContentendpoint, which typically has higher rate limits compared to the batch endpoint. - Batch embeddings: When embedding multiple values with
embedMany()or multiple values inembed(), the provider uses the:batchEmbedContentsendpoint.
Google embedding models support additional settings. You can pass them as an options argument:
When using embedMany, provide per-value multimodal content via the content option. Each entry corresponds to a value at the same index; use null for text-only entries:
The following optional provider options are available for Google embedding models:
-
outputDimensionality: number
Optional reduced dimension for the output embedding. If set, excessive values in the output embedding are truncated from the end.
-
taskType: string
Optional. Specifies the task type for generating embeddings. Supported task types include:
SEMANTIC_SIMILARITY: Optimized for text similarity.CLASSIFICATION: Optimized for text classification.CLUSTERING: Optimized for clustering texts based on similarity.RETRIEVAL_DOCUMENT: Optimized for document retrieval.RETRIEVAL_QUERY: Optimized for query-based retrieval.QUESTION_ANSWERING: Optimized for answering questions.FACT_VERIFICATION: Optimized for verifying factual information.CODE_RETRIEVAL_QUERY: Optimized for retrieving code blocks based on natural language queries.
-
content: array
Optional. Per-value multimodal content parts for embedding non-text content (images, video, PDF, audio). Each entry corresponds to the embedding value at the same index — its parts are merged with the text value in the request. Use
nullfor entries that are text-only. The array length must match the number of values being embedded. Each non-null entry is an array of parts, where each part can be{ text: string },{ inlineData: { mimeType: string, data: string } }for inline base64 data, or{ fileData: { fileUri: string, mimeType: string } }to reference remote content via HTTP URL or Google Cloud Storage URI (gs://...). Supported bygemini-embedding-2-preview.
Model Capabilities
| Model | Default Dimensions | Custom Dimensions | Multimodal |
|---|---|---|---|
gemini-embedding-001 | 3072 | ||
gemini-embedding-2 | 3072 | ||
gemini-embedding-2-preview | 3072 |
Image Models
You can create image models that call the Google Generative AI API using the .image() factory method.
For more on image generation with the AI SDK see generateImage().
The .image() factory supports Gemini multimodal language models with image
output capabilities through the :generateContent API.
Gemini Image Models
Gemini image models (e.g. gemini-3.1-flash-image-preview) are technically multimodal output language models, but they can be used with the generateImage() function for a simpler image generation experience. Internally, the provider calls the language model API with responseModalities: ['IMAGE'].
Gemini image models also support image editing by providing input images:
You can also use URLs for input images:
Gemini image models do not support the size or n parameters. Use
aspectRatio instead of size. Mask-based inpainting is also not supported.
For more advanced use cases where you need both text and image outputs, or
want more control over the generation process, you can use Gemini image models
directly with generateText(). See Image Outputs for
details.
Google Search Grounding
Gemini image models support Google Search grounding through providerOptions.google.googleSearch. The value matches the args of google.tools.googleSearch(...); pass {} to enable with defaults, or { searchTypes: { imageSearch: {} } } to ground on reference photos.
Gemini Image Model Capabilities
| Model | Image Generation | Image Editing | Aspect Ratios |
|---|---|---|---|
gemini-2.5-flash-image | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9 | ||
gemini-3-pro-image-preview | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9 | ||
gemini-3.1-flash-image-preview | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9 | ||
gemini-nano-banana-2.1 | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9, 1:8, 8:1, 1:4, 4:1 |
gemini-3-pro-image-preview supports additional features including up to 14
reference images for editing (6 objects, 5 humans), resolution options (1K,
2K, 4K via providerOptions.google.imageConfig.imageSize), and Google Search
grounding.
Speech Models
You can create models that call the Gemini text-to-speech API
using the .speech() factory method.
The first argument is the model ID, such as gemini-3.8-flash-tts or
gemini-3.8-flash-lite-tts. Speech generation uses the generateContent API.
The voice argument can be set to one of Gemini's 30 prebuilt voices
e.g. Kore, Puck, Zephyr, or Charon. Voice names are case-sensitive.
The default voice is Kore. Custom voice IDs and replication keys are not supported.
By default the generated audio is returned as a playable WAV file
(result.audio.mediaType is audio/wav). Gemini 3.8 returns a complete WAV file;
the provider passes those bytes through unchanged. Older models return raw PCM,
which the provider wraps in a WAV container. Write result.audio.uint8Array
directly to disk without adding another WAV header.
Set language to a supported BCP-47 language code, such as en-US or de-DE.
The provider forwards explicit codes unchanged as speechConfig.languageCode
for both single- and multi-speaker requests. Omit language or set it to 'auto'
to let Gemini detect the language from the input text.
Gemini 3.8 supports the following outputFormat values:
| Format | Values | Result |
|---|---|---|
| WAV | wav, audio/wav | WAV with a RIFF header |
| PCM | pcm, audio/l16 | Headerless signed 16-bit PCM |
| Mu-law | mulaw, audio/mulaw | Headerless mu-law audio |
| A-law | alaw, audio/alaw | Headerless A-law audio |
These map to generationConfig.responseFormat.audio.mimeType in
generateContent. Older models support wav and pcm. The source MIME type and
sample rate are available in result.providerMetadata.google.
Gemini 2.5 and 3.1 model IDs retain the legacy request format. Other model IDs, including custom aliases, use the structured speech format.
Speech directions in Gemini 3.8
Gemini 3.8 treats text as a verbatim transcript. The provider sends instructions
as speechMetadata.style, leaving the text unchanged. You can also supply
providerOptions.google.speechMetadata with style and speaker fields.
An explicit style overrides instructions, including an empty style string.
Put sustained delivery directions such as whispering, speaking slowly, or being
out of breath in metadata. Keep momentary vocal events and pauses such as
<laugh>, <sigh>, <cough>, <breath>, and <short pause> in the transcript.
Avoid sound-effect tags such as applause or thuds. The provider preserves inline
tags as written.
For Gemini 3.1 and earlier, the provider retains the existing behavior of
prepending instructions to single-speaker text. It ignores instructions with a
warning for older multi-speaker requests. Structured turns and speechMetadata
require Gemini 3.8.
Multi-speaker audio
For Gemini 3.8 dialogue, pass multiSpeakerVoiceConfig and turns through
providerOptions.google. Every turn must include speechMetadata.speaker
matching a configured speaker; missing or unknown speakers produce an error
before the request is sent. Speaker labels belong in metadata, not the text.
The turns array replaces the top-level text, so pass text: ''. The combined
turn text must be non-empty. Passing
nonempty top-level text with turns produces a warning. Put metadata on each turn
instead of combining turns with top-level speechMetadata. Each turn's style
overrides instructions; turns without a style inherit instructions.
multiSpeakerVoiceConfig overrides the top-level voice. Each voiceConfig
uses prebuiltVoiceConfig: { voiceName: 'Kore' }. Google supports up to two
prebuilt voices in a multi-speaker request.
Older models continue to use labelled transcripts, such as
text: 'Joe: How are you? Jane: Doing great, thanks!', with
multiSpeakerVoiceConfig and without turns.
Gemini TTS models do not support the speed option; passing it adds a warning
to result.warnings.
Model Capabilities
| Model | Multi-speaker | Style via instructions |
|---|---|---|
gemini-2.5-flash-preview-tts | ||
gemini-2.5-pro-preview-tts | ||
gemini-3.1-flash-tts-preview | ||
gemini-3.8-flash-tts | ||
gemini-3.8-flash-lite-tts |
Decision Models
Create an experimental decision model with google.decisionModel(modelId).
It calls the Gemini Decisions API directly for Choice, Score, and Boolean questions.
The endpoint requires v1beta and is not available on Vertex AI.
Choice and Score answers include native probability distributions. Scores are
probability-weighted positions on the zero-based ordered rubric. Boolean questions
map to Google's noul type, and answers expose its calibrated P(true) as
probability. During EAP, that probability is between 0.01 and 0.99.
Apply thresholds in application code, for example
answers.requestsRefund.probability >= 0.5.
Confidence values are available at result.providerMetadata.google.confidence,
keyed by question ID. Score legends are at result.providerMetadata.google.legend.
Token usage includes input tokens and zero output tokens; modality details are
preserved in result.providerMetadata.google.usage. A safety-blocked request
produces refusals, which experimental_decide surfaces as
Experimental_DecisionRefusalError.
State can contain text, JSON, images, audio, video, and PDFs. Structured state, instructions, and criteria descriptions are serialized as text. Every question requires instructions. Choice questions require 2–255 options; Score questions require 2–255 non-empty level descriptions. Question IDs must contain 1–64 ASCII letters, digits, underscores, or hyphens. Choice option IDs must contain 1–128 characters without leading or trailing whitespace. Google sorts Choice IDs alphabetically and breaks exact probability ties in that order.
The factory respects createGoogle settings, including API keys, base URLs,
headers, and custom fetch implementations. Generation options such as
maxOutputTokens, reasoning, and providerOptions.google.thinkingConfig are
unsupported and produce warnings. Decisions do not use structured output or
fall back to generateContent. The deprecated evaluationModel factory and
doEvaluate method remain aliases for the native implementation.
Decision models can also be accessed through customProvider aliases or
createProviderRegistry().decisionModel('provider:model'). Direct string IDs
use Gateway by default, or an explicitly configured default provider with a
decisionModel method. See
model aliases and registries.
Extraction Models
Create an experimental extraction model with google.extractionModel(modelId).
It wraps a Google language model and uses the Gemini API's structured output to
extract typed fields in one call.
The adapter supports every field type and returns null when the state has no
value for a field. It does not return pNull, confidence, guess, or
source, since a prompted model cannot give calibrated estimates or exact
offsets. Image and PDF files are passed to the language model, which must
support them. The adapter requests reasoning: 'none' by default and forwards
providerOptions.google. See Extractions.