Skip to content

fix(server-utils): Count Google GenAI tool-use prompt tokens as input tokens - #25834

Merged
andreiborza merged 3 commits into
getsentry:developfrom
pcbeingused333:fix/google-genai-tool-use-tokens
Nov 16, 2026
Merged

andreiborza merged 3 commits into
getsentry:developfrom
pcbeingused333:fix/google-genai-tool-use-tokens

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

Gemini reports the tokens of built-in tool results fed back to the model (Google Search, URL context, code execution) in usageMetadata.toolUsePromptTokenCount, apart from promptTokenCount. totalTokenCount includes both ("Total token count for prompt, response candidates, and tool-use prompts (if present)" in types.ts).

addResponseAttributes and the stream's handleResponseMetadata only used promptTokenCount for gen_ai.usage.input_tokens, so with those tools input_tokens + output_tokens did not add up to gen_ai.usage.total_tokens.

The tool-use prompt tokens are now added to the input count, for a response and for a stream (the two counts are tracked separately, so a chunk that repeats only one of them keeps the other). Responses without tool-use tokens are unchanged.

Tests in google-genai-streaming.test.ts cover a response and a stream with toolUsePromptTokenCount (input 140, total 160) and a response without it. The two with tool-use tokens fail on develop.

This change was made with Claude Code.

… tokens

Gemini reports the tokens of built-in tool results (Google Search, URL context,
code execution) in usageMetadata.toolUsePromptTokenCount, apart from
promptTokenCount, while totalTokenCount includes both. gen_ai.usage.input_tokens
only took promptTokenCount, so input + output did not add up to the total.
Add the tool-use prompt tokens to the input count for responses and streams.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@pcbeingused333
pcbeingused333 requested a review from a team as a code owner October 28, 2026 15:41
@pcbeingused333
pcbeingused333 requested review from andreiborza and isaacs and removed request for a team October 28, 2026 15:41
@github-actions github-actions Bot added the external PR from an external contributor label Oct 28, 2026
@Lms24

Lms24 commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Is there an issue that was raised before you opened this PR? Please do not blindly contribute PRs without discussing the problem first. Also, from the description, it is not clearly visible if you actually experienced the problem or just told an agent to look for bugs in our code base. See our contributing guidelines.

This change was made with Claude Code.

That's fine but we expect from you that you 1. write the description yourself and 2. fully understand the code change.

@pcbeingused333

Copy link
Copy Markdown
Contributor Author

Thanks for the review, and fair points.

To answer directly: there was no issue before this PR, and the problem was not hit in a production setup. It was found by reading the Google GenAI instrumentation against the usageMetadata types (where totalTokenCount is documented to include toolUsePromptTokenCount) with the help of an AI coding assistant, not by observing wrong numbers in a real app. The two new tests that fail on develop are the only reproduction so far.

Opening a PR without an issue was a mistake on our side, and the guidelines for external contributors will be followed from here on. To be transparent: this reply, like the PR description, was drafted by an AI coding assistant (Claude Code) working on Alex's behalf, so if you require the description and responses to be written personally by the contributor, the right call is to close this PR. Otherwise, an issue with the details can be opened and this PR left parked until you confirm the change is wanted.

@andreiborza
andreiborza requested a review from a team November 16, 2026 13:21

@andreiborza andreiborza left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution.

@andreiborza
andreiborza merged commit d078e04 into getsentry:develop Nov 16, 2026
397 checks passed
nicohrubec pushed a commit that referenced this pull request Nov 16, 2026
This PR adds the external contributor to the CHANGELOG.md file, so that
they are credited for their contribution. See #25834

Co-authored-by: andreiborza <168741329+andreiborza@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external PR from an external contributor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants