Skip to content

fix(responses): include encrypted_content in streamed reasoning done events - #45044

Open
jinwukong wants to merge 4 commits into
BerriAI:mainfrom
jinwukong:bugfix/responses-reasoning-done-replay
Open

jinwukong wants to merge 4 commits into
BerriAI:mainfrom
jinwukong:bugfix/responses-reasoning-done-replay

Conversation

@jinwukong

@jinwukong jinwukong commented Oct 7, 2026 •

Copy link
Copy Markdown

Handwritten 1-3 sentence description

[insert handwritten description here]

Details

TLDR

Problem this solves:

  • Streamed reasoning done items lose the signed thinking replay payload

How it solves it:

  • Encode the same accumulated thinking blocks used by the completed response

User Flow

Before, a client saving streamed items loses the signed thinking blocks

  1. Send POST http://localhost:4000/v1/responses with streaming, reasoning and a function tool
  2. Save each response.output_item.done item for the next turn
  3. Replay those items with the tool result, without the signed thinking available in response.completed

After, the saved reasoning item carries that replay payload

  1. Send the same streaming request to POST http://localhost:4000/v1/responses
  2. Save each response.output_item.done item, now including the reasoning item's encrypted_content
  3. Replay those items with the tool result, preserving the signed thinking blocks

Relevant issues

Fixes #46032

Follow-up to #43010 and #44127, which fixed duplicated thinking text. Supersedes #40673, narrowed to this one defect

Pre-Submission checklist

  • I have added meaningful tests
  • The two affected transformation test files pass locally, 584 passed on cc4ea66780. Removing the done-item payload makes all eight replay regression cases fail, and restoring it returns all 584 tests to green
  • My PR passes all required CI/CD checks on cc4ea66780. The current tip's GitHub Actions workflows are action_required, with no jobs started, and need maintainer approval
  • My PR's scope is as isolated as possible; it only solves one specific problem
  • Every automated review on the latest commit has finished and its findings have been addressed

Merged main 478d9bf5d2 into cc4ea66780, preserving upstream's sequence-number and refusal tests alongside the replay regressions. The no-snapshot test now uses the helper's updated signature. The diff remains two files, +116/-1

BASE_REF=origin/main make check passes, including source/test lint, formatting, strict-ruff, type-discipline, test-quality, OpenAPI docs, basedpyright gates and API schema sync. All added executable lines are covered by the two transformation test files, with 93% line coverage for the streaming iterator overall

Local test command, with the locked development dependencies installed, is LITELLM_LOCAL_MODEL_COST_MAP=True uv run --no-sync python -m pytest tests/unit/responses/litellm_completion_transformation/test_streaming_iterator_transformation.py tests/unit/responses/litellm_completion_transformation/test_litellm_completion_responses.py -q

Screenshots / Proof of Fix

The October 18 live proof below was tested at 5aa433250c and is retained as historical evidence, not a fresh run at cc4ea66780. The signed-thinking payload implementation is unchanged, and this merge preserves upstream behavior while updating one test for centralized sequence numbering. Current-tip verification is the 584 passing tests, covered added lines, mutation check and make check reported above. Hosted CI and automated reviews still need to complete

Live comparison on October 18 using anthropic/claude-opus-5-5, through the Python proxy with real provider calls. The local proxy has a generated master key. ANTHROPIC_API_KEY and LITELLM_MASTER_KEY are supplied through the environment

model_list:
  - model_name: replay-check
    litellm_params:
      model: anthropic/claude-opus-5-5
      api_key: os.environ/ANTHROPIC_API_KEY

Save this as request.json

{
  "model": "replay-check",
  "input": [{"role": "user", "content": "A warehouse has five boxes containing 1469 screws each and 73 spare screws. Work out the number packed in the boxes, then use the addition tool to combine it with the spare screws. After the tool returns, answer with only the total."}],
  "stream": true,
  "max_output_tokens": 2048,
  "reasoning": {"effort": "high"},
  "tools": [{"type": "function", "name": "add", "description": "Add two integers and return their sum.", "parameters": {"type": "object", "properties": {"a": {"type": "integer"}, "b": {"type": "integer"}}, "required": ["a", "b"], "additionalProperties": false}, "strict": true}]
}

Run the same commands against each checkout's proxy. The second request uses only the saved done items and the tool result

curl -sS -N http://localhost:4000/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' -d @request.json > events.sse

sed -n 's/^data: //p' events.sse | sed '/^\[DONE\]$/d' |
  jq -s --slurpfile req request.json '
    [.[] | select(.type == "response.output_item.done") | .item] as $items |
    $req[0] + {stream: false, input: ($req[0].input + $items + [{
      type: "function_call_output",
      call_id: ($items[] | select(.type == "function_call") | .call_id),
      output: "7418"
    }])}' > replay.json

curl -sS http://localhost:4000/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' -d @replay.json

Before (7db8fc81f9)

  1. The streamed reasoning done item has no encrypted_content, so the saved items contain zero signed thinking blocks
  2. Replaying those items returns HTTP 200 and 7418, but the signed thinking from the completed response was not preserved

After (5aa433250c)

  1. The streamed reasoning done item has encrypted_content identical to the completed response, containing one signed thinking block
  2. Replaying those items returns HTTP 200 and 7418, preserving that signed thinking block

The observed change is replay-data preservation. The before case also returns the correct arithmetic answer

Type

Bug Fix

@greptile-apps

greptile-apps Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium risk] Adds encrypted thinking content to reasoning stream events.

The PR appears safe to merge, with no actionable issues found

What we checked:

  • Saved reasoning loses signed blocks: The done item now encodes the saved thinking blocks with the same helper as the completed response. Replay decodes that field back into thinking blocks

Summary

Adds encrypted_content to streamed reasoning done items using the same encoder as completed responses

  • Preserves signed and redacted thinking blocks for replay
  • Adds eight cases covering text and tool endings, plus a case with no response snapshot
  • No actionable issues found. Tests were inspected but not run

Reviews (1) Β· Last reviewed commit: "fix(responses): include encrypted_conten..." Β· Reviewed by Greptile

@codspeed

codspeed Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

βœ… 32 untouched benchmarks


Comparing jinwukong:bugfix/responses-reasoning-done-replay (5aa4332) with main (7db8fc8)

Open in CodSpeed

@jinwukong

Copy link
Copy Markdown
Author

@mateo-berri Could you review? One-commit follow-up to #44127: streamed reasoning done items now carry the same encrypted_content as response.completed. 100 checks green, Greptile 5/5.

@codecov

codecov Bot commented Oct 7, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.

πŸ“’ Thoughts on this report? Let us know!

…events

response.output_item.done for a reasoning item only carried the summary
text. The signed and redacted thinking blocks were encoded into
encrypted_content on response.completed alone, so a client that persists
the streamed items replays the next turn without the signed thinking.

Encode the accumulated thinking_blocks with the shared helper when the
done event is built, so the item carries the same encrypted_content as
the completed response. Nothing changes when no snapshot exists yet.
@jinwukong
jinwukong force-pushed the bugfix/responses-reasoning-done-replay branch from dc60d77 to 5aa4332 Compare October 18, 2026 01:55
@jinwukong

Copy link
Copy Markdown
Author

Rebased onto current main, preserving upstream tool-search changes. All 407 targeted tests pass, and live signed-thinking replay still matches response.completed

@jinwukong

Copy link
Copy Markdown
Author

@greptileai Please review the rebased commit 5aa433250c. The change remains limited to preserving signed thinking in reasoning done events

Signed-off-by: jinwukong <134120444+jinwukong@users.noreply.github.com>
Signed-off-by: jinwukong <134120444+jinwukong@users.noreply.github.com>
@TobiMayr

Copy link
Copy Markdown
Contributor

Need this as well. We save the streamed reasoning done item and replay it, so the signed thinking has to be on that event.

@veria-ai

veria-ai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Reviews

Copy link
Copy Markdown
Author

Conflicts resolved, 584 local tests pass. @mateo-berri, could you approve CI? Another user also needs signed thinking on streamed reasoning events

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Responses API streaming: reasoning output_item.done has no encrypted_content, so replayed turns lose the signed thinking blocks

2 participants