fix(anthropic-adapter): correct streaming for reasoning_content and empty tool_calls - #29
Closed
lukesorvikDO wants to merge 4 commits into
Closed
Conversation
lukesorvikDO
changed the base branch from
release/v1.83.10-stable.do.2
to
release/v1.83.10-stable.do.3
June 30, 2026 18:43
3 tasks
ShivaniKumar1
requested changes
Jun 30, 2026
Comment on lines
+1406
to
+1412
| elif isinstance(choice, StreamingChoices) and hasattr( | ||
| choice.delta, "reasoning_content" | ||
| ): | ||
| if choice.delta.reasoning_content is not None: | ||
| return "thinking", ChatCompletionThinkingBlock( | ||
| type="thinking", thinking="", signature="" | ||
| ) |
Collaborator
There was a problem hiding this comment.
Empty reasoning_content can open a thinking block without subsequent thinking_delta, violating Anthropic streaming sequence and breaking clients. Can you make sure this is covered?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Testing Doc : https://docs.google.com/document/d/1LpUJYUc2a2s5iMxbg_6JMiqQEc0sDDOuTeJvVqonMB0/edit?tab=t.f2qn8vnp2dx9#heading=h.xf1r939ub9pb
Original pr: #21
Relevant issues
Fixes invalid Anthropic SSE from the experimental pass-through adapter when models return reasoning_content. Claude Code rejects the stream (Content block is not a thinking block) and retries with a non-streaming request, causing 2× POST /v1/messages per turn through inference-proxy.
Problem
When a model returns reasoning_content (e.g. reasoning models), the adapter opened a text content block but emitted thinking_delta events. That violates the Anthropic SSE contract. Claude Code aborts the stream and falls back to non-streaming → duplicate requests and double routing cost.
Local live test
Setup: pip install -e ".[proxy]" in venv, Ollama deepseek-r1:1.5b at http://localhost:11434
Before fix ❌
thinking_delta on a text block → invalid SSE.
After fix ✅
content_block_start correctly opens a thinking block before thinking_delta events.
Unit tests
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
make test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes