-
Notifications
You must be signed in to change notification settings - Fork 2.3k
feat: add thinking/reasoning data support to LiteLLM provider #8498
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
- Handle reasoning, thinking, and reasoning_content fields in streaming responses - Pass through thinking data from underlying models as reasoning chunks - Add comprehensive test coverage for all thinking data scenarios - Fix handling of chunks without choices array Fixes #8497
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Self-review: judging my own code like a mirror wired to unit tests—existential and merciless.
// Check for reasoning/thinking content in the delta | ||
// LiteLLM may pass through reasoning content from underlying models | ||
if (delta) { | ||
if ("reasoning" in delta && delta.reasoning && typeof delta.reasoning === "string") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[P2] Emit at most one reasoning chunk per delta to avoid duplicated output when multiple alias fields are present in a single chunk.
if ("reasoning" in delta && delta.reasoning && typeof delta.reasoning === "string") { | |
if ("reasoning" in delta && delta.reasoning && typeof delta.reasoning === "string") { | |
yield { type: "reasoning", text: delta.reasoning } | |
} else if ("thinking" in delta && delta.thinking && typeof delta.thinking === "string") { | |
yield { type: "reasoning", text: delta.thinking } | |
} else if ( | |
"reasoning_content" in delta && | |
delta.reasoning_content && | |
typeof delta.reasoning_content === "string" | |
) { | |
yield { type: "reasoning", text: delta.reasoning_content } | |
} |
yield { type: "reasoning", text: delta.reasoning_content } | ||
} | ||
|
||
if (delta.content) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[P2] Add a type guard for delta.content to ensure only string fragments are emitted (prevents accidental non-string values from being yielded).
if (delta.content) { | |
if (typeof delta.content === "string" && delta.content) { | |
yield { type: "text", text: delta.content } | |
} |
Summary
This PR attempts to address Issue #8497 by adding support for displaying thinking/reasoning data from LiteLLM in Roo Code's thinking block.
Problem
LiteLLM can display thinking data from underlying models, but Roo Code wasn't capturing and displaying this data in the thinking block.
Solution
reasoning
,thinking
,reasoning_content
) for compatibility with different LiteLLM model responsesApiStreamReasoningChunk
objects to be displayed in the UIChanges
src/api/providers/lite-llm.ts
to handle thinking/reasoning fields in streaming responsessrc/api/providers/__tests__/lite-llm.spec.ts
Testing
reasoning
field in deltathinking
field in deltareasoning_content
field in deltaReview Confidence
Implementation review showed 92% confidence with no issues identified.
Fixes #8497
Feedback and guidance are welcome!
Important
Adds support for handling and displaying thinking/reasoning data from LiteLLM in Roo Code's UI.
LiteLLMHandler
inlite-llm.ts
now detectsreasoning
,thinking
, andreasoning_content
fields in streaming responses, yielding them asApiStreamReasoningChunk
objects.lite-llm.spec.ts
for handlingreasoning
,thinking
, andreasoning_content
fields.lite-llm.ts
.This description was created by
for dab8d4c. You can customize this summary. It will automatically update as commits are pushed.