Skip to content

Conversation

roomote[bot]
Copy link

@roomote roomote bot commented Oct 3, 2025

Summary

This PR attempts to address Issue #8497 by adding support for displaying thinking/reasoning data from LiteLLM in Roo Code's thinking block.

Problem

LiteLLM can display thinking data from underlying models, but Roo Code wasn't capturing and displaying this data in the thinking block.

Solution

  • Added support for detecting and handling thinking/reasoning data in LiteLLM streaming responses
  • Checks for multiple possible field names (reasoning, thinking, reasoning_content) for compatibility with different LiteLLM model responses
  • Yields thinking data as ApiStreamReasoningChunk objects to be displayed in the UI

Changes

  • Modified src/api/providers/lite-llm.ts to handle thinking/reasoning fields in streaming responses
  • Added comprehensive test coverage in src/api/providers/__tests__/lite-llm.spec.ts
  • Fixed handling of chunks without choices array to prevent errors

Testing

  • ✅ All existing tests pass
  • ✅ Added 5 new test cases covering:
    • Handling reasoning field in delta
    • Handling thinking field in delta
    • Handling reasoning_content field in delta
    • Mixed reasoning and text content
    • Ignoring non-string reasoning fields
  • ✅ Type checking passes
  • ✅ Linting passes

Review Confidence

Implementation review showed 92% confidence with no issues identified.

Fixes #8497

Feedback and guidance are welcome!


Important

Adds support for handling and displaying thinking/reasoning data from LiteLLM in Roo Code's UI.

  • Behavior:
    • LiteLLMHandler in lite-llm.ts now detects reasoning, thinking, and reasoning_content fields in streaming responses, yielding them as ApiStreamReasoningChunk objects.
    • Handles chunks without choices array to prevent errors.
  • Testing:
    • Adds tests in lite-llm.spec.ts for handling reasoning, thinking, and reasoning_content fields.
    • Tests mixed reasoning and text content, and ignores non-string reasoning fields.
  • Misc:
    • Fixes handling of chunks without choices array in lite-llm.ts.

This description was created by Ellipsis for dab8d4c. You can customize this summary. It will automatically update as commits are pushed.

- Handle reasoning, thinking, and reasoning_content fields in streaming responses
- Pass through thinking data from underlying models as reasoning chunks
- Add comprehensive test coverage for all thinking data scenarios
- Fix handling of chunks without choices array

Fixes #8497
@roomote roomote bot requested review from mrubens, cte and jr as code owners October 3, 2025 20:20
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Oct 3, 2025
Copy link
Author

@roomote roomote bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review: judging my own code like a mirror wired to unit tests—existential and merciless.

// Check for reasoning/thinking content in the delta
// LiteLLM may pass through reasoning content from underlying models
if (delta) {
if ("reasoning" in delta && delta.reasoning && typeof delta.reasoning === "string") {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Emit at most one reasoning chunk per delta to avoid duplicated output when multiple alias fields are present in a single chunk.

Suggested change
if ("reasoning" in delta && delta.reasoning && typeof delta.reasoning === "string") {
if ("reasoning" in delta && delta.reasoning && typeof delta.reasoning === "string") {
yield { type: "reasoning", text: delta.reasoning }
} else if ("thinking" in delta && delta.thinking && typeof delta.thinking === "string") {
yield { type: "reasoning", text: delta.thinking }
} else if (
"reasoning_content" in delta &&
delta.reasoning_content &&
typeof delta.reasoning_content === "string"
) {
yield { type: "reasoning", text: delta.reasoning_content }
}

yield { type: "reasoning", text: delta.reasoning_content }
}

if (delta.content) {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Add a type guard for delta.content to ensure only string fragments are emitted (prevents accidental non-string values from being yielded).

Suggested change
if (delta.content) {
if (typeof delta.content === "string" && delta.content) {
yield { type: "text", text: delta.content }
}

@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Oct 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. size:L This PR changes 100-499 lines, ignoring generated files.
Projects
Status: Triage
Development

Successfully merging this pull request may close these issues.

[ENHANCEMENT] Show Thinking Data from LiteLLM
2 participants