Support for DeepSeek V3.1 Terminus/Turbo Variants and Hybrid Model Reasoning via Chutes.ai Provider

[kenOfYugen]

Problem (one or two sentences)

Users cannot access the specific DeepSeek V3.1 Terminus and Turbo model variants through the Chutes.ai provider in Roo Code. Furthermore, the critical "reasoning" mode for hybrid models like DeepSeek V3.1 and GLM-4.5 is not functional (#7370), despite being a core feature of these models that significantly improves output quality on complex tasks.

Context (who is affected and when)

This affects all Roo Code users who rely on state-of-the-art open-weight models for complex reasoning, coding, and creative tasks. When a user selects an OpenAI-Compatible provider like Chutes.ai and attempts to use the latest DeepSeek V3.1 models (Terminus/Turbo) or enable the reasoning effort for hybrid models, the functionality is either unavailable or does not work as expected.

Desired behavior (conceptual, not technical)

Model Variant Support: Users should be able to select and use deepseek-ai/DeepSeek-V3.1-Terminus and deepseek-ai/DeepSeek-V3.1-turbo as model options when the Chutes.ai (or similar OpenAI-compatible) provider is selected.

Functional Reasoning Mode: When the "Enable reasoning" toggle is enabled for supported hybrid models (DeepSeek V3.1, GLM-4.5), the model should actually engage its chain-of-thought reasoning process. The "Thinking" section should become visible in the UI, showing the model's step-by-step reasoning before presenting the final answer.

Constraints / preferences (optional)

UX Consistency: The experience should be consistent with how reasoning works for other supported models (e.g., claude-sonnet models).

Provider Flexibility: The solution should ideally be adaptable to work with OpenAI-compatible API endpoint that offers these models and supports their reasoning mode, including [Chutes.ai](https://chutes.ai/).

Request checklist

• I've searched existing Issues and Discussions for duplicates
• This describes a specific problem with clear context and impact

Roo Code Task Links (optional)

No response

Acceptance criteria (optional)

Criterion 1: Model Selection

Given the [Chutes.ai](https://chutes.ai/) provider is active in Roo Code,

When the user opens the model selection dropdown,

Then the options "DeepSeek-V3.1-Terminus" and "DeepSeek-V3.1-Turbo" should be present and selectable.

And these options should be clearly labeled and accessible.

But no unsupported or incorrect models should appear in the dropdown for this provider.

Criterion 2: API Call Verification

Given the user has selected "DeepSeek-V3.1-Terminus" (or a similar variant) from the dropdown,

When the user sends a chat message,

Then the API request to the [Chutes.ai](https://chutes.ai/) endpoint must include the correct model parameter (e.g., "deepseek-ai/DeepSeek-V3.1-Terminus").

And the request should be properly structured according to the API specifications.

But the model parameter should not be missing, default to an incorrect value, or cause API errors.

Criterion 3: Reasoning Toggle Efficacy

Given the user has selected a DeepSeek V3.1 model that supports reasoning (e.g., DeepSeek-V3.1),

When the user enables the "Enable reasoning" toggle in the UI,

Then the API request must include the necessary parameters to trigger reasoning mode (e.g., a specific token like "<｜begin▁of▁sentence｜>" or a JSON flag).

And the toggle should provide clear visual feedback (e.g., highlighted) to indicate it is active.

But the reasoning mode should not be activated if the toggle is disabled, or if the model does not support it.

Criterion 4: UI Feedback for Reasoning

Given reasoning mode is enabled and a chat request is sent,

When the model streams its response,

Then the UI must display a distinct "Thinking" section that shows the reasoning process in real-time, separate from the final answer.

And the reasoning content should be parsed correctly from the stream (e.g., based on delimiters like "<｜begin▁of▁sentence｜>").

But the "Thinking" section should not appear if reasoning mode is not enabled, or if the response does not contain reasoning content.

Proposed approach (optional)

No response

Trade-offs / risks (optional)

Provider-Specific Logic: The solution might require some provider-specific logic if different endpoints implement the reasoning feature differently. The goal should be to find a common pattern or make it configurable.

Maintenance Overhead: As these models and their APIs evolve, the implementation may need updates to remain compatible. However, supporting these popular models is a significant value-add for users.

[roomote]

I'm exploring a potential fix for adding DeepSeek V3.1 Terminus/Turbo model support and enabling reasoning mode for hybrid models through the Chutes.ai provider. I'll open a PR to share progress.

[roomote]

I've made progress on implementing support for DeepSeek V3.1 Terminus/Turbo variants and reasoning mode for hybrid models. Here's what I've completed and where I need clarification:

✅ Completed:

• Added deepseek-ai/DeepSeek-V3.1-Terminus and deepseek-ai/DeepSeek-V3.1-Turbo model variants to the Chutes provider
• Implemented basic reasoning mode detection for DeepSeek V3.1 and GLM-4.5 models
• Added tests for the new model variants

❓ Questions about reasoning mode implementation:

1. Message formatting for non-R1 models: The current implementation uses convertToR1Format for DeepSeek R1 models, but for V3.1 and GLM-4.5 models, I'm unsure about the exact message format needed to trigger reasoning mode. Should these models:

   • Use a specific token/delimiter (like <｜begin▁of▁sentence｜> mentioned in the issue)?
   • Include a special parameter in the API request?
   • Use the same <think> tag parsing approach?

2. API parameters: Does the Chutes.ai API require any special parameters to enable reasoning for these hybrid models (similar to how some providers use a reasoning_effort parameter)?

3. UI integration: For the "Thinking" section to appear in the UI, should the reasoning content be:

   • Parsed from <think> tags in the response?
   • Extracted from a separate field in the API response?
   • Handled differently for each model type?

The implementation is ready in branch feat/chutes-deepseek-v3-1-variants-reasoning, but I want to ensure the reasoning mode works correctly before creating the PR. Could you provide more details on how Chutes.ai expects reasoning to be triggered for these specific models?

[kenOfYugen]

Message formatting for non-R1 models

From V3.1 README.md

Chat Template

The details of our chat template is described in tokenizer_config.json and assets/chat_template.jinja. Here is a brief description.

Non-Thinking

First-Turn

Prefix:
<｜begin▁of▁sentence｜>{system prompt}<｜User｜>{query}<｜Assistant｜></think>

With the given prefix, DeepSeek V3.1 generates responses to queries in non-thinking mode. Unlike DeepSeek V3, it introduces an additional token </think>.

Multi-Turn

Context:
<｜begin▁of▁sentence｜>{system prompt}<｜User｜>{query}<｜Assistant｜></think>{response}<｜end▁of▁sentence｜>...<｜User｜>{query}<｜Assistant｜></think>{response}<｜end▁of▁sentence｜>

Prefix:
<｜User｜>{query}<｜Assistant｜></think>

By concatenating the context and the prefix, we obtain the correct prompt for the query.

Thinking

First-Turn

Prefix:
<｜begin▁of▁sentence｜>{system prompt}<｜User｜>{query}<｜Assistant｜><think>

The prefix of thinking mode is similar to DeepSeek-R1.

Multi-Turn

Context:
<｜begin▁of▁sentence｜>{system prompt}<｜User｜>{query}<｜Assistant｜></think>{response}<｜end▁of▁sentence｜>...<｜User｜>{query}<｜Assistant｜></think>{response}<｜end▁of▁sentence｜>

Prefix:
<｜User｜>{query}<｜Assistant｜><think>

The multi-turn template is the same with non-thinking multi-turn chat template. It means the thinking token in the last turn will be dropped but the </think> is retained in every turn of context.

ToolCall

Toolcall is supported in non-thinking mode. The format is:

<｜begin▁of▁sentence｜>{system prompt}\n\n{tool_description}<｜User｜>{query}<｜Assistant｜></think> where the tool_description is

## Tools
You have access to the following tools:

### {tool_name1}
Description: {description}

Parameters: {json.dumps(parameters)}

IMPORTANT: ALWAYS adhere to this exact format for tool use:
<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>tool_call_name<｜tool▁sep｜>tool_call_arguments<｜tool▁call▁end｜>{additional_tool_calls}<｜tool▁calls▁end｜>

Where:
- `tool_call_name` must be an exact match to one of the available tools
- `tool_call_arguments` must be valid JSON that strictly follows the tool's Parameters Schema
- For multiple tool calls, chain them directly without separators or spaces

Code-Agent

We support various code agent frameworks. Please refer to the above toolcall format to create your own code agents. An example is shown in assets/code_agent_trajectory.html.

Search-Agent

We design a specific format for searching toolcall in thinking mode, to support search agent.

For complex questions that require accessing external or up-to-date information, DeepSeek-V3.1 can leverage a user-provided search tool through a multi-turn tool-calling process.

Please refer to the assets/search_tool_trajectory.html and assets/search_python_tool_trajectory.html for the detailed template.

tokenizer_config.json

{
  "add_bos_token": true,
  "add_eos_token": false,
  "bos_token": {
    "__type": "AddedToken",
    "content": "<｜begin▁of▁sentence｜>",
    "lstrip": false,
    "normalized": true,
    "rstrip": false,
    "single_word": false
  },
  "clean_up_tokenization_spaces": false,
  "eos_token": {
    "__type": "AddedToken",
    "content": "<｜end▁of▁sentence｜>",
    "lstrip": false,
    "normalized": true,
    "rstrip": false,
    "single_word": false
  },
  "legacy": true,
  "model_max_length": 131072,
  "pad_token": {
    "__type": "AddedToken",
    "content": "<｜end▁of▁sentence｜>",
    "lstrip": false,
    "normalized": true,
    "rstrip": false,
    "single_word": false
  },
  "sp_model_kwargs": {},
  "unk_token": null,
  "tokenizer_class": "LlamaTokenizerFast",
  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% if not thinking is defined %}{% set thinking = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, system_prompt='', is_first_sp=true, is_last_user=false) %}{%- for message in messages %}{%- if message['role'] == 'system' %}{%- if ns.is_first_sp %}{% set ns.system_prompt = ns.system_prompt + message['content'] %}{% set ns.is_first_sp = false %}{%- else %}{% set ns.system_prompt = ns.system_prompt + '\n\n' + message['content'] %}{%- endif %}{%- endif %}{%- endfor %}{{ bos_token }}{{ ns.system_prompt }}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{%- set ns.is_first = false -%}{%- set ns.is_last_user = true -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['tool_calls'] is defined and message['tool_calls'] is not none %}{%- if ns.is_last_user %}{{'<｜Assistant｜></think>'}}{%- endif %}{%- set ns.is_last_user = false -%}{%- set ns.is_first = false %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls'] %}{%- if not ns.is_first %}{%- if message['content'] is none %}{{'<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>'+ tool['function']['name'] + '<｜tool▁sep｜>' + tool['function']['arguments'] + '<｜tool▁call▁end｜>'}}{%- else %}{{message['content'] + '<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['function']['name'] + '<｜tool▁sep｜>' + tool['function']['arguments'] + '<｜tool▁call▁end｜>'}}{%- endif %}{%- set ns.is_first = true -%}{%- else %}{{'<｜tool▁call▁begin｜>'+ tool['function']['name'] + '<｜tool▁sep｜>' + tool['function']['arguments'] + '<｜tool▁call▁end｜>'}}{%- endif %}{%- endfor %}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- if message['role'] == 'assistant' and (message['tool_calls'] is not defined or message['tool_calls'] is none) %}{%- if ns.is_last_user %}{{'<｜Assistant｜>'}}{%- if message['prefix'] is defined and message['prefix'] and thinking %}{{'<think>'}}  {%- else %}{{'</think>'}}{%- endif %}{%- endif %}{%- set ns.is_last_user = false -%}{%- if ns.is_tool %}{{message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{%- set content = message['content'] -%}{%- if '</think>' in content %}{%- set content = content.split('</think>', 1)[1] -%}{%- endif %}{{content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_last_user = false -%}{%- set ns.is_tool = true -%}{{'<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endfor -%}{%- if add_generation_prompt and ns.is_last_user and not ns.is_tool %}{{'<｜Assistant｜>'}}{%- if not thinking %}{{'</think>'}}{%- else %}{{'<think>'}}{%- endif %}{% endif %}"
}

assets/chat_template.jinja

{% if not add_generation_prompt is defined %}
  {% set add_generation_prompt = false %}
{% endif %}
{% if not thinking is defined %}
  {% set thinking = false %}
{% endif %}
{% set ns = namespace(is_first=false, is_tool=false, system_prompt='', is_first_sp=true, is_last_user=false) %}
{%- for message in messages %}
  {%- if message['role'] == 'system' %}
    {%- if ns.is_first_sp %}
      {% set ns.system_prompt = ns.system_prompt + message['content'] %}
      {% set ns.is_first_sp = false %}
    {%- else %}
      {% set ns.system_prompt = ns.system_prompt + '\n\n' + message['content'] %}
    {%- endif %}
  {%- endif %}
{%- endfor %}
{{ bos_token }}{{ ns.system_prompt }}
{%- for message in messages %}
  {%- if message['role'] == 'user' %}
    {%- set ns.is_tool = false -%}
    {%- set ns.is_first = false -%}
    {%- set ns.is_last_user = true -%}
    {{'<｜User｜>' + message['content']}}
  {%- endif %}
  {%- if message['role'] == 'assistant' and message['tool_calls'] is defined and message['tool_calls'] is not none %}
    {%- if ns.is_last_user %}
      {{'<｜Assistant｜></think>'}}
    {%- endif %}
    {%- set ns.is_last_user = false -%}
    {%- set ns.is_first = false %}
    {%- set ns.is_tool = false -%}
    {%- for tool in message['tool_calls'] %}
      {%- if not ns.is_first %}
        {%- if message['content'] is none %}
          {{'<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>'+ tool['function']['name'] + '<｜tool▁sep｜>' + tool['function']['arguments'] + '<｜tool▁call▁end｜>'}}
        {%- else %}
          {{message['content'] + '<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['function']['name'] + '<｜tool▁sep｜>' + tool['function']['arguments'] + '<｜tool▁call▁end｜>'}}
        {%- endif %}
        {%- set ns.is_first = true -%}
      {%- else %}
        {{'<｜tool▁call▁begin｜>'+ tool['function']['name'] + '<｜tool▁sep｜>' + tool['function']['arguments'] + '<｜tool▁call▁end｜>'}}
      {%- endif %}
    {%- endfor %}
    {{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}
  {%- endif %}
  {%- if message['role'] == 'assistant' and (message['tool_calls'] is not defined or message['tool_calls'] is none) %}
    {%- if ns.is_last_user %}
      {{'<｜Assistant｜>'}}
      {%- if message['prefix'] is defined and message['prefix'] and thinking %}
        {{'<think>'}}  
      {%- else %}
        {{'</think>'}}
      {%- endif %}
    {%- endif %}
    {%- set ns.is_last_user = false -%}
    {%- if ns.is_tool %}
      {{message['content'] + '<｜end▁of▁sentence｜>'}}
      {%- set ns.is_tool = false -%}
    {%- else %}
      {%- set content = message['content'] -%}
      {%- if '</think>' in content %}
        {%- set content = content.split('</think>', 1)[1] -%}
      {%- endif %}
      {{content + '<｜end▁of▁sentence｜>'}}
    {%- endif %}
  {%- endif %}
  {%- if message['role'] == 'tool' %}
    {%- set ns.is_last_user = false -%}
    {%- set ns.is_tool = true -%}
    {{'<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}
  {%- endif %}
{%- endfor -%}
{%- if add_generation_prompt and ns.is_last_user and not ns.is_tool %}
  {{'<｜Assistant｜>'}}
  {%- if not thinking %}
    {{'</think>'}}
  {%- else %}
    {{'<think>'}}
  {%- endif %}
{% endif %}

When configured as open ai compatible in roo code, I am able to see the 'thinking' when enabling reasoning_effort for non-hybrid, but standard reasoning models.

Perhaps for this hybrid model the reasoning mode may have to be triggered by applying the correct chat template with thinking=True.

From my understanding Chutes.ai is a standard OpenAI compatible provider. I can't comment on whether reasoning_effort is required or not, or reply accurately to the UI integration question. Maybe someone more knowledgeable could chime in?

[kenOfYugen]

Just an update, by using the following:

curl -X POST \

		https://llm.chutes.ai/v1/chat/completions \

		-H "Authorization: Bearer $CHUTES_API_TOKEN" \

	-H "Content-Type: application/json" \

	-d '  {

    "model": "deepseek-ai/DeepSeek-V3.1-Terminus",

    "messages": [

      {

        "role": "user",

        "content": "Tell me a 250 word story."

      }

    ],

    "chat_template_kwargs": {
        "thinking": true
    },

    "stream": true,

    "max_tokens": 1024,

    "temperature": 0.7

  }'

I see the reasoning_content:

data: {"id":"1835e364a38b4ebea84dcdbe6f16b07e","object":"chat.completion.chunk","created":1758638166,"model":"deepseek-ai/DeepSeek-V3.1-Terminus","choices":[{"index":0,"delta":{"role":"assistant","content":"","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"1835e364a38b4ebea84dcdbe6f16b07e","object":"chat.completion.chunk","created":1758638166,"model":"deepseek-ai/DeepSeek-V3.1-Terminus","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"H","tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"1835e364a38b4ebea84dcdbe6f16b07e","object":"chat.completion.chunk","created":1758638166,"model":"deepseek-ai/DeepSeek-V3.1-Terminus","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"mm, the","tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"1835e364a38b4ebea84dcdbe6f16b07e","object":"chat.completion.chunk","created":1758638166,"model":"deepseek-ai/DeepSeek-V3.1-Terminus","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" user wants a","tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"1835e364a38b4ebea84dcdbe6f16b07e","object":"chat.completion.chunk","created":1758638166,"model":"deepseek-ai/DeepSeek-V3.1-Terminus","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" 250-word","tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

...

when executed with:

    "chat_template_kwargs": {
        "thinking": false
    }

I can see the content:

data: {"id":"0a67a23fd268492bae0fe1522908403f","object":"chat.completion.chunk","created":1758638433,"model":"deepseek-ai/DeepSeek-V3.1-Terminus","choices":[{"index":0,"delta":{"role":"assistant","content":"","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"0a67a23fd268492bae0fe1522908403f","object":"chat.completion.chunk","created":1758638433,"model":"deepseek-ai/DeepSeek-V3.1-Terminus","choices":[{"index":0,"delta":{"role":null,"content":"The","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"0a67a23fd268492bae0fe1522908403f","object":"chat.completion.chunk","created":1758638433,"model":"deepseek-ai/DeepSeek-V3.1-Terminus","choices":[{"index":0,"delta":{"role":null,"content":" first time Leo","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"0a67a23fd268492bae0fe1522908403f","object":"chat.completion.chunk","created":1758638433,"model":"deepseek-ai/DeepSeek-V3.1-Terminus","choices":[{"index":0,"delta":{"role":null,"content":" saw the ocean","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"0a67a23fd268492bae0fe1522908403f","object":"chat.completion.chunk","created":1758638433,"model":"deepseek-ai/DeepSeek-V3.1-Terminus","choices":[{"index":0,"delta":{"role":null,"content":", he was","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

...

So the "Enable reasoning" toggle should adjust the boolean:

    "chat_template_kwargs": {
        "thinking": false | true
    }

[yieldsurfer]

Claiming. I've implemented the requested DeepSeek V3.1 variants (Terminus and turbo) with reasoning support via chat_template_kwargs. I also added GLM-4.6-FP8 model and fixed reasoning implementation for all GLM models to properly use the reasoning_content field. All tests passing.