Skip to content

Conversation

caozhiyuan
Copy link
Contributor

This pull request introduces a new /responses API endpoint and integrates it into the server, enabling support for a new Copilot "Responses API" with streaming and non-streaming capabilities. The changes include routing, handler logic, payload translation, and new utility functions to support the endpoint. The request handler for /v1/messages is refactored to conditionally use the new Responses API for supported models, and comprehensive type definitions are added for the new API. Additionally, tests are introduced for the streaming translation logic.

New Responses API Integration

  • Added a new /responses endpoint with routing (responsesRoutes) and a corresponding handler (handleResponses). The handler validates model support, processes requests, and forwards both streaming and non-streaming responses. [1] [2]
  • Integrated the /responses endpoint into the main server routing, including under both /responses and /v1/responses paths. [1] [2]

Conditional API Selection and Translation

  • Refactored the /v1/messages handler to conditionally use the Responses API for supported models, including translation between Anthropic and Responses payloads/results, and streaming event translation. [1] [2] [3]

Type Definitions and Utilities

  • Added comprehensive type definitions for Responses API payloads, results, and streaming types in create-responses.ts, and introduced utility functions for request option parsing and vision/initiator detection. [1] [2]
  • Extended model metadata to include supported endpoints and additional capabilities. [1] [2]

Testing

  • Added unit tests for streaming translation logic, particularly for function call argument streaming in the Responses API.

@caozhiyuan
Copy link
Contributor Author

@ericc-ch help review it?

@eurotrip11
Copy link

@caozhiyuan I tested your feature. Using codex and copilot's API wrap.

I often get these errors:

⚠️ stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 1/5 in 192ms…

⚠️ stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 2/5 in 436ms…

⚠️ stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 3/5 in 879ms…

⚠️ stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 4/5 in 1.752s…

⚠️ stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 5/5 in 3.372s…

■ stream disconnected before completion: Transport error: error decoding response body

@caozhiyuan
Copy link
Contributor Author

@eurotrip11 command line add --verbose to log stream event, check which event is causing the issue . The currently provided openai responses API interface has almost no complex logic, it's just simple forwarding. My premium requests for this month have run out, so I can only test next month. In codex cli, I only tested simple prompts and tool calls, and there were no issues during my testing.

@vtomnet
Copy link

vtomnet commented Sep 28, 2025

@caozhiyuan I tested your feature. Using codex and copilot's API wrap.

I often get these errors:

⚠️ stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 1/5 in 192ms…

⚠️ stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 2/5 in 436ms…

⚠️ stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 3/5 in 879ms…

⚠️ stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 4/5 in 1.752s…

⚠️ stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 5/5 in 3.372s…

■ stream disconnected before completion: Transport error: error decoding response body

I also saw this, alongside 429s and "we're currently experiencing high demand." Sometimes the stream disconnects, other times it doesn't. I think it's a problem on Github's side. See: #118

@caozhiyuan
Copy link
Contributor Author

caozhiyuan commented Sep 28, 2025

@vtomnet @eurotrip11 This seems to be a codex cli error openai/codex#3835 ; responses API return is actually normal. I'm using gpt-5-codex in claude code without any abnormalities.

@caozhiyuan
Copy link
Contributor Author

@vtomnet @eurotrip11 I have added the ping event, which currently resolves the codex disconnection issue: Transport error: error decoding response body. I've tested it and it's working fine now.

@caozhiyuan
Copy link
Contributor Author

when use in claude code, please permissions deny WebSearch Tool , Otherwise, it will meaninglessly consume Premium requests . https://github.com/ericc-ch/copilot-api/pull/115/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R315

@eurotrip11
Copy link

@vtomnet @eurotrip11 I have added the ping event, which currently resolves the codex disconnection issue: Transport error: error decoding response body. I've tested it and it's working fine now.

Now, It works for me. Thank you!

@dylbarne
Copy link

dylbarne commented Oct 1, 2025

when use in claude code, please permissions deny WebSearch Tool , Otherwise, it will meaninglessly consume Premium requests . https://github.com/ericc-ch/copilot-api/pull/115/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R315

I have not thoroughly investigated myself, but perhaps when is calling agentic websearch it is not being correctly classified as tool call around logic with respect to "X-Initiator" header (from my understanding this is partially what dictates whether premium request is used or not #68) - I may have also missed if this is already accounted for in PR changes, but just potentially something to look into if can otherwise be circumvented that way

@caozhiyuan
Copy link
Contributor Author

caozhiyuan commented Oct 1, 2025

@dylbarne The claude code websearch tool will initiate a separate search request with only a llm built-in websearch tool. Currently, the translation code has not been adapted for the llm built-in websearch tool, and the github copilot API does not support the llm built-in websearch tool.

@ericc-ch
Copy link
Owner

ericc-ch commented Oct 4, 2025

Hey @caozhiyuan I really genuinely appreciate that you opened a lot of PRs to contribute to this project
But damn they are big 😆

I will try to get through all of them, try to understand it and review them
Again, thank you! (No promises though sometimes life gets in the way)

@caozhiyuan
Copy link
Contributor Author

@ericc-ch The response API stream event parsing has added specific type constraints and requires certain testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants