[Bug]: Stream TTS via proxy by deferring OpenAI streaming context; add verify script

[mfittko]

Fixes #14891

Summary

• Problem: /v1/audio/speech (OpenAI TTS) via the proxy buffered or yielded zero bytes, preventing progressive playback.
• Root cause: OpenAI TTS adapter returned an object tied to a closed streaming context; proxy generator iterated the stream incorrectly.
• Fix: defer opening the OpenAI streaming context until aiter_bytes() consumption and iterate the async iterator correctly. Keep sync speech() unchanged. Add a small script to verify streaming behavior.

Changes

• OpenAI provider (async TTS): return a deferred streaming object whose aiter_bytes() opens with_streaming_response.create(...) and yields chunks.
• Proxy generator: ensure we iterate bytes correctly from the returned stream.
• Scripts: add scripts/verify_tts_streaming.py to validate headers, time-to-first-byte, and bytes received.
• Tests: add tests/test_litellm/test_tts_deferred_streaming.py to minimally validate deferred streaming yields bytes (skipped if async plugin missing).

Checklist

• I have added at least one test under tests/litellm/
• I have attached/provided local test results
• My PR scope is isolated and solves one problem (TTS streaming)

Notes

• For browser clients, use response.body streaming (ReadableStream) instead of arrayBuffer()/blob() for progressive TTS playback.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
litellm	Error			Sep 26, 2025 10:30am

[CLAassistant]

All committers have signed the CLA.

[krrishdholakia]

move to test_litellm/

[mfittko]

@krrishdholakia will do. No idea why builds fail, the tests run for me locally.

[mfittko]

Do we need to update the documentation? I don't really think so since, the patch only enables what is already possible using the OpenAI endpoint directly.

[mfittko]

There's still some issues regarding spend logging in the streaming context. Will push a fix.