How We Made a Text-to-Speech Model Respond in Sub-50 ms

Discussion (5 Comments)Read Original on HackerNews

toebee•about 2 hours ago

time-to-first-audio (TTFA) is critical for realtime voice applications. open source implementations (e.g. vLLM-Omni, SGLang-Omni) are often too slow for production and can have issues with realtime playback if you push for lower latency. we wanted to fix that.

we optimized qwen3-tts, a popular OSS TTS model, to achieve 34 ms p95 TTFA at 10 requests per second on 1 x H100. we open source the implementation and benchmark, as well as a breakdown of how it was done.

github: https://github.com/nari-labs/nari-qwen3-tts

kamranjon•about 1 hour ago

Hi there! I actually thought your Dia models were amazing and very natural sounding, I haven’t tried qwen 3 tts yet - has your focus shifted away from building your Dia models and shifted more towards hosting and infrastructure?

narrationbox•23 minutes ago

Haven't read the full report yet, just a quick question. Are your numbers for cold start without pre fill or is it after warmed cache?

bityard•about 1 hour ago

How fast is it on consumer-level hardware?

dominotw•16 minutes ago

chatgpt responds super fast but says filler words like 'hmm..' 'let me think' and responds later with delay

How We Made a Text-to-Speech Model Respond in Sub-50 ms

⚡ Community Insights

Discussion (5 Comments)Read Original on HackerNews