Back to News
Advertisement
aabdik about 3 hours ago 27 commentsRead Article on speko.ai

ZH version is available. Content is displayed in original English for accuracy.

Hi HN! I'm Bek, founder of Speko, a platform that finds an optimal combination of speech-to-text, LLM, and text-to-speech models, given your constraints, among all our public benchmarked options, and tells you why.

Demo: https://www.youtube.com/watch?v=no2LY2gRh-c

Typical production voice agent is an ensemble of three models: STT, an LLM, and TTS.

Each of those layers offers a dozen credible vendors, and each month there are new models on the market. Almost everyone evaluates once, picks a stack of their choice, and never rechecks because switching from a vendor to another involves yet another integration and arguments about the numbers.

The result is that you use voice agents running last quarter's models while better and cheaper options are available.

Before founding Speko, I spent four years as cofounder and CTO building voice agents for enterprises across Asia in 10+ languages. Each time a new speech model would arrive, we repeated the same ritual: hire native-speaking raters, benchmark it against our existing stack, and update production if it improved. Speko turns this process into an API. A team running thousands of calls a day told us: "we can literally go to this dashboard, switch the model, and it will do it for us."

How it works: you send a request with your optimization criteria (accuracy, latency, cost or balanced), language and region. The router filters to models which we measured for the given combination of constraints, benchmarks them, selects the winner, and returns a response with headers containing provider, model names, and the scores. The gateway prefetches signed session plans, so a new session dials the provider straight from memory; no control-plane round trip while a caller waits.

Failover happens only during connection setup stage: if the provider refuses the connection attempt, we start connecting to the runners-up.

Some of the customer stories: one founder came to us not knowing what to pick at all: he gave us his use case and now routes everything through the platform. A property management AI runs LiveKit in Python and had not updated STT or TTS since launch: they did not know their STT had high error rates on their calls, better options existed, and swapping always looked like an R&D project. One team did not know which models to pick for Spanish. A medical team did not know which STT handles medical vocabulary best. In every case we helped find the right stack from the benchmarks, and now they route through us.

The measuring part is public: we pass the same inputs to every model in one region in different dated runs and we publish the boards, including those where our selections perform worse than alternatives. A launch demo answers which 30-second clip sounds better; production asks which model survives minute eight, so we test spontaneous speech, money and dates, ten-minute takes, and the rankings change. We trained an automatic scorer for TTS naturalness on our blind head-to-head listening votes; on providers it has never seen a vote for, it picks the same winner our raters do about as often as raters agree with each other.

We don't train or sell models ourselves, that's precisely how we keep our rankings impartial.

We also open sourced the gateway for teams who want to avoid an extra network hop on the audio path and don't want to share keys with our cloud (https://github.com/SpekoAI/gateway, MIT): one Go binary, which is running as a sidecar in your agent's container, speaks one local protocol over Unix socket, pins provider hosts and attaches your keys. In BYOK mode it doesn't communicate with us at all.

Notice that the anonymous, content-free telemetry is enabled by default, and one env var disables it.

Cost: the gateway and BYOK setup will be free forever, we charge for the hosted router and managed keys with consolidated billing. Since we started the batch in late June, external usage has grown about 25 percent per week on average, front-loaded toward the launch weeks.

I would love feedback from the community: how do you pick speech models now, and what makes you trust the third-party benchmark?

https://speko.ai/

Advertisement

⚡ Community Insights

Discussion Sentiment

85% Positive

Analyzed from 1031 words in the discussion.

Trending Topics

#voice#models#prompt#model#https#speko#end#evals#lot#benchmarks

Discussion (27 Comments)Read Original on HackerNews

cnqso2 minutes ago
What does WER/CER stand for? I see it as some kind of key metric under each model but not clear what it's measuring
maho24 minutes ago
Which model best allows me to transcribe speech that uses a lot of domain-specific terms? For example, when I say "Claude Code", it often gets transcribed as "Cloud Code", and I have to go back and edit or do a second pass with a traditional LLM (which can introduce additional errors).
jeffrwells4 minutes ago
I’ve had a lot of success in the past with fine tuning STT using synthetic data.

I was doing it for Veterinary (ambient recording -> SOAP notes) which has tons of complex domain-specific language AND it is critically important to get right.

“CPR” transcribing as “see pee are” just doesn’t cut it in that industry.

omneity9 minutes ago
Good old Whisper allows you to enter a prompt with domain specific terms and it will use them for transcription.
Tepix40 minutes ago
Ever since OpenAI launched their improved voice mode, I've been looking for a capable LLM with builtin voice-in and -out. Llama 4 was supposed to be it but turned out to be a dud. I haven't followed the topic closely lately, did I miss anything? Are there capable (!) open weights omni models that allow low latency voice chat? If so, what software do you use them with? Can you use a PWA on your phone? WebRTC? WebTransport?
weboabout 1 hour ago
The benchmarks page seems interesting and something I can use to help make an informed decision. Can you talk about how you're measuring some of these? I imagine it needs to involve some human input.

https://benchmarks.speko.ai/turntaking

spmartin823about 2 hours ago
Does this include a turn taking API? It'd be great to have one API that could do "Conversation in a box". One of the biggest annoyances is daisy chaining many models together for turn taking, dumb models for immediate responses, with smarter models returning and taking over after.
cjjuiceabout 1 hour ago
I made a completely free 100% on device translation app https://apps.apple.com/us/app/arda-translate/id6778970560 and hard to image a world where TTS and STT will not be done locally in the future
bewareofscamsabout 1 hour ago
Seems to be useless, the state of art for all categories is local on-device, voice model vendors are just rent seekers for those who know no better.
echelon40 minutes ago
You're not the customer. This is for people building products that support thousands of users.

Linux on desktop is great for you, but this is a tool for people delivering solutions.

bewareofscams38 minutes ago
1) what if I told you I can leverage local models and serve thousands of users?

2) you know nothing about me

3) of course I am not! I do know better

owebmaster24 minutes ago
> You're not the customer. This is for people building products that support thousands of users.

That's their dream. Your dream. The AI dream. Many would say it's AI psychosis.

sparklingabout 1 hour ago
Just canceled my WisprFlow subscription a few days ago to switch to a open source, free, local alternative. In my case Happy.computer with the Cohere model.
alabhyajindalabout 1 hour ago
You mean handy.computer?
MikhailTalabout 2 hours ago
What is the difference with Livekit Gateway? https://livekit.com/blog/introducing-livekit-inference

Or even something more managed like Vapi?

abdikabout 2 hours ago
The main difference from gateway is we help with picking the right voice stack, which seems to be a big problem for users: we benchmark the models continuously and route based on those measurements for your language and constraints, and the boards are public at https://benchmarks.speko.ai/

Second difference is where it runs. Our gateway is open source and runs in your own container, including with self-hosted livekit/pipecat. You get a temporary token before the session starts, and then your orchestration connects directly to the provider.

Vapi is a managed platform: you use their infra to use the voice AI stack. In our case you can have your own infra and switch between models, so you are not locked into a vendor. A lot of teams we talk to build their own infra as they mature, and that is where the router comes handy.

IgorBlink13 minutes ago
any progress with on-device models??
dayvoughabout 2 hours ago
Looks awesome, can't wait to try it for some Filipino workflows when it's available!
abdikabout 1 hour ago
thanks! actually, we have the filipino already, can you check out and share your feedback?
dhruv3006about 2 hours ago
the concept is interesting I must say - good luck !
abdikabout 2 hours ago
Thank you!
Advertisement
narrationboxabout 2 hours ago
> Typical production voice agent is an ensemble of three models: STT, an LLM, and TTS.

To use a claudism, I would like to push back on this. The industry is very much moving towards one-model-does-all end to end trained similar to LLMs and VLMs. Mostly for latency reasons and partially because the results for the end to end trained models are just so much better than those using three pieces architectures.

I think most of the value prop is in automatic evals, not routing specifically. A better pitch for you would be "the LM Arena of voice models" rather than comparing yourself to openrouter because the value add is rather questionable. For TTS specifically, the current SOTA for production systems are all using prompt based voice gen i.e. instead of having 10 different Tacotron models trained on 10 different models, these days it's all a single large model and the "style" is a prompt in the system prompt. The input is usually something like

  <System prompt>
  Speak in a deep smooth voice similar to a documentary narrator
  </System prompt>
  <Text to Narrate>
  Speko is the ultimate evaluation platform for voice agents. We do automatic  evals.
  </Text to Narrate>
It's the same for voice cloning too, you just pass the reference speech as an input file for all generations. A lot of systems don't have any separate style vector extraction step or model-specific fine-tuning anymore.

So something like OpenRouter for voices offer questionable value given that stakeholders usually make this sort of decisions once at the start of the project. On the other hand if you can offer automatic evals and figure out which prompts give the most similar results across different voice providers, that would offer a lot more value. It would be nice to be able to switch from e.g. Grok voice agents to ChatGPT voice agents knowing that the output style won't change too much. There are many companies now with evals as a core business model: LM Arena, Artificial Analysis, Prompt foo (before they got acquired and pivoted to security only) so many take a look at them.

Source: we have been building TTS systems for over a decade too https://narrationbox.com

abdikabout 2 hours ago
Fair pushback. On end to end: we measure those too, same methodology: https://benchmarks.speko.ai/s2s. If the single models win, we route to them the same way, so we do not care which architecture (s2s or cascaded) wins. For now, what we see in production so far is that most teams still want to control each piece: swap the STT for medical vocabulary, keep the LLM, keep the voice.

On "promptfoo of voice models": that is closer to how it started. At my last company we ran these evals manually, we would even hire native-speaking raters, benchmark, switch if it wins. The evals are the value, agreed. The routing is what makes them actionable: teams told us swapping always looked like an R&D project, so scores alone did not change what ran in production.

On prompt-based voice gen and reference-audio cloning: agreed, that is what we see too. It makes continuous measurement more important: the same style prompt behaves differently per language and per content type, so we rank the voices themselves, tagged by use case: https://benchmarks.speko.ai/tts-voices

vdev123about 1 hour ago
totally agree with this
greybabout 2 hours ago
The link, since it seems to be missing?

https://speko.ai/

abdikabout 2 hours ago
Yes, i added it. that's the right link.
greybabout 1 hour ago
Awesome. Thanks for sharing!