What is the value of Ultralvox?

[Gpwner]

I tested the latency of Ultravox and compared it with the latency of Whisper Large v3 + Llama 3.1 8B Instruct, and it feels like both are similar.
Apart from the fact that the model's intelligence does not decline, I'm curious to know what other motivations you have for doing this work. I've been thinking about it for a long time but couldn't figure it out, and I look forward to hearing your answer.

[Gpwner]

@zqhuang211 @farzadab @zkoch @eltociear Can anyone help me figure it out? thank you so much.

[zqhuang211]

Latency is one of the motivations for developing end-to-end models and solutions, though it is becoming less significant over time. Transcribing speech into text not only introduces errors but also results in the loss of important information about speech that cannot be captured in transcripts. We believe that achieving a comprehensive understanding and generation of speech/audio for human-level communication requires novel modeling that moves beyond the traditional ASR+LLM+TTS pipeline.