Tsung-Hsien (Shawn) Wen, CTO of PolyAI, tells Tim Scarfe why voice agents are harder than text agents. Voice adds time, and a good conversation depends on adapting to the person on the line, not just on reasoning to the best answer. Shawn describes an audio-native model (Dialog-RSN-1) that first predicts a turn-taking signal, then replies in text with citations, and writes the transcript last so enterprises can audit it.Along the way: training on real, noisy calls with synthetic noise added, and why over-cleaned audio made the new model worse. Latency, and what a voice agent should do while it thinks. Why a voice with a hint of regional accent beats a generic one. Why public benchmarks fall short for voice, why enterprises want to own their agent harness, and whether behaviour belongs in the harness or in the weights.The last stretch is about working with agents: cognitive debt, the shift from producing content to checking it, Wispr Flow, building tools that agents can use, and whether slop is in the eye of the reader.This episode was produced in partnership with PolyAI.https://poly.ai CHAPTERS Why voice agents are harder than text What enterprises want, and why PolyAI built its own model How an audio-native voice model works Training data, spectrograms and synthetic noise The cocktail party problem and the future of turn-taking Latency, adaptive reasoning and keeping callers' trust Voices, personality and the uncanny valley How do you benchmark a voice agent? Harness engineering and owning the intelligence Well-specified problems and auditable agents Weight adaptation and cognitive debt Agents at work: Wispr Flow, voice and tool building The next decade of voice, and what counts as slopREFERENCESThe Bitter Lesson: http://www.incompleteideas.net/IncIdeas/BitterLesson.html []Retrieval-augmented generation: https://arxiv.org/abs/2005.11401 []Mel scale: https://en.wikipedia.org/wiki/Mel_scale []Victor Zue: https://en.wikipedia.org/wiki/Victor_Zue []Cocktail party effect: https://en.wikipedia.org/wiki/Cocktail_party_effect []Speaker diarisation: https://en.wikipedia.org/wiki/Speaker_diarisation []
