Session brief
Voice Agent Engineering — Nik Caryotakis, SuperDial
Overview
This talk explores the engineering challenges and best practices for building reliable voice AI applications, particularly in production environments. It emphasizes the shift from prescriptive to descriptive development in voice user interfaces and highlights the importance of focusing on conversational content and vertical integrations over superficial realism. The speaker advocates for a pragmatic approach, prioritizing reliability and functionality, especially in sensitive domains like healthcare administration.
Who should watch
- AI Engineers
- Product Managers
- Builders working with voice AI
- Those developing conversational agents
- Engineers dealing with real-time audio processing and LLM integration
Key takeaways
- Voice AI in 2025 benefits from faster, more affordable LLMs, but requires specific techniques to transition from chat to voice agents.
- Production voice AI applications often need to prioritize reliability over hyper-realism, as cutting-edge speech-to-speech models may not yet be production-ready.
- The role of a voice AI engineer involves handling multimodal data, real-time latency, asynchronous programming, and the unique constraints of voice conversations.
- Key sayings for building voice AI include "say the right thing at the right time" and "build this plane while we fly it."
- Open-source frameworks like Pipcat can be valuable for voice AI orchestration, especially for self-hosting and scaling.
- Controlling LLM endpoints, managing latency-sensitive generative responses, and ensuring HIPAA compliance are crucial infrastructure decisions.
- Addressing last-mile problems in text-to-speech, such as pronunciation and spelling, is vital for a good user experience.
- Focusing on conversational content, design, and vertical integrations is what truly differentiates voice agents, not just their voice or realism.
Notable quotes
*A boring call for us is an excellent call because it turns out a lot of work is boring.*
*Ultimately what is going to make your voice bot unique isn't its Voice or its Interruption handling or how realistic it sounds or how it does turn taking ultimately it's going to be in the conversational content and the design there and the vertical Integrations around it that make your agents work actually valuable.*
*Laser focus on the last mile because this is where ultimately you can provide a lot of value and put your agents to work.*
Unofficial community note. Prefer the recording for nuance.