Imagine a caller saying, “I would like to check…” and pausing to find an order number. The agent starts answering before the number arrives. Before blaming model speed, check why the system decided that the caller had finished.
Detecting speech, deciding when a turn ends, and allowing a caller to interrupt an agent are separate decisions.
A pause can happen inside a sentence
Voice activity detection, or VAD, identifies speech in audio. The official Silero VAD example returns the start and end timestamps of speech segments. Its repository lists support for 8 kHz and 16 kHz audio. Those timestamps help locate speech; they do not establish that a person has completed a thought.
Someone reading digits, recalling a date, or choosing their words may pause without yielding the turn. LiveKit's turn detector adds signals beyond VAD. Check the input requirements, language coverage, and deployment requirements for the specific detection mode you choose. Capabilities do not automatically carry over between modes.
Endpointing controls the wait before the system commits to ending the user's turn. LiveKit's tuning documentation describes different timing rules: in VAD mode, the wait uses the larger of the silence duration and the minimum endpointing delay; in STT mode, it is added after the speech-to-text provider's end signal. One configuration value cannot tell you the full time until the caller hears a response.
A shorter wait can feel more responsive and can also cut off a mid-sentence pause. A longer wait can leave an awkward gap. Dynamic endpointing can adjust using pause statistics from the conversation, but recordings from the intended workload still need to be checked.
Interrupting the agent is a separate decision
An agent beginning its response after a caller finishes is different from a caller saying “hold on” while the agent is already speaking. The second situation needs interruption handling.
LiveKit distinguishes VAD-based interruptions from adaptive interruptions. The adaptive approach uses an audio model to distinguish an interruption from a brief acknowledgment. A short “mm-hmm” may not mean “stop,” while an explicit interruption should not leave the old answer playing.
Preemptive generation is another option. It starts model generation, and potentially speech synthesis, before the user's turn is finally committed. It may reduce waiting, but canceled work consumes compute. Include discarded work in the evaluation rather than looking only at a fast response.
Test against the calls you actually expect
For a Mandarin telephone deployment, use a non-production project and check three things: whether pauses between digits get cut off, whether brief acknowledgments stop playback by mistake, and whether an explicit interruption stops the old response promptly.
Include Mandarin speech, accents that occur in the intended business, and background noise. Keep speech recognition, the language model, and speech synthesis fixed while changing one group of turn-taking settings at a time. Record premature responses, false interruptions, and the time from the caller finishing to audible playback. A FreeSWITCH integration needs measurements through its actual media path; latency figures from a different system are not a substitute.
Match the incoming audio sample rate to the detector's requirements. A component supporting a sample rate does not establish that the complete pipeline is configured correctly. Also confirm which components run locally, which depend on cloud services, and whether the target network can reach them.
Once you know which decision fired too early, you can choose whether to adjust the wait, change detection, or optimize generation further downstream. These are proposed checks, not findings from a test of your deployment.
Sources: LiveKit turns overview, tuning, turn detector, and Silero VAD.