Voice Interface

This content is not available in your language yet.

Real-time voice is live and working today — this page covers the underlying pipeline concepts; for the actual component you embed to get voice working, see Avatar & 3D Rendering, which now has full reference documentation for <vai-avatar>’s attributes, methods, and events.

Try it: play.wetel.dev/sdk/demo.html — a real, running voice + 3D avatar session, speech-to-text in, text-to-speech out, barge-in included.

Voice sessions use the same Core API Flow and Events & Subscriptions primitives as text — with AvatarSpeakEvent and InterruptedEvent as the voice-specific additions to the same sessionEvents union every session type uses. <vai-avatar> handles the actual audio pipeline (microphone capture, voice-activity detection, /stt and /tts calls) internally, so most integrators never touch these primitives directly — but if you’re building a fully custom voice UI instead of using <vai-avatar>, this is the mechanism it’s built on:

The client captures microphone audio and sends it to the avatar-service’s /stt REST endpoint (see Avatar Service REST API) for transcription.
The transcribed text is sent as a normal turn via sdkSendMessage, exactly like a text session.
The agent’s reply streams back over sessionEvents as AiResponseEvents; each one is also sent to /tts to synthesize speech audio, which the client plays back.
If the user starts speaking while the agent’s audio is still playing, the client stops local playback immediately and calls interruptSession — the server publishes an InterruptedEvent once it has cancelled the in-flight generation.

A deeper, standalone guide to building a fully custom voice pipeline (independent of <vai-avatar>) — covering voice-activity-detection tuning, audio streaming formats, and latency considerations in more depth than the summary above — is planned. For now, <vai-avatar>’s source is the canonical real reference for every detail not covered here.