Avatar & 3D Rendering

<vai-avatar> is Wetel’s embeddable avatar web component: a single custom element that renders a lip-synced 3D avatar, streams the agent’s replies through it via text-to-speech, and (optionally) listens for the user’s spoken replies with voice-activity detection. It is plain JavaScript with no framework dependency — drop the script tag into any page, set a handful of attributes from your sdkStart response, and the element handles the WebSocket subscription, avatar rendering, TTS playback, and STT round trip internally.

This page documents the actual public surface of the component: its HTML attributes, its JS methods, and the custom events it dispatches. If you only need text captions with no 3D rendering at all, you don’t need this component — sessionEvents alone (see Events & Subscriptions) is enough to drive a plain chat UI.

When to use this vs. building your own UI

Wetel is headless by design (see Core API Flow) — you’re never required to use <vai-avatar>. Use it when you want the 3D avatar rendering and its built-in TTS/STT loop without writing that integration yourself. If you want the audio pipeline (voice activity detection, speech-to-text, text-to-speech) without the 3D rendering — or you want to understand what’s happening inside this component’s audio handling in more depth — see Voice Interface, which covers that pipeline independent of any particular UI component.

Avatar rendering options

By default, the agent appears as a 3D avatar rendered by this component (<vai-avatar>). The embedded SDK includes Three.js and the TalkingHead 3D library, so all rendering happens client-side with no additional backend calls needed beyond the session and TTS endpoints.

As an alternative, agents can be configured to render a hosted photorealistic video avatar instead of the default 3D model. This is controlled at the agent level via the avatarBackend configuration (see Agents: avatarBackend). When avatarBackend: HOSTED_API is selected, video streams to your page instead of 3D rendering, providing a more realistic visual experience. This hosted option is currently in limited pilot availability.

HOSTED_API is a rendering tier, not one fixed vendor — which underlying provider actually serves a given agent’s video is a server-side configuration choice, not something your integration code needs to know or branch on. Both are driven the same way: your agent’s own TTS audio is streamed to the provider purely for lip-sync — the provider never runs its own STT/LLM/TTS, so the conversation logic is 100% yours regardless of which one is active.

• Photorealistic, pixel-diffusion rendering (e.g. Anam) — higher visual fidelity, higher latency and compute cost, best when realism matters more than speed.
• Lightweight, mesh-deformation rendering (e.g. Simli) — lower latency and cost, a good fit for high-volume or latency-sensitive use cases where a slightly more stylized look is an acceptable trade.

Ask your Wetel contact which provider is active for your agent and which is the better fit if you’re deciding between them for a new one.

The embedding and integration remain identical regardless of which rendering backend — or which vendor behind HOSTED_API — is selected. The component’s attributes, events, and JavaScript API work the same way in every case.

Installation

Include the script on any page. It has no build step and pulls its 3D rendering dependencies (Three.js, the TalkingHead library) from a CDN via an import map, injected automatically the first time the script runs:

<script
  type="module"
  src="https://your-avatar-service-host/sdk/vai-avatar.js"
></script>

vai-avatar.js is served by apps/avatar-service, the same service that exposes the /tts and /stt REST endpoints documented in Avatar Service REST API. Ask your Wetel contact for the correct URL for your environment.

Never remove and re-inject this <script> tag

vai-avatar.js declares its module-level state with top-level consts. Those bindings persist in the browser’s global scope for the lifetime of the page — removing the <script> element from the DOM does not unload or reset them. If your app (an SPA re-mounting the embed on route change, or any custom loader that removes and re-adds the tag) injects this script a second time in the same page session, the second injection throws SyntaxError: Identifier '...' has already been declared and the avatar fails to (re-)initialize.

A single static <script> tag, present once for the life of the page, is unaffected — this only bites integrations with their own script-injection/removal logic. If your integration mounts and unmounts the <vai-avatar> element itself (e.g. as part of a component lifecycle), keep the <script src="...vai-avatar.js"> tag itself in the DOM permanently and only add/remove the <vai-avatar> element — never remove and re-add the script tag.

Attributes

Every attribute maps directly onto a field from the sdkStart mutation’s response (see Core API Flow for the full mutation). The mapping is deliberate — you should be able to spread the sdkStart response straight onto the element with no manual translation:

Attribute	GraphQL field	Type	Required	Default if omitted
session-id	sessionId	number	Yes	—
token	avatarToken	string	Yes	—
graphql-endpoint	graphqlEndpoint	string	Yes	—
avatar-service	avatarServiceUrl	string	Yes	—
tts-voice	ttsVoice	string	No	en-US-Standard-C
tts-lang	ttsLang	string	No	en-US
lipsync-lang	lipsyncLang	string	No	en
speaking-rate	speakingRate	number	No	1.0
avatar-glb	avatarGlb	string	No	brunette-t.glb
avatar-gender	avatarGender	string	No	F
vad-threshold	vadThreshold	number	No	0.8

The four required attributes (session-id, token, graphql-endpoint, avatar-service) gate connection entirely — the element does nothing until all four are present. token is the session’s short-lived avatarToken, the same credential used for sdkSendMessage/sdkEndSession calls documented in Core API Flow — never a long-lived API key.

vad-threshold controls the Silero VAD (voice activity detection) model’s positive-speech-detection sensitivity, on a 0.1–1.0 scale. Lower values make the microphone more sensitive (may trigger on background noise); higher values require louder/clearer speech before a segment is treated as real.

All attributes are reactive — updating any one of them (e.g. swapping avatar-glb to change the model, or setting a fresh session-id/token pair to move the element to a different session) triggers the element to tear down its current connection and reconnect with the new values. Rapid attribute changes are batched into a single reconnect, so you can set several attributes in a row without triggering redundant connections.

JavaScript API

These methods are available on the element instance, but are only safe to call after the vai-ready event has fired — before that, the 3D avatar hasn’t finished loading and calls are silently ignored.

const avatar = document.querySelector("vai-avatar");
avatar.addEventListener("vai-ready", () => {
  // avatar.speak(), avatar.sendAudio(), avatar.setVoiceActive() are now safe to call
});

speak(text)

Triggers avatar speech programmatically — the text is sent straight to the underlying TalkingHead instance’s TTS/lipsync pipeline, bypassing any GraphQL round trip. Useful for a scripted opening line the host page wants to inject client-side, independent of whatever the backend’s own opening-greeting logic does.

avatar.speak("Hi! I can help you find a book — what are you looking for?");

sendAudio(blob)

Sends a recorded audio Blob to the avatar-service’s /stt endpoint for transcription, then automatically forwards the resulting transcript to the backend via sdkSendMessage — the same as if the user had spoken and the built-in VAD loop had picked it up. Use this if you’re driving audio capture yourself (e.g. a custom push-to-talk button using MediaRecorder) instead of relying on the component’s own always-listening VAD.

const recorder = new MediaRecorder(stream);
const chunks = [];
recorder.ondataavailable = e => chunks.push(e.data);
recorder.onstop = () => {
  const blob = new Blob(chunks, { type: "audio/webm" });
  avatar.sendAudio(blob);
};

sendText(text)

Sends a typed message as if the user had spoken it — forwards straight to sdkSendMessage and the reply arrives on the same subscription (and is spoken by the avatar) as any voice-driven turn. Use this to offer a text input alongside voice, without building a second connection or subscription.

textInput.addEventListener("keydown", e => {
  if (e.key === "Enter" && textInput.value.trim()) {
    avatar.sendText(textInput.value.trim());
    textInput.value = "";
  }
});

handleBargeIn()

Manually interrupts the avatar’s in-progress reply right now — the same effect as a confirmed voice barge-in (see Barge-in and interruptions below), but triggered directly rather than through the VAD/setVoiceActive gate. Works identically for both the default CLIENT_3D avatar and the pilot HOSTED_API backend. Use this for an explicit “stop” button in your UI rather than relying on voice detection to catch an interruption.

stopButton.addEventListener("click", () => avatar.handleBargeIn());

setVoiceActive(active)

Arms or disarms the push-to-talk barge-in gate. This is the mechanism for letting a user interrupt the avatar mid-speech — see Barge-in and interruptions below for why this is push-to-talk rather than always-on.

Call setVoiceActive(true) right before the user is expected to interrupt (e.g. on mousedown/touchstart of a push-to-talk button), and the next sustained speech detected while the avatar is talking is treated as a real barge-in. The gate auto-disarms itself after it fires once, or after a short timeout if the user never actually speaks — so you don’t strictly need to call setVoiceActive(false) yourself, though doing so on button release is good practice for a responsive UI.

pushToTalkButton.addEventListener("mousedown", () =>
  avatar.setVoiceActive(true)
);
pushToTalkButton.addEventListener("mouseup", () =>
  avatar.setVoiceActive(false)
);

setVoiceActive has no effect on normal turn-taking — when the avatar is silent and simply listening for the user’s next reply, that path is always active regardless of this gate. It only matters for interrupting the avatar while it’s speaking.

Events

All events are dispatched on the element itself with bubbles: true, so a listener on any ancestor element works too — you don’t need to attach listeners to the <vai-avatar> element directly.

Event	detail payload	Fires when
vai-ready	none	The avatar model has finished loading and the session’s WebSocket connection is established. The JS API methods above are safe to call from this point on.
vai-response	{ text }	The agent has something to say. This is the primary event for rendering captions or a transcript — it fires as soon as reply text is available, even before the 3D avatar has finished speaking it aloud.
vai-ended	{ sessionId, durationSeconds }	The session was ended server-side (e.g. via sdkEndSession, or the agent’s own closing logic) — not the same as the host page tearing down the element itself.
vai-error	{ code, message }	Something failed — a connection error, a failed STT/send request, etc. code is a short machine-readable string (e.g. send-failed, ws-error) for programmatic handling; message is human-readable.
vai-voice-active-changed	{ active }	The push-to-talk gate (set via setVoiceActive) toggled on or off — including the automatic disarm after it fires or times out. Useful for keeping a mic-icon UI in sync with the actual armed state rather than tracking it independently.

avatar.addEventListener("vai-response", e => {
  transcriptEl.textContent += e.detail.text;
});

avatar.addEventListener("vai-error", e => {
  console.error(`[avatar] ${e.detail.code}: ${e.detail.message}`);
});

avatar.addEventListener("vai-voice-active-changed", e => {
  micButton.classList.toggle("armed", e.detail.active);
});

Barge-in and interruptions

The component supports letting a user interrupt the avatar mid-sentence, but deliberately does not do this automatically by default. Without a headset, a microphone reliably picks up the avatar’s own speaker output as if it were the user talking — auto-detecting a barge-in from raw voice activity alone would trigger constantly on echo, not on genuine interruptions. Instead, barge-in is gated behind the explicit setVoiceActive(true) call described above: only speech detected after the host page has signaled real user intent to interrupt (e.g. a push-to-talk button press) is treated as a real barge-in candidate.

You can also trigger an interrupt directly via handleBargeIn() (see above) instead of going through the VAD/setVoiceActive gate — useful for an explicit “stop” button. Barge-in works the same way for both the default CLIENT_3D avatar and the pilot HOSTED_API backend (see Avatar rendering options above); the component picks the correct stop mechanism for whichever backend is active.

When a barge-in is confirmed, two things happen, and in this order:

1. Local audio playback stops immediately, synchronously, client-side — the avatar’s in-progress TTS/lipsync is cleared right away, with no network round trip in the critical path. The user sees/hears the avatar stop the instant the interrupt is locally confirmed.
2. The component separately calls the interruptSession GraphQL mutation, authenticated with the same avatarToken the element already holds, to tell the backend to cancel the in-flight LLM stream / stop the workflow from advancing further. This call is fire-and-forget from the client’s perspective — the local stop already happened regardless of how long this takes or whether it succeeds.

The backend also emits its own InterruptedEvent over the sessionEvents subscription once it has processed the interrupt server-side; the component handles that event idempotently as a confirmation, but does not wait for it before stopping local playback.

Worked example: full session with a push-to-talk avatar

This example starts a session, embeds <vai-avatar> wired from the sdkStart response, renders captions from vai-response, and adds a push-to-talk button using setVoiceActive.

1. Start the session (backend) — see Core API Flow for the full mutation reference. SdkStartResult returns every field the element’s attributes need:

mutation StartSession {
  sdkStart(input: { agentId: 3, clientId: "end-user-abc123" }) {
    sessionId
    avatarToken
    graphqlEndpoint
    avatarServiceUrl
    avatarGender
    avatarGlb
    lipsyncLang
    speakingRate
    ttsLang
    ttsVoice
    vadThreshold
  }
}

Hand the full response object to your frontend — every field maps to an attribute below.

2. Embed and wire up the element (frontend):

<script
  type="module"
  src="https://your-avatar-service-host/sdk/vai-avatar.js"
></script>

<vai-avatar id="avatar" style="width: 480px; height: 640px;"></vai-avatar>
<div id="transcript"></div>
<button id="ptt">Hold to talk</button>

<script type="module">
  async function startSession(sdkStartResult) {
    const avatar = document.getElementById("avatar");

    // Map the sdkStart response fields directly onto the element's attributes.
    avatar.setAttribute("session-id", String(sdkStartResult.sessionId));
    avatar.setAttribute("token", sdkStartResult.avatarToken);
    avatar.setAttribute("graphql-endpoint", sdkStartResult.graphqlEndpoint);
    avatar.setAttribute("avatar-service", sdkStartResult.avatarServiceUrl);
    avatar.setAttribute("tts-voice", sdkStartResult.ttsVoice);
    avatar.setAttribute("tts-lang", sdkStartResult.ttsLang);
    avatar.setAttribute("lipsync-lang", sdkStartResult.lipsyncLang);
    avatar.setAttribute("speaking-rate", String(sdkStartResult.speakingRate));
    avatar.setAttribute("avatar-glb", sdkStartResult.avatarGlb);
    avatar.setAttribute("avatar-gender", sdkStartResult.avatarGender);
    avatar.setAttribute("vad-threshold", String(sdkStartResult.vadThreshold));

    const transcript = document.getElementById("transcript");
    const pttButton = document.getElementById("ptt");

    avatar.addEventListener("vai-ready", () => {
      console.log("avatar loaded, session connected");
    });

    avatar.addEventListener("vai-response", e => {
      transcript.textContent += e.detail.text;
    });

    avatar.addEventListener("vai-error", e => {
      console.error(`[avatar] ${e.detail.code}: ${e.detail.message}`);
    });

    avatar.addEventListener("vai-voice-active-changed", e => {
      pttButton.classList.toggle("armed", e.detail.active);
    });

    avatar.addEventListener("vai-ended", e => {
      console.log("session ended after", e.detail.durationSeconds, "seconds");
    });

    // Push-to-talk: arm the barge-in gate while the button is held.
    pttButton.addEventListener("mousedown", () => avatar.setVoiceActive(true));
    pttButton.addEventListener("mouseup", () => avatar.setVoiceActive(false));
  }

  // sdkStartResult would come from your backend, which called the
  // sdkStart mutation above with your long-lived API key.
  const sdkStartResult = await fetch("/api/start-avatar-session", {
    method: "POST",
  }).then(r => r.json());

  startSession(sdkStartResult);
</script>

From here, the element handles everything else itself: subscribing to sessionEvents, sending user speech via sdkSendMessage once VAD detects and transcribes it, and driving the avatar’s lipsync from the agent’s replies. You never need to call sdkSendMessage or manage the subscription yourself when using <vai-avatar> — that’s the point of the component. If you need finer control over sending messages (e.g. injecting a message from a non-voice UI element alongside the avatar), you can still call sdkSendMessage directly per Core API Flow; the reply will still arrive at the avatar over its own subscription and be spoken normally.

Next steps

• Core API Flow — the sdkStart/sdkSendMessage/sdkEndSession mutations this component wraps.
• Events & Subscriptions — the full sessionEvents event catalogue, including InterruptedEvent and SessionEndedEvent, which the component consumes internally.
• Avatar Service REST API — the /tts and /stt endpoints this component calls under the hood, if you need to call them directly for a custom integration.
• Voice Interface — a deeper look at the underlying audio pipeline (VAD, STT, TTS) independent of this component’s 3D rendering.
• Agents — where avatarGender, avatarGlb, ttsVoice, and speakingRate are configured server-side as agent defaults, which sdkStart then returns for you to map onto this component.