Modes

Set mode once in your config:

const config = {
  mode: "call", // "call" | "tts" | "audio" | "player"
  // ...
};

call — Live voice conversation

The default. Bidirectional voice over WebSocket. Your user speaks, the avatar listens, thinks, and responds — all in real time, with native acoustic echo cancellation so the avatar doesn't hear itself.

<AvatarProvider
  config={{
    getSessionToken: async () => fetchSessionToken(),
    mode: "call",
    avatar: {
      avatarId: "default",
      systemPrompt: "Returning customer, has an open ticket about billing.",
    },
  }}
>
  <View style={{ width: 320, height: 320 }}>
    <Avatar />
  </View>
</AvatarProvider>

How it works: the SDK captures the user's microphone and streams it to the server. The server understands what was said, generates a reply, and sends back the spoken audio along with the timing data that drives lip-sync. Echo cancellation is built in, so the avatar doesn't hear its own voice during the call.

Best for: Mobile support agents, AI companions, interactive tutors.

Turn taking

Inside call mode, a second choice: who decides a turn is over.

turnTaking: { mode: "auto" }          // the server's endpointer decides, from the audio
turnTaking: { mode: "push-to-talk" }  // the user decides, by holding a button

Push-to-talk suits noisy or shared spaces, where an endpointer would keep triggering on background speech. It is forgiving at both ends: 200 ms of already-captured audio is released on press, and 250 ms of real audio is still sent after release, so nothing is clipped. A hold longer than two minutes is treated as a stuck control and the turn is closed.

Wire it with startTalking() / stopTalking() from useAvatar, or let the built-in controls render the hold-to-talk button.

Camera

call mode can also let the avatar see, if you opt in with perception: { camera: true }. The camera stays closed until the avatar actually needs to look at something. See Camera perception for what is captured, and Permissions for the native setup.

tts — Text-to-speech

Send text programmatically — the avatar speaks it. No microphone, no voice input.

import { View, Button } from "react-native";

function TTSControls() {
  const { speakText, start, isReady } = useAvatar();

  return (
    <View>
      <Button title="Connect" onPress={start} />
      <Button
        title="Speak"
        onPress={() => speakText?.("Hello! Welcome to our platform.")}
        disabled={!isReady}
      />
    </View>
  );
}

The server processes the text, generates audio, and returns it with alignment data for lip-sync.

Best for: Announcements, notifications, narration, scripted onboarding flows.

audio — Bring your own audio

You already have audio (from another TTS provider or a pre-recorded clip). Send it to the avatar to lip-sync and play.

function AudioControls() {
  const { speakAudio, start, isReady } = useAvatar();

  const playClip = async () => {
    const buffer = await loadAudioBuffer(); // raw PCM audio, 16 kHz mono
    speakAudio?.(buffer);
  };

  return (
    <View>
      <Button title="Connect" onPress={start} />
      <Button title="Play clip" onPress={playClip} disabled={!isReady} />
    </View>
  );
}

Accepted input types: ArrayBuffer, Float32Array, or Blob. The audio must be raw PCM (16 kHz, mono) — if you have a compressed file like MP3 or AAC, decode it first.

Best for: Custom TTS pipelines (ElevenLabs, Play.ai, etc.), pre-recorded content with dynamic delivery.

player — Offline playback

No server, no WebSocket. You provide pre-rendered audio chunks and alignment data — the SDK plays them locally.

const welcomeJob = {
  audioChunks: ["base64-audio-chunk..."],
  alignments: [
    {
      characters: ["H", "e", "l", "l", "o"],
      character_start_times_seconds: [0, 0.08, 0.16, 0.24, 0.32],
      character_end_times_seconds:   [0.08, 0.16, 0.24, 0.32, 0.4],
    },
  ],
};

function PlayerControls() {
  const { play, isReady } = useAvatar();

  return (
    <Button
      title="Play Welcome"
      onPress={() => play?.(welcomeJob)}
      disabled={!isReady}
    />
  );
}

Best for: Onboarding screens, product demos, cached responses, zero-latency playback experiences.

Mode comparison