// Open-ended: the server validates, so newer providers work without an SDK release.type BrainProvider = "openai" | "anthropic" | "groq";type AvatarBrain = { provider: BrainProvider | (string & {}); model?: string; useOwnBrain?: boolean; // bill the model to your own vendor key};
type AvatarLanguage = | "en" | "es" | "fr" | "de" | "hi" | "ru" | "pt" | "ja" | "it" | "nl";// "auto" lets the server detect and switch mid-conversation.type AvatarLanguages = "auto" | AvatarLanguage[];
GreetingConfig
tsx
// Note: no Dutch — nine languages, against ten for the conversation.type GreetingLanguage = "en" | "es" | "fr" | "de" | "hi" | "ru" | "pt" | "ja" | "it";type GreetingConfig = { enabled: boolean; message?: string; // omit for the built-in line language?: GreetingLanguage;};
ConnectToneConfig
tsx
type ConnectToneConfig = { enabled: boolean; src?: string; // URL or data URI; omit for the built-in tone. Web only. loop?: boolean; // default true volume?: number; // 0–1, default 0.4 fadeOutMs?: number; // default 350};
The tone is capped at 15 seconds regardless of loop, so a stalled connect
cannot leave it playing forever.
PerceptionConfig
tsx
// Camera is OFF unless enabled, and even then it opens only just-in-time.type PerceptionConfig = { camera?: boolean };type CameraPreviewCorner = "top-left" | "top-right" | "bottom-left" | "bottom-right";
Avatar words carry real timings from the speech alignment. User words do not —
their speech was already said by the time recognition landed.
SessionNotice
A graceful, server-initiated message — a limit reached, a plan stop, a test key
expiring. Not an error: the session is ending deliberately, and the avatar
speaks the message first.
concern is the blame-resolved conclusion to render — scope is the
actionability axis: network is the user's to fix, device is their machine
failing to keep up, service is ours. Neither degraded nor failed is fatal;
the session stays open and the server drives recovery.
AvatarSpeech
A pre-rendered speech job, used by play() in player mode.
The three arrays are parallel — characters[i] starts at
character_start_times_seconds[i] and ends at
character_end_times_seconds[i].
LiveContextArgs
tsx
type LiveContextArgs = { context: string; // non-empty, max 2000 characters update?: boolean; // true replaces the previous context instead of appending};
Errors
AvatarError
tsx
type AvatarError = { // --- Branch on these --- code: AvatarErrorCode; fatal: boolean; // is the session over? retryable: boolean; // would start() plausibly succeed? Only meaningful when fatal. // --- Log these; don't render them --- message: string; // written for you, not your users source: AvatarErrorSource; cause?: unknown;};type AvatarErrorSource = "sdk" | "server" | "network";
Grouped by who fixes it — the only distinction you can act on.
tsx
type AvatarErrorCode = // Your integration — you fix these in your own code. | "SESSION_TOKEN_FETCH_FAILED" | "INVALID_MODE" | "INVALID_INPUT" | "NOT_CONNECTED" // Your user — they must act; retrying alone won't help. | "MIC_PERMISSION_DENIED" // The service — retry, or report to us. | "CONNECTION_FAILED" | "SERVER_ERROR" | "SERVER_CLOSE_REQUESTED" | "AVATAR_LOAD_FAILED" | "BRAIN_ERROR" // Expected lifecycle. | "SESSION_IDLE_TIMEOUT" // Nothing else fit. | "UNKNOWN";
These are deliberately coarse: they split where the remedy splits, and nowhere
else. Every transport failure is one CONNECTION_FAILED — which socket call
failed lives in message and cause, so it never costs you a branch.
play, speakText, speakAudio, and addLiveContext are optional because
they are undefined in unsupported modes. Always use optional chaining:
speakText?.("hello").
AvatarProps
The props <Avatar /> accepts.
tsx
type AvatarProps = { className?: string; style?: React.CSSProperties; loader?: React.ReactNode; errorFallback?: React.ReactNode; // Camera cameraPreviewCorner?: CameraPreviewCorner; hideCameraPreview?: boolean; usePictureInPicture?: boolean; pictureInPictureCorner?: CameraPreviewCorner; // Overlay styling hooks captionClassName?: string; captionStyle?: React.CSSProperties; statusBannerClassName?: string; statusBannerStyle?: React.CSSProperties; thinkingIndicatorClassName?: string; thinkingIndicatorStyle?: React.CSSProperties; // Overrides the server's rive fit ("contain", "cover", …). fit?: string;};
AvatarConnectingLoaderProps
The SDK's own animated loader, exported so you can use it as a loader.
tsx
type AvatarConnectingLoaderProps = { label?: string; // spelled out under the orb, one letter at a time className?: string; style?: React.CSSProperties;};
It sizes itself from a container query, so one node works in a 64px bubble and a
full-bleed stage.
Lifecycle enums
Exported as values, not just types — useful for logging and debugging.