View as Markdown Open in ChatGPT Open in Claude

Avatar actions

Let the avatar do things in your app — book a demo, look up an order — with every change read back and confirmed by the user first.

An action is something your app can do that the avatar may offer during a call. You declare it with a name, a description and the details it needs; the avatar collects those details in conversation, reads them back, and your handler runs only once the user says yes.

tsx

"use client";
import { useAvatarAction } from "@avatarfactory/react";

function DemoBooking() {
  useAvatarAction({
    name: "book_demo",
    description: "Book a 30-minute demo call with our sales team.",
    params: {
      email: { type: "email", description: "Where to send the invite" },
      slot: { type: "datetime", description: "Start time" },
    },
    handler: async ({ email, slot }) => {
      const response = await fetch("/api/demo", {
        method: "POST",
        body: JSON.stringify({ email, slot }),
      });
      return response.ok
        ? { ok: true, result: `Booked for ${slot}. The invite is on its way to ${email}.` }
        : { ok: false, error: "that slot was just taken" };
    },
  });

  return null;
}

Handlers never leave the device. Only name, description and params are sent to the server. The server decides whether an action runs; your handler decides how.

Before you start

Actions need two things, and without either one nothing reaches the avatar:

Requirement	Where it is set
Call mode	mode: "call" in your config. Only a call has a turn to confirm in, so registering in any other mode reports a non-fatal INVALID_MODE error.
The Site actions grant	Site actions in the avatar's Tool calling panel on the Platform. Off by default. While it is off the SDK declares nothing and logs one warning per session.

Use actions when the avatar should change something or fetch something your app owns. To tell the avatar what the user is doing, use notify() or addLiveContext() instead.

triggerAction and availableActions on useAvatar() are rig gestures (a wave, a nod), not these actions. See Gestures.

How an action runs

Every action of the default kind goes through the same steps, enforced by the server:

Stage	What happens
`collecting`	The avatar asks for each detail it still needs, by voice. The user can also type a value into your UI.
`review`	The avatar reads back the exact details it will use and waits for a yes.
`running`	The user agreed — by voice on a later turn, or by tapping Confirm in your UI. Your handler runs with those exact details.
`done`	The handler returned. The avatar tells the user how it went, in the words your handler gave it.
`cancelled`	The user backed out before the run.
`expired`	No yes arrived within 3 user turns or 2 minutes, or a different action started and replaced this one.

Because the user has already agreed to the exact details when the handler runs, your handler never needs to ask "are you sure?" There is no option to skip the read-back: confirmation is server policy, not a flag in the SDK.

Registering an action

useAvatarAction(definition)

Registers on mount and unregisters on unmount, so an action exists only while the component that can perform it is on screen. Call it inside any component under <AvatarProvider>.

tsx

useAvatarAction({
  name: "track_order",
  description: "Look up where an order is.",
  kind: "lookup",
  params: {
    orderNumber: { type: "string", description: "The order number, e.g. A-1042" },
  },
  handler: async ({ orderNumber }) => {
    const status = await fetchOrderStatus(String(orderNumber));
    return { ok: true, result: `Order ${orderNumber} is ${status}.` };
  },
});

A re-render with a new handler closure is picked up without re-declaring it. Only a change to the schema — name, description, params, kind, ui, or whether there is a check — sends a new declaration.

registerAction(definition)

The imperative form, from useAvatar(). It returns the unregister function.

tsx

const { registerAction } = useAvatar();

useEffect(() => {
  if (!registerAction) return;
  return registerAction({
    name: "start_trial",
    description: "Start a 14-day free trial on the user's account.",
    handler: async () => {
      await startTrial();
      return { ok: true, result: "The trial is active." };
    },
  });
}, [registerAction]);

Register before or during a call. Registrations made before the session starts are held and declared once it does, and they survive reconnects.
A name has one owner. Registering the same name again replaces the first registration, and the first one's unregister then does nothing.
A refused definition never throws. It is reported as a non-fatal INVALID_INPUT error naming the problem, and gets an unregister that does nothing.
Changes are batched. Several registrations in one render (a route change) are sent as one update, and an unchanged set is not sent at all.

ActionDefinition

Prop	Type	Default	Description
`name`Required	`string`	No default	Matches ^[a-z][a-z0-9_]{0,63}$ — lowercase, digits and underscores. It becomes a tool name for the model, so it cannot contain dots (unlike an app-event slug). confirm_action and cancel_action are reserved.
`description`Required	`string`	No default	What it does, written for the model. Max 300 characters. State the order the avatar should ask for details in, if it matters.
`params`Optional	`Record<string, ActionParam>`	No default	The details the action needs. At most 8. Param names match ^[a-zA-Z][a-zA-Z0-9_]{0,39}$.
`kind`Optional	`"action" \| "lookup"`	`"action"`	"action" is read back and confirmed before it runs. "lookup" only finds something out and runs at once. See Lookups below.
`ui`Optional	`ActionUi`	No default	What your own UI shows for this action, so the avatar can mention it. See Your own UI below.
`handler`Required	`(args, context) => ActionResult \| Promise<ActionResult>`	No default	Runs only after the user confirmed these exact args (a lookup: at once). A throw is reported as a failure.
`check`Optional	`(param, value, args) => ActionCheckResult \| Promise<ActionCheckResult>`	No default	Vets the value of any param marked check: true before the read-back. Required if any param sets check: true.

Params

Prop	Type	Default	Description
`type`Required	`ActionParamType`	No default	The value's shape. See the table below.
`description`Optional	`string`	No default	Written for the model. Max 120 characters.
`required`Optional	`boolean`	`true`	Set false for an optional detail. The handler receives only the params that have a value.
`values`Optional	`string[]`	No default	enum only, and required for it: 1–20 non-empty strings.
`check`Optional	`boolean`	No default	Vet each value with the action's check function before it is read back. Not allowed on a lookup.

Param types

Type	Your handler receives
`string`	A string.
`email`	A string. The server normalizes a spoken address before it reaches you.
`phone`	A string.
`date`	A string, YYYY-MM-DD.
`datetime`	A string, YYYY-MM-DDTHH:mm:ss.
`number`	A finite number.
`boolean`	true or false.
`enum`	One of the strings in values.

date and datetime are the user's local time, with no zone. The SDK sends the device's time zone when the call starts, so "Thursday at 2" means the user's Thursday. new Date(args.slot) already reads the value as local time.

The handler

tsx

type ActionContext = {
  callId: string;
  signal: AbortSignal; // aborts when the client gives up on the run
};

type ActionResult =
  | { ok: true; result?: string }
  | { ok: false; error: string };

result and error are what the avatar is told happened, so write them as plain words — the avatar turns them into its reply. Both are capped at 1000 characters.

Before your handler runs, the SDK re-checks the args against the current declaration. Every run ends in exactly one result sent back to the server, even when something goes wrong:

What happened	What the avatar is told
The handler threw, or returned something without ok	the action failed
The handler took longer than 12 seconds	it took too long — and context.signal aborts
The action was unregistered before the run	that action is not available right now
The args no longer match the declaration	the details did not match what that action needs
ok: false with an empty error	the action failed

A callId runs at most once. A replayed run would be a second booking, so the SDK ignores a duplicate.

Pass context.signal to your fetch calls. If the run times out, the request is cancelled instead of completing after the avatar has already said it failed.

Lookups

kind: "lookup" is for finding something out: open slots, an order's status, a price. It runs at once, with no read-back and no yes.

A lookup must change nothing. The server cannot tell what your handler does, so it trusts the kind you declare. Anything that books, buys, sends or saves is an "action".

A lookup cannot use check or ui.review.

Checking a value before the read-back

Mark a param check: true and give the action a check function to vet each value as the user gives it — a slot that is still free, an email domain you accept. A failed check drops the value, and the avatar offers your suggestions instead.

tsx

useAvatarAction({
  name: "book_demo",
  description: "Book a 30-minute demo call. Ask for the time first, then the email.",
  params: {
    slot: { type: "datetime", description: "Start time", check: true },
    email: { type: "email" },
  },
  check: async (param, value) => {
    if (param !== "slot") return { ok: true };
    const openSlots = await fetchOpenSlots(String(value));
    return openSlots.includes(String(value))
      ? { ok: true }
      : { ok: false, reason: "that time is booked", suggestions: openSlots.slice(0, 3) };
  },
  handler: bookDemo,
});

tsx

type ActionCheckResult =
  | { ok: true }
  | { ok: false; reason?: string; suggestions?: unknown[] };

suggestions are alternatives in the param's own type — for a datetime, YYYY-MM-DDTHH:mm:ss strings. The SDK forwards the first 5 and the first 200 characters of reason.

A check gets 2.5 seconds. A throw or a timeout lets the value through unverified, so a broken validator can never block a booking. Keep the same check inside your handler — it is the one that counts.

Your own UI

Voice alone completes every action. To also show the user what is happening — an input box for the detail the avatar is asking for, a confirm card at the read-back, the outcome — use useAvatarActionFlow.

tsx

const { flow, input, confirm, cancel } = useAvatarActionFlow("book_demo");

Pass a name to see only that action's flow, or no name to see any action's.

Prop	Type	Default	Description
`flow`Optional	`ActionFlow \| null`	No default	The current flow as the server last reported it, or null. It stays at its last stage until another action starts or the session ends.
`input`Optional	`(args: Record<string, unknown>) => boolean`	No default	Send what the user typed or picked. Only declared params of the declared type are accepted. Starts the action if none is in progress.
`confirm`Optional	`() => boolean`	No default	The user tapped Confirm on the details shown. Only valid during review.
`cancel`Optional	`() => boolean`	No default	The user backed out, at any stage before the run.

Each method returns false when nothing was sent — no live session, no action in progress, or a value that does not match its param — and reports a non-fatal error saying why.

ActionFlow

Prop	Type	Default	Description
`actionId`Optional	`string`	No default	Identifies this run of the action.
`revision`Optional	`number`	No default	Rises on every report. confirm() sends it back, so a card showing old details cannot agree to new ones.
`name`Optional	`string`	No default	The action's name.
`stage`Optional	`ActionFlowStage`	No default	collecting, review, running, done, cancelled or expired. Ignore a stage you do not recognise — a newer server may add one.
`args`Optional	`Record<string, unknown>`	No default	The exact values so far, in the shapes your handler receives.
`missing`Optional	`string[]`	No default	Params still to be collected.
`awaiting`Optional	`ActionAwaiting \| null`	No default	What the avatar is waiting for right now — your cue for which card to show.
`outcome`Optional	`{ ok: boolean; message: string }`	No default	On done: how the handler's run went, in its own words.

awaiting

kind	Fields	Show
`input`	param, type, description?, values?	An input for that param. values lists the choices for an enum.
`check`	param	A pending state: your check function is vetting the value.
`confirm`	—	The details in flow.args with Confirm and Cancel.

A newer server may send a kind not listed here. Render nothing for it.

ui hints

ui tells the avatar what your screen offers, so it can say "you can also type it". It is a hint only: input() is accepted for any declared param either way.

Prop	Type	Default	Description
`inputs`Optional	`string[] \| "*"`	No default	The params your UI shows an input for, or "*" for all of them. Every name must be a declared param.
`review`Optional	`boolean`	No default	Your UI shows the details with Confirm and Cancel during the read-back. Not allowed on a lookup.

Starting an action from code

No method runs a handler directly — that would skip the user's yes. To start an action from your own UI, send its first details with input() from a flow hook named for that action. The avatar takes it from there: it asks for anything missing, reads the details back, and waits for a yes.

tsx

function BookFromCalendar({ selectedSlot }: { selectedSlot: string }) {
  const { input } = useAvatarActionFlow("book_demo");
  return <button onClick={() => input({ slot: selectedSlot })}>Book this time</button>;
}

Typed values are taken as exact. A change made during review still needs a fresh yes.

What the avatar can and cannot do

It can only run actions you declared, and only while they are registered.
It never runs an action without a yes. The details are read back first, and the yes must come on a later turn or from your Confirm button. Only a lookup skips this, which is why a lookup must change nothing.
There is no way to turn confirmation off. No field on the definition skips the read-back.
One action at a time. Starting a different action replaces the one in progress, which ends as expired.
It never sees your handler, only its name, description and params. Log lines carry action names and call IDs, never the values — those are your users' emails and phone numbers.

Actions and page controls

With the page grants on (see page awareness), the avatar can also press buttons and fill fields on the page it sits on. Those controls are deliberately narrower than an action:

A click never submits a form. A button that submits is refused as a click.
A fill never submits, and never types into a password or payment field.
A form is sent only when the user's own words ask for it. The avatar does not offer or nudge toward sending.

Anything with consequences beyond the page — a booking, a purchase, an email — belongs in an action, where the details are read back first.

Errors

Actions add no events to AvatarEventMap. Problems surface through the usual error channel, and none of them ends the session.

Code	When
`INVALID_MODE`	registerAction() outside call mode.
`INVALID_INPUT`	A refused definition (bad name, too many actions or params, oversized description, a bad enum, check: true with no check function), or input() / confirm() / cancel() with a value or a stage that does not fit.
`NOT_CONNECTED`	input(), confirm() or cancel() with no live session.

Limits

Limit	Value
Actions registered at once	`8`
Params per action	`8`
Action name	`^[a-z][a-z0-9_]{0,63}$`
Param name	`^[a-zA-Z][a-zA-Z0-9_]{0,39}$`
description	`300 characters`
Param description	`120 characters`
enum values	`1–20`
result / error	`1000 characters`
Handler time	`12 s, then the signal aborts`
check time	`2.5 s, then the value passes unverified`
Waiting for a yes	`3 user turns or 2 minutes`

Complete example: booking a demo

A booking action with a slot check, and a card that follows the avatar through the flow. The card is optional — remove it and the same booking works by voice.

tsx

"use client";
import {
  AvatarProvider,
  Avatar,
  useAvatarAction,
  useAvatarActionFlow,
  type ActionCheckResult,
  type ActionContext,
  type ActionResult,
} from "@avatarfactory/react";

async function checkSlot(param: string, value: unknown): Promise<ActionCheckResult> {
  if (param !== "slot") return { ok: true };
  const response = await fetch(`/api/demo/slots?around=${encodeURIComponent(String(value))}`);
  const { open } = (await response.json()) as { open: string[] };
  return open.includes(String(value))
    ? { ok: true }
    : { ok: false, reason: "that time is taken", suggestions: open.slice(0, 3) };
}

async function bookDemo(
  args: Record<string, unknown>,
  { signal }: ActionContext,
): Promise<ActionResult> {
  const response = await fetch("/api/demo/book", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(args),
    signal,
  });
  if (response.status === 409) return { ok: false, error: "that slot was taken a moment ago" };
  if (!response.ok) return { ok: false, error: "the booking system is not responding" };
  return { ok: true, result: `Booked. The invite is on its way to ${args.email}.` };
}

function DemoBookingAction() {
  useAvatarAction({
    name: "book_demo",
    description:
      "Book a 30-minute product demo. Ask for the time first, then the name, then the email.",
    params: {
      slot: { type: "datetime", description: "Demo start time", check: true },
      name: { type: "string", description: "The visitor's full name" },
      email: { type: "email", description: "Where to send the calendar invite" },
      teamSize: { type: "enum", values: ["1-10", "11-50", "51+"], required: false },
    },
    ui: { inputs: ["email"], review: true },
    check: checkSlot,
    handler: bookDemo,
  });
  return null;
}

function DemoBookingCard() {
  const { flow, input, confirm, cancel } = useAvatarActionFlow("book_demo");
  if (!flow) return null;

  const awaiting = flow.awaiting;
  const asksForEmail =
    awaiting?.kind === "input" && "param" in awaiting && awaiting.param === "email";

  if (asksForEmail) {
    return (
      <form
        onSubmit={(event) => {
          event.preventDefault();
          const email = new FormData(event.currentTarget).get("email");
          input({ email: String(email) });
        }}
      >
        <label>
          Or type your email
          <input name="email" type="email" required />
        </label>
        <button type="submit">Use this email</button>
      </form>
    );
  }

  if (flow.stage === "review") {
    return (
      <div role="dialog" aria-label="Confirm your demo">
        <p>{new Date(String(flow.args.slot)).toLocaleString()}</p>
        <p>{String(flow.args.name)} · {String(flow.args.email)}</p>
        <button onClick={confirm}>Confirm</button>
        <button onClick={cancel}>Cancel</button>
      </div>
    );
  }

  if (flow.stage === "done" && flow.outcome) {
    return <p role="status">{flow.outcome.message}</p>;
  }

  return null;
}

export default function DemoPage() {
  return (
    <AvatarProvider
      config={{
        getSessionToken: async () => fetchSessionToken(),
        mode: "call",
        avatar: { avatarId: "your-avatar-id" },
      }}
    >
      <Avatar />
      <DemoBookingAction />
      <DemoBookingCard />
    </AvatarProvider>
  );
}

Turn on Site actions for this avatar in the Platform before you test. With it off, the code above runs without errors and the avatar simply cannot book — check the console for actions not granted for this avatar.

App Events & Page Awareness

Events