View as Markdown Open in ChatGPT Open in Claude

Screen share

Let the user show the avatar their screen mid-call. Opt-in, picked by the user in the browser's own dialog, and sent only when the screen changes.

Screen share lets the avatar help with what is on the user's screen ("why is this form rejecting my card?", "which of these settings do I change?"). It is off unless you turn it on, and turning it on does not start a share. The avatar asks when the conversation needs it, and the user chooses what to share in the browser's own picker, or says no.

tsx

const config = {
  getSessionToken: async () => fetchSessionToken(),
  mode: "call",
  perception: { screen: true }, // permission to ask — not an instruction to share
  avatar: { avatarId: "default" },
};

Two switches, both required

A share can only start when both of these are on. With either one off, the SDK answers every request as unavailable and no picker ever opens.

Switch	Where it is set
perception.screen	Your provider config. Off by default.
Screen share	The avatar's Tool calling panel in the Platform. Off by default, and the server enforces it.

perception: { screen: true } is a grant, not a switch. Nothing is shared until the avatar asks and the user picks a tab, window or screen in the browser's dialog. Plan for a session where the user never shares.

What the user sees

The browser's picker. When the avatar asks to see the screen, the SDK opens the browser's own share dialog straight away. The user picks a tab, a window or the whole screen, or cancels. On Chromium browsers the tab the avatar is running in is left out of the list, so the avatar cannot be shared to itself.
A Share button, only if the browser insists. Some browsers will not open the picker unless the user has just clicked something. In that case a Share your screen / Not now prompt appears inside the avatar frame instead. A click on it opens the picker. With no click for 60 seconds, the request is declined.
A "Sharing your screen" pill stays in the avatar frame for as long as the share is live, with a Stop button. The browser shows its own sharing indicator as well.

The avatar can never press the Share button for the user. The picker needs a real click, and a click made by code does not count.

Placing and styling the prompt

The prompt and the pill sit at the top centre of the avatar frame by default, clear of the default bottom-centre captions. Move them when your captions sit at the top.

tsx

const config = {
  // ...
  perception: { screen: true },
  transcript: { enabled: true, position: "top-left" },
  screenShare: { position: "bottom-center" },
};

Prop	Type	Default	Description
`perception.screen`Optional	`boolean`	`false`	Allow the avatar to ask to see the user's screen. Desktop browsers only. Every share goes through the browser's own picker.
`screenShare.position`Optional	`ScreenSharePosition`	`"top-center"`	Where the Share prompt and the sharing pill sit inside the avatar frame: top-left, top-center, top-right, bottom-left, bottom-center or bottom-right. The bottom anchors lift clear of the controls bar while it is on screen.

How the prompt looks is an Avatar prop, like the other overlays:

Prop	Type	Default	Description
`screenShareClassName`Optional	`string`	No default	Class on the Share prompt and the sharing pill.
`screenShareStyle`Optional	`React.CSSProperties`	No default	Inline styles on the Share prompt and the sharing pill.

AvatarWidget renders the prompt and pill inside its own panel. It does not take the two styling props; screenShare.position on the provider config still applies.

What is sent, and when

Property	Value	Why
Audio	`never`	The share asks the browser for video only. Nothing the shared tab or window plays is captured.
Check rate	`once a second`	The SDK compares the newest frame with the last one it sent.
When a frame is sent	`on change`	A new page, a scroll or a dialog is sent at once. Small edits, like typed text or a ticked box, at most every 4 s. A still screen sends nothing.
Frame size	`1280 px longest edge`	Downscaled before encoding. Body text stays legible; anything larger only adds upload.
Encoding	`JPEG, quality 0.75`	Small enough to send inside a live call.

The SDK keeps no frames. Each frame is encoded, sent and dropped. The server holds only the latest one, in memory, and reads it when the avatar looks at the screen to answer the user. That frame is sent to a vision model to answer the question.

Event	What happens
The user presses Stop on the pill	Capture stops and the server is told the screen is off.
The user stops sharing from the browser's own control, or closes the shared window	Treated the same as Stop.
The avatar is done looking	It can end the share itself. The pill goes away.
The call ends	The share is released with the call. A share never outlives the session.
The user cancels the picker, or presses Not now	The request is declined and the avatar is told, so it can carry on without it.

If the avatar asks again while a share is already live, the SDK answers with the share in progress. It never opens a second picker.

Where it works

Environment	Result
Desktop Chrome, Edge, Firefox, Safari	Supported wherever the browser offers screen capture. The picker's pane hints (tab, window or screen first) and the hiding of the avatar's own tab are Chromium-only; other browsers show their default picker.
Mobile browsers	Unavailable. Neither iOS Safari nor Android Chrome can share a screen from a web page.
Inside an iframe	The frame needs allow="display-capture", or every request is unavailable.
React Native	Not supported. The avatar is told screen sharing is unavailable.

If your app runs inside an iframe you control, add the permission to it:

tsx

<iframe src="https://your-app.example.com" allow="display-capture; microphone" />

There is no screen-share state on useAvatar() in this release. The built-in prompt and pill are the whole UI; you cannot replace them with your own.

Complete example

tsx

import { Avatar, AvatarProvider } from "@avatarfactory/react";
import "@avatarfactory/react/styles.css";

const fetchSessionToken = async () => {
  const response = await fetch("/api/avatar-session", { method: "POST" });
  const { token } = await response.json();
  return token;
};

export default function SupportCall() {
  return (
    <AvatarProvider
      config={{
        getSessionToken: fetchSessionToken,
        mode: "call",
        controls: { enabled: true },
        perception: { screen: true },
        transcript: { enabled: true, position: "top-left" },
        screenShare: { position: "bottom-center" },
        avatar: { avatarId: "default" },
      }}
    >
      <div style={{ width: 480, height: 640 }}>
        <Avatar screenShareClassName="shadow-lg" />
      </div>
    </AvatarProvider>
  );
}

Remember the second switch: turn on Screen share in the avatar's Tool calling panel, or the avatar will never ask.

Privacy checklist

Before shipping screen share, make sure your own product does these. The SDK cannot do them for you:

Say so before the call. The browser's picker asks what to share; it does not explain why your avatar wants it.
Tell users to share a tab or window, not the whole screen, when that is enough. A whole screen can show notifications and other apps.
Cover it in your privacy policy. Frames are not stored, but they are still images of the user's screen, sent to a model vendor.
Leave the pill alone. It is the user's signal inside your UI that the share is live, and its Stop button is the fastest way out.

If the avatar only needs to know about your own page, page awareness is cheaper and asks nothing of the user. See App events & page awareness.

Camera Perception

App Events & Page Awareness

Screen share

Two switches, both required

What the user sees

Placing and styling the prompt

What is sent, and when

When the share ends

Where it works

Complete example

Privacy checklist

Screen share

Two switches, both required

What the user sees

Placing and styling the prompt

What is sent, and when

When the share ends

Where it works

Complete example

Privacy checklist