# React Native (1.1.0) > React Native SDK Integration Guide - Voice-enabled AI agent with real-time communication capabilities (React Navigation, events & Action Intelligence) URL: /embed/integration/react-native/v1.1.0 Markdown: /embed/integration/react-native/v1.1.0.md # React Native Integration Guide [#react-native-integration-guide] The complete guide for adding the RevRag embedded voice agent ("AI Saathi") to a React Native app. | | | | ---------------- | ------------------------------------------------------------------------------- | | **Package** | `@revrag-ai/embed-react-native` | | **SDK version** | **1.1.0** on [npm](https://www.npmjs.com/package/@revrag-ai/embed-react-native) | | **React Native** | **0.76 – 0.86** (reference: 0.78, Hermes) | | **API key** | Get it from [https://app.revrag.ai](https://app.revrag.ai) | **How this guide works** | | | | ----------------- | --------------------------------------------------------------------------------------------------- | | **Order** | **Basic → intermediate → advanced** | | **Every feature** | What is it? → Why do I need it? → How do I configure it? → Example → Common problem → How to fix it | | **SDK internals** | You do **not** need to know how the SDK works internally to follow this | **A working widget takes five steps**, all in [1. Basic Integration](#1-basic-integration): | Step | What you do | Details | | ------------------- | ----------------------------------------- | ----------------------------------------------------------- | | 1. **Install** | Install the SDK and its dependencies | [Step 1](#step-1--install-the-sdk-and-its-dependencies) | | 2. **Native setup** | Required, one time | [Step 2](#step-2--native-setup-required-one-time) | | 3. **Initialize** | Initialize the SDK once, at the app root | [Step 3](#step-3--initialize-the-sdk-once-at-the-app-root) | | 4. **Wrap** | Wrap your app and add the widget | [Step 4](#step-4--wrap-your-app-and-add-the-widget) | | 5. **Identify** | Tell the SDK who the user is, after login | [Step 5](#step-5--tell-the-sdk-who-the-user-is-after-login) | **If you only read one section, read [1. Basic Integration](#1-basic-integration).** That alone gets a working widget on screen. *** ## Table of contents [#table-of-contents] 1. [Basic Integration](#1-basic-integration) — install, native setup, provider, first call 2. [Configuration](#2-configuration) — `useInitialize` and `EmbedProvider` options 3. [Screen & Navigation](#3-screen--navigation) — how the SDK knows where the user is 4. [Widget Visibility](#4-widget-visibility) — where the button shows, and why it matters 5. [Event Capturing](#5-event-capturing) — data events, click tracking, agent lifecycle 6. [Action Intelligence / UI Capture](#6-action-intelligence--ui-capture) — how the agent sees and acts 7. [Advanced Integration](#7-advanced-integration) — manual control, groups, permissions 8. [Troubleshooting](#8-troubleshooting) — problem → why → fix 9. [Debugging Guide](#9-debugging-guide) — rule out one layer at a time 10. [Integration Examples](#10-integration-examples) — copy-paste patterns 11. [Best Practices](#11-best-practices) *** ## 1. Basic Integration [#1-basic-integration] **Goal:** a floating voice-agent button on your screens that a user can tap to talk to the agent. There are five steps. ### Step 1 — Install the SDK and its dependencies [#step-1--install-the-sdk-and-its-dependencies] ```bash # The SDK npm install @revrag-ai/embed-react-native # Required peer dependencies (the SDK will not run without these) npm install \ @livekit/react-native \ @livekit/react-native-webrtc \ @react-native-async-storage/async-storage \ react-native-gesture-handler \ react-native-reanimated \ react-native-linear-gradient \ lottie-react-native \ react-native-safe-area-context # iOS only: install native pods cd ios && pod install && cd .. # iOS only: register the SDK's bundled fonts (run once) npx react-native-asset ``` **Why these dependencies?** The agent is a live voice call (LiveKit + WebRTC), the widget is animated (reanimated, lottie, linear-gradient), it stores a little config on device (async-storage), and the button is draggable and safe-area aware (gesture-handler, safe-area-context). They are marked as *peer* dependencies so you control their exact versions. **Read this before pinning versions.** Recommended version for every peer dependency, two verified copy-paste stacks (RN 0.78–0.81 and RN 0.82–0.86), the supported React Native matrix, and the dependency pitfalls that break calls — like a duplicate `livekit-client`. ### Step 2 — Native setup (required, one time) [#step-2--native-setup-required-one-time] **Babel** — add the reanimated plugin as the **last** plugin in `babel.config.js`: ```js module.exports = { presets: ['module:@react-native/babel-preset'], plugins: [ // ...any other plugins first... 'react-native-reanimated/plugin', // MUST be last ], }; ``` **LiveKit native init is required** — skipping it makes the first call crash with `audioRecordSamplesDispatcher is not initialized!`. Android — `android/app/src/main/java//MainApplication.kt`: ```kotlin import com.livekit.reactnative.LiveKitReactNative class MainApplication : Application(), ReactApplication { override fun onCreate() { super.onCreate() LiveKitReactNative.setup(this) // ← before React Native starts // ...rest of onCreate... } } ``` iOS — `ios//AppDelegate.swift`: ```swift import livekit_react_native // note the module name (lowercase, underscores) func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: ...) -> Bool { LivekitReactNative.setup() // ← before React Native starts (lowercase "k") // ...rest of setup... return true } ``` **iOS microphone permission** — add to `ios//Info.plist` (missing this **crashes** the app when a call starts): ```xml NSMicrophoneUsageDescription This app needs microphone access to talk to the AI agent. ``` **Android permissions — you add nothing.** The SDK's manifest merges `RECORD_AUDIO`, `MODIFY_AUDIO_SETTINGS` and `BLUETOOTH_CONNECT` (LiveKit adds `INTERNET`, `CAMERA`, …), and the microphone runtime prompt is shown by the SDK during `startCall`. Voice-only app? You may strip camera: ``. **Do not call LiveKit's `registerGlobals()` yourself** — `useInitialize` does it. A second call duplicates native event listeners (relevant if your app already uses LiveKit — keep your existing native `setup()`, don't add another, and tell RevRag). After native changes, **rebuild fully** (`gradlew clean` / `pod install`) — a Metro reload is not enough. ### Step 3 — Initialize the SDK once, at the app root [#step-3--initialize-the-sdk-once-at-the-app-root] **What:** `useInitialize` registers your app with the RevRag backend and loads the widget configuration. **Why:** nothing else works until this has run. **Where:** the top-level `App` component, once. ```tsx import { useInitialize } from '@revrag-ai/embed-react-native'; function App() { useInitialize({ apiKey: 'YOUR_API_KEY', // required — from RevRag appVersion: '1.0.0', // required — your app's version (see section 2) }); // ... } ``` ### Step 4 — Wrap your app and add the widget [#step-4--wrap-your-app-and-add-the-widget] **What:** `EmbedProvider` is the wrapper that tracks the current screen and **renders the floating button for you** — you do **not** render the button yourself. **Why the order matters:** it must sit *outside* your navigation container so it can see route changes, and *inside* `GestureHandlerRootView` + `SafeAreaProvider`. Put it **above** any portal / paper / bottom-sheet providers too, or the widget renders underneath them. **`navigationRef` is required for the button to appear.** The SDK learns the current screen only from the navigation listener attached through this ref — with no ref (or a different ref than the one on `NavigationContainer`) the button never shows and `startCall()` fails. Mount the container within \~1 s of the provider (don't gate it behind async auth/font loading — gate *inside* it instead). ```tsx import { GestureHandlerRootView } from 'react-native-gesture-handler'; import { SafeAreaProvider } from 'react-native-safe-area-context'; import { EmbedProvider } from '@revrag-ai/embed-react-native'; import { NavigationContainer } from '@react-navigation/native'; import { useRef } from 'react'; export default function App() { const navigationRef = useRef(null); useInitialize({ apiKey: '…', appVersion: '1.0.0' }); return ( {/* appVersion lives in useInitialize (required); no need to repeat it here */} {/* your navigator */} ); } ``` That is enough to **see** the button. To let it actually place a **call**, do Step 5. ### Step 5 — Tell the SDK who the user is (after login) [#step-5--tell-the-sdk-who-the-user-is-after-login] **What:** a `USER_DATA` event with your app's user id. **Why:** the agent needs to know which user it is helping. Until it's sent, every other event is dropped and a call's token request aborts (`[Embed Call] no app_user_id stored`). Note `startCall()` still resolves `true` in that case — treat `AGENT_CONVERSATION_STARTED` as the real signal. It is stored only once the backend accepts it, so if sent offline, send it again. ```tsx import { Embed, EmbedEventKeys } from '@revrag-ai/embed-react-native'; // call this right after your user logs in await Embed.Event(EmbedEventKeys.USER_DATA, { app_user_id: 'user_123' }); ``` ### Complete copy-paste example [#complete-copy-paste-example] ```tsx import React, { useEffect, useRef } from 'react'; import { GestureHandlerRootView } from 'react-native-gesture-handler'; import { SafeAreaProvider } from 'react-native-safe-area-context'; import { NavigationContainer } from '@react-navigation/native'; import { createNativeStackNavigator } from '@react-navigation/native-stack'; import { useInitialize, EmbedProvider, Embed, EmbedEventKeys, } from '@revrag-ai/embed-react-native'; import HomeScreen from './HomeScreen'; import LoginScreen from './LoginScreen'; const Stack = createNativeStackNavigator(); export default function App() { const navigationRef = useRef(null); useInitialize({ apiKey: 'YOUR_API_KEY', appVersion: '1.0.0' }); // In a real app, send this after the user actually logs in. useEffect(() => { Embed.Event(EmbedEventKeys.USER_DATA, { app_user_id: 'user_123' }); }, []); return ( ); } ``` **That's the whole minimum integration.** Everything below is optional refinement. *** ## 2. Configuration [#2-configuration] Two places hold configuration: **`useInitialize(...)`** (credentials) and **``** (behavior). Backend-controlled settings (agent voice, colors, feature flags) are managed by RevRag remotely — you do not set those in code. ### `useInitialize` options [#useinitialize-options] | Option | Required | Default | What it does / when to use | | ------------ | -------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `apiKey` | ✅ | — | Your RevRag API key. Identifies your tenant. | | `embedUrl` | optional | RevRag default | Your RevRag server URL. Set it if RevRag gave you a specific host. | | `appVersion` | ✅ | — | Your app's version string, attached to every event and sync. **This is the canonical place to set it** (alongside `apiKey`/`embedUrl`), and it **takes priority over** the optional `EmbedProvider` `appVersion` prop. Use `require('./package.json').version`. | ### `EmbedProvider` props [#embedprovider-props] | Prop | Required | Default | What it does / when to use | | ------------------------------- | -------------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `children` | ✅ | — | Your app tree (must contain the `NavigationContainer`). | | `appVersion` | optional | — | **Override only.** Set `appVersion` in `useInitialize` (required) — that value wins. Pass it here only if the version isn't available at the init call site. Omitting it everywhere logs a warning and falls back to the native build version. | | `navigationRef` | ✅ (for the widget to show) | — | Your React Navigation ref — the **same one** passed to `NavigationContainer`. It is how the SDK learns the current screen; **without it the button never appears** and `startCall()` fails. Also enables the agent's **back** action. | | `showOnAllScreens` | optional | `false` | Show the button on every screen except `excludeScreens`. The simplest setup. | | `includeScreens` | optional | `[]` (none) | Allow-list of screen names where the button shows. **Only used when `showOnAllScreens` is `false`.** | | `excludeScreens` | optional | `[]` (none) | Screens the button must **never** show on. Wins over everything. Landing on one also **ends an active call** (see [section 4](#4-widget-visibility)). Use for Login/Splash/checkout. | | `endCallWhenHiddenByVisibility` | optional | `true` | When `true`, navigating to a screen where the button is hidden ends the live call. Set `false` to let a call continue in the background across such screens. | | `embedButtonContinuity` | optional | `'continuous'` | `'continuous'` keeps the button mounted across allowed screens (delay applied once). `'perScreen'` re-evaluates and re-delays the button on every screen (card collapses, may flicker). **Neither ends a call** — only the visibility rules / `excludeScreens` do. | | `embedButtonDelayMs` | optional | `0` | Delay before the button appears on a screen (ms). Use to avoid a flash during screen transitions. | | `embedButtonVisibilityConfig` | optional | — | Advanced per-group visibility (delay/inset/continuity per set of screens). See [section 7](#7-advanced-integration). | | `sessionScreenshotEnabled` | optional | `false` | Your consent for in-call screenshots. Capture also requires a backend flag (double consent). Leave off unless RevRag asks. | | `passiveScreenshotEnabled` | optional | `false` | Your consent for background/cache screenshots. Same double-consent rule. Leave off unless RevRag asks. | **Minimal, recommended, and full examples** (all assume `useInitialize({ …, appVersion })` ran at the app root — so `appVersion` is not repeated on the provider): ```tsx // Minimal — button on every screen (navigationRef is still required) {children} // Recommended — screen tracking + hide on auth screens {children} // Only on specific screens {children} ``` *** ## 3. Screen & Navigation [#3-screen--navigation] **What:** the SDK needs to know **which screen the user is on** — to decide where the button shows, to give the agent context, and to let the agent navigate. **Why:** every visibility and context decision is keyed on the current screen **name**. ### How screen detection works [#how-screen-detection-works] Pass your navigation ref as `navigationRef`. The SDK listens to route changes and uses the **active route name** as the current screen. So the names in `includeScreens` / `excludeScreens` must match your **route names exactly**. ```tsx // → the SDK's screen name for this route is "submit_interest" ``` Name routes with stable, backend-friendly ids (e.g. `submit_interest`, `home_tab`) rather than display titles — they become the agent's screen identity. ### React Navigation (recommended) [#react-navigation-recommended] Wrap `NavigationContainer` with `EmbedProvider` and pass the shared ref (see [section 1, Step 4](#1-basic-integration)). Nested navigators (tabs inside a stack) work automatically — the SDK tracks the deepest active route. ### Non-React-Navigation / custom / WebView screens [#non-react-navigation--custom--webview-screens] Screen detection (and therefore widget visibility and UI capture) works **only** through a React Navigation `navigationRef`. A `SCREEN_STATE` event does **not** drive it — it only tags your later analytics events with a screen name: ```tsx import { Embed, EmbedEventKeys } from '@revrag-ai/embed-react-native'; Embed.Event(EmbedEventKeys.SCREEN_STATE, { screen: 'checkout_webview' }); // analytics tag only ``` If your app uses React Native Navigation (Wix), a state-driven router, or Expo Router, talk to RevRag before integrating — the supported path is real React Navigation routes. WebView content is opaque to the agent; name the wrapping route and exclude it if sensitive. ### Custom screen matching [#custom-screen-matching] **What:** helps the backend recognize a screen even when its name varies (aliases), by ancestors, or by structure. **When to use:** only if the agent is failing to identify a screen correctly. Off by default — the backend controls the tiers. ```tsx import { setScreenMatching, SCREEN_MATCHING_ALL } from '@revrag-ai/embed-react-native'; // Turn all matching tiers on (call once at startup) setScreenMatching(SCREEN_MATCHING_ALL); // Or opt into specific tiers setScreenMatching({ aliasMatching: true, ancestorMatching: true }); ``` ### Visibility props recap [#visibility-props-recap] Full details are in [section 4](#4-widget-visibility). * `showOnAllScreens` — button everywhere except `excludeScreens`. * `includeScreens` — allow-list (used only when `showOnAllScreens` is `false`). * `excludeScreens` — never-show list; also ends a live call on entry. * `endCallWhenHiddenByVisibility` — whether a call ends when the button hides. ### When the SDK can't identify a screen [#when-the-sdk-cant-identify-a-screen] * **Wrong or empty screen name?** Make sure `navigationRef` is the **same ref** passed to `NavigationContainer`, and that `EmbedProvider` wraps the container. * **Custom/native/WebView screen?** Send `SCREEN_STATE` manually (above). * **Duplicate/ambiguous names?** Give routes unique names, or enable custom matching. *** ## 4. Widget Visibility [#4-widget-visibility] ### How visibility works [#how-visibility-works] The button shows based on this precedence: 1. No screen name yet (no `navigationRef`, container not mounted) → **hidden**. 2. `showOnAllScreens` is `true` → **shown**, unless the screen is in `excludeScreens`. 3. Otherwise → shown only if the screen is in `includeScreens` (or a configured group); an empty list means all. **In this mode `excludeScreens` is not consulted** — use `showOnAllScreens` + `excludeScreens` when you need an exclusion list. Names match **exactly** (case-sensitive) unless you enable `setScreenMatching` tiers. `embedButtonDelayMs` can delay the appearance; `embedButtonContinuity` decides whether it stays mounted across navigation. ### Show / hide the widget yourself [#show--hide-the-widget-yourself] You normally don't need to — the provider handles it. For programmatic control of the **expanded card** (not the button itself): ```tsx import { expandWidget, collapseWidget, isWidgetExpanded } from '@revrag-ai/embed-react-native'; expandWidget(); // open the card — call ONLY when the user asked (counts as a tap) collapseWidget(); // close the card isWidgetExpanded(); // → boolean ``` ### Why visibility matters (this is important) [#why-visibility-matters-this-is-important] **Widget visibility is the call's licence.** Two rules follow: * **`startCall` is refused** on a screen where the button is not shown. * **Arriving on a hidden screen ends a live call** — unless you set `endCallWhenHiddenByVisibility={false}`. Visibility also affects what the agent can **do** vs **see**: * The agent's live in-call UI reading (Action Intelligence) follows the same screens. **UI capture for the durable cache continues on every screen regardless** — hiding the button hides the *button*, not necessarily all capture. Use `excludeScreens` (not just leaving a screen out of `includeScreens`) for screens that must never be walked or transmitted. ### Troubleshooting [#troubleshooting] **Widget not appearing.** Why: not initialized, no `USER_DATA` yet is *not* required for the button to show, but the screen may not be included, or the provider isn't wrapping navigation. Fix: confirm `useInitialize` ran; confirm the current route name is included (`showOnAllScreens` or in `includeScreens`); confirm `EmbedProvider` wraps `NavigationContainer`; check `embedButtonDelayMs` isn't set too high. **Widget on the wrong screen.** Why: route name mismatch between your navigator and `includeScreens`/`excludeScreens`. Fix: log the active route name and make the lists match it exactly. **Widget disappears during/after navigation.** Why: `embedButtonContinuity: 'perScreen'` re-evaluates (and re-delays) on each screen; or the destination isn't included. Fix: use `'continuous'` (default) to keep it across navigation; ensure the destination screen is included. **Widget shows on some screens but not others.** Why: that's `includeScreens`/`excludeScreens` working as configured. Fix: adjust the lists, or switch to `showOnAllScreens` with a small `excludeScreens`. *** ## 5. Event Capturing [#5-event-capturing] The SDK captures user activity and forwards it to the agent/analytics. Some events are automatic; some you send yourself. ### The public event channel [#the-public-event-channel] ```tsx import { Embed, EmbedEventKeys } from '@revrag-ai/embed-react-native'; Embed.Event(EmbedEventKeys.USER_DATA, { app_user_id: 'user_123' }); // identify user (enables calls) Embed.Event(EmbedEventKeys.SCREEN_STATE, { screen: 'checkout' }); // manual screen report Embed.Event(EmbedEventKeys.CUSTOM_EVENT, { name: 'coupon_applied' }); // your own event Embed.Event(EmbedEventKeys.ANALYTICS_DATA, { event_name: 'cta_click', metadata: {} }); ``` `Embed.Event` never throws; pass a third callback to observe delivery. Use `Embed.on` / `Embed.off` for local listeners. ### Automatic click / press tracking [#automatic-click--press-tracking] **What:** the SDK patches React Native touchables so taps become analytics events. **Supported components:** `Pressable`, `TouchableOpacity`, `TouchableHighlight`, `TouchableWithoutFeedback`, and `TouchableNativeFeedback` (Android). **When it emits:** only while the widget is visible on the current screen, and only if click tracking is enabled on the backend. **How the element id is chosen** (this determines your analytics event name): ``` id prop → name prop → generated fallback (e.g. "touchableopacity-7") ``` `testID` and `accessibilityLabel` are **not** used for the click id. If a button has neither `id` nor `name`, you get a generated id like `pressable-3`, and the analytics event is not human-readable. Example: ```tsx Submit ``` This emits an `ANALYTICS_DATA` event named **`submit_pressed`**, with metadata `{ timestamp, type: 'press', component: 'Pressable', clickSequence }`. To get meaningful analytics, **always put an `id` or `name` on interactive elements.** ### Navigation, screen change, scroll, UI interactions [#navigation-screen-change-scroll-ui-interactions] * **Screen changes** are captured automatically when you pass `navigationRef` (or via manual `SCREEN_STATE`). * **Scroll and general UI interaction** are captured internally to keep the agent's view fresh during a call (they feed UI capture, [section 6](#6-action-intelligence--ui-capture)) — you don't wire these up. * **Manual actions** — call control (`startCall`/`endCall`), widget control, and custom events are the actions you trigger yourself. ### Agent lifecycle events (listen to the agent) [#agent-lifecycle-events-listen-to-the-agent] **What:** one subscription that receives **every** SDK lifecycle event — the same model as Android's `SdkLifecycleListener`. A single `embedOnAgent` handler is called for all events; you branch on `event.type`. `embedOnAgent` returns a **handle string**; pass it to `embedOffAgent` to unsubscribe. **The complete list of events you can listen to:** | `AgentEvent` value | Fires when | | ------------------------------- | -------------------------------------------------- | | `AGENT_VISIBLE` | The floating button appears on the current screen. | | `AGENT_TAP_TO_OPEN` | The user expands the widget card. | | `AGENT_TAP_TO_CLOSE` | The user collapses the widget card. | | `POPUP_MESSAGE_VISIBLE` | An inactivity tooltip / nudge is shown. | | `AGENT_CONVERSATION_STARTED` | A voice call connects. | | `AGENT_CONVERSATION_ENDED` | A voice call ends (any reason). | | `GEN_TOOL_TRIGGERED` | The agent invokes an AI tool mid-call. | | `MICROPHONE_PERMISSION_ALLOWED` | Mic permission was granted. | | `MICROPHONE_PERMISSION_DENIED` | Mic permission was denied/blocked. | `AGENT_CONNECTED` / `AGENT_DISCONNECTED` also exist but are **deprecated** aliases of `AGENT_CONVERSATION_STARTED` / `AGENT_CONVERSATION_ENDED` — use the newer names. **Full example — handle every event with one listener:** ```tsx import React, { useEffect } from 'react'; import { embedOnAgent, embedOffAgent, AgentEvent } from '@revrag-ai/embed-react-native'; function useAgentLifecycle() { useEffect(() => { const handle = embedOnAgent((event) => { // `event.type` is the AgentEvent; `event.timestamp` and event-specific // fields (e.g. event.metadata) ride along on the same object. switch (event.type) { case AgentEvent.AGENT_VISIBLE: console.log('Button visible on screen'); break; case AgentEvent.AGENT_TAP_TO_OPEN: console.log('User opened the widget'); break; case AgentEvent.AGENT_TAP_TO_CLOSE: console.log('User closed the widget'); break; case AgentEvent.POPUP_MESSAGE_VISIBLE: console.log('Nudge shown, trigger:', event.metadata?.trigger); break; case AgentEvent.AGENT_CONVERSATION_STARTED: console.log('Call started at', event.timestamp); break; case AgentEvent.AGENT_CONVERSATION_ENDED: console.log('Call ended, duration:', event.metadata?.callDuration); break; case AgentEvent.GEN_TOOL_TRIGGERED: console.log('Agent used a tool mid-call'); break; case AgentEvent.MICROPHONE_PERMISSION_ALLOWED: console.log('Mic granted'); break; case AgentEvent.MICROPHONE_PERMISSION_DENIED: console.warn('Mic denied:', event.error_message); break; default: break; } }); // ALWAYS unsubscribe on unmount, or handlers leak. return () => embedOffAgent(handle); }, []); } ``` ### Analytics helpers [#analytics-helpers] ```tsx import { trackEmbedEvent, trackFormEvent, trackRageClick } from '@revrag-ai/embed-react-native'; trackEmbedEvent('promo_seen', { id: 'diwali' }); ``` ### Two ids that are easy to confuse [#two-ids-that-are-easy-to-confuse] | | Click-event `elementId` | Action-Intelligence `stableId` | | ------------ | --------------------------- | ------------------------------------------------ | | Purpose | Click analytics event name | Identify a node in the UI tree for the agent | | Derived from | `id` → `name` → generated | `testID` → `id`/`nativeID` → visible-text → role | | Example | `submit` → `submit_pressed` | `button_submit_interest` | They are **independent**. The same button can be `submit` in click analytics and `button_submit_interest` in the UI tree. Setting `testID` improves the `stableId`; setting `id`/`name` improves the click `elementId`. For the cleanest result, set a `testID` **and** an `id`/`name` with the same value. *** ## 6. Action Intelligence / UI Capture [#6-action-intelligence--ui-capture] **What it is:** the agent can *see* your screen and *act* on it (tap, type, scroll, highlight) to help the user complete a task. **Why you rarely configure it:** it works out of the box; your only real job is to **name elements well** ([section 11](#11-best-practices)) so the agent targets the right thing. ### The flow, in plain terms [#the-flow-in-plain-terms] ``` Your app UI → Capture (the SDK reads the on-screen elements) → UI Snapshot (a clean, structured tree of those elements) → Backend (sent live during a call, and cached durably over HTTP) → Action Intelligence (the agent decides what to do) → Action (tap / type / select / scroll / highlight on your UI) → Verification (the SDK checks the action actually changed the screen) ``` ### What is captured, and when [#what-is-captured-and-when] * **What:** interactive and meaningful elements — buttons, inputs, text, form fields, checkboxes/radios/switches/sliders — with their on-screen position and a stable id. Purely decorative wrapper views are dropped automatically. * **When:** on connect (a baseline), and then on navigation, scroll, taps, and when the user finishes speaking. Identical screens are de-duplicated (nothing re-sent if nothing changed). * **Order:** elements are sent **top-to-bottom, matching the screen**, with a structure map so the backend can rebuild the exact hierarchy. ### Two channels (you don't manage these — just know they exist) [#two-channels-you-dont-manage-these--just-know-they-exist] * **Live (in-call):** a size-capped snapshot of the current screen, sent over the call's data channel for real-time planning. * **Durable cache (HTTP):** a persistent map of screens/elements/navigation built up over time, synced in the background. ### How elements are identified [#how-elements-are-identified] The agent targets elements by `stableId`: `testID` → `id`/`nativeID` → visible text → role. **The single most useful thing you can do is add `testID`s** to the elements you want the agent to operate reliably. ```tsx Submit ``` ### Sensitive fields (do this) [#sensitive-fields-do-this] * **Passwords/OTPs:** use `secureTextEntry` on the `TextInput`. The SDK **never captures the value**, and the agent refuses to read or type into it. * **Other sensitive fields (masking capture but still lettable):** add `nativeID="embed-redact-"`. The value/text is never captured, but the agent may still type into it. ### What you need to do on the host side [#what-you-need-to-do-on-the-host-side] 1. Pass `navigationRef` so screen context is correct. 2. Add `testID`s to important interactive elements. 3. Mark sensitive fields (`secureTextEntry` / `embed-redact`). 4. Use `excludeScreens` for screens that must never be captured/transmitted. ### Common problem [#common-problem] **Agent taps the wrong element / can't find a field.** Why: the element has no stable id, so it falls back to text/role and collides with another node. Fix: add a unique `testID`. For dynamic lists, make the `testID` unique per row. *** ## 7. Advanced Integration [#7-advanced-integration] Most apps never need this section. Reach for it only when the basics don't cover a real requirement — **over-configuring causes more bugs than it prevents.** ### When advanced config is actually necessary [#when-advanced-config-is-actually-necessary] | You need… | Use | | ----------------------------------------------------------------------- | --------------------------------------- | | Agent to keep running while the user visits a screen without the button | `endCallWhenHiddenByVisibility={false}` | | Different button delay/position per group of screens | `embedButtonVisibilityConfig` | | The agent to recognize a hard-to-identify screen | `setScreenMatching` | | Reliable agent targeting on custom components | `testID`s + `nativeID` | | Your own conversation UI (agent shouldn't hold the mic) | `pauseAgent` / `resumeAgent` | | Drive calls from your own buttons | `startCall` / `endCall` | | Report screens without React Navigation | `SCREEN_STATE` events | ### Custom per-group visibility [#custom-per-group-visibility] ```tsx {children} ``` Full prop tables, JSON examples, and behavior notes for grouped visibility live in the companion guide: [EmbedProvider advanced](/embed/integration/react-native/v1.1.0/embed-provider-advanced). ### Manual call control [#manual-call-control] ```tsx import { startCall, endCall, isCallActive, getAgentTriggerMode, } from '@revrag-ai/embed-react-native'; const started = await startCall(); // returns false if refused (no widget on screen, no user set) if (isCallActive()) await endCall(); ``` `startCall()` runs permission → token → connect. It returns **`false`** when no button is shown on the current screen (or none is mounted yet). If `USER_DATA` was never sent the token request aborts but the call **still resolves `true`** — so observe `AGENT_CONVERSATION_STARTED` / `isCallActive()` for the real outcome. ### Pause the agent without ending the call [#pause-the-agent-without-ending-the-call] ```tsx import { pauseAgent, resumeAgent, isAgentPaused } from '@revrag-ai/embed-react-native'; await pauseAgent(); // agent stops listening/speaking; room stays up await resumeAgent(); // give the mic back ``` ### Session / user switch [#session--user-switch] ```tsx import { EmbedLogout } from '@revrag-ai/embed-react-native'; await EmbedLogout(); // ends call, wipes on-device cache, forgets the user // then send a fresh USER_DATA for the next user ``` ### Permissions (manual) [#permissions-manual] ```tsx import { checkPermissions, openAppSettings, MIC_PERMISSION_REQUIRED_MESSAGE } from '@revrag-ai/embed-react-native'; const result = await checkPermissions(); // 'granted' | 'denied' | 'blocked'; never throws if (result === 'blocked') openAppSettings(); // user must re-enable mic in OS settings ``` ### Screenshots (opt-in, double consent) [#screenshots-opt-in-double-consent] Both `sessionScreenshotEnabled` and `passiveScreenshotEnabled` default `false` and also require a backend flag. Leave off unless RevRag explicitly enables the feature for you. ### Platform / version notes [#platform--version-notes] * **React Native:** the tested reference is **0.78** with the **New Architecture** and **Hermes**; **0.76 – 0.86** are supported with the right dependency stack (RN ≥ 0.82 needs Reanimated 4 + `react-native-worklets` + Gesture Handler 3). Version pins per RN line: [Dependency & Compatibility Guide](/embed/integration/react-native/v1.1.0/react-native-dependencies#3-supported-react-native-versions). * **Android:** add the LiveKit `MainApplication` setup ([section 1](#1-basic-integration)); if using ProGuard, keep the Embed classes (see below) — WebRTC/LiveKit/Lottie ship their own consumer keep rules. LiveKit increases native binary size — for production APK size, see [Android app size optimization](/embed/integration/android/android-app-size-optimization). * **iOS:** `NSMicrophoneUsageDescription` is mandatory; run `npx react-native-asset` once for fonts; `pod install` after every dependency change. ProGuard (`android/app/proguard-rules.pro`): ```text -keep class com.revragai.embedreactnative.** { *; } # WebRTC / LiveKit / Lottie ship their own consumer keep rules — nothing else needed. ``` *** ## 8. Troubleshooting [#8-troubleshooting] Format for each: **Problem → Why → Fix → Example.** ### Crashes [#crashes] **`audioRecordSamplesDispatcher is not initialized!` on first call.** Why: LiveKit native setup missing. Fix: add `LiveKitReactNative.setup(...)` to `MainApplication.kt` **and** `AppDelegate.swift`, then rebuild. Example: see [section 1, Step 2](#1-basic-integration). **iOS crash: "attempted to access privacy-sensitive data without a usage description".** Why: `NSMicrophoneUsageDescription` missing from `Info.plist`. Fix: add the key and rebuild. **Reanimated errors / white screen after install.** Why: the reanimated Babel plugin is missing or not last. Fix: put `'react-native-reanimated/plugin'` **last** in `babel.config.js`, then `npx react-native start --reset-cache` and rebuild. **Native dependency / version conflict at build time.** Why: a peer dependency is missing or mismatched. Fix: install all peer deps from [section 1](#1-basic-integration); check every version against the [Dependency & Compatibility Guide](/embed/integration/react-native/v1.1.0/react-native-dependencies#2-recommended-versions); on iOS run `pod install`; clean builds (`gradlew clean`, delete `Pods/`). **When reporting a crash, include:** the full stack trace, platform + OS version, RN version, SDK version (`@revrag-ai/embed-react-native` from `package.json`), the screen it happened on, and whether it was during install, a call, or navigation. ### Widget visibility [#widget-visibility] **Not visible.** Why: `useInitialize` didn't run, screen not included, or provider not wrapping navigation. Fix: verify all three (see [section 4](#4-widget-visibility)). **On the wrong screen.** Why: route-name mismatch. Fix: match `includeScreens`/`excludeScreens` to actual route names. **Disappears during navigation.** Why: `embedButtonContinuity: 'perScreen'` or destination not included. Fix: use `'continuous'`; include the destination. **Not back after returning to a screen.** Why: a delay (`embedButtonDelayMs`) re-applies, or `perScreen` continuity re-evaluates. Fix: lower the delay; use `'continuous'`. ### Screen matching [#screen-matching] **Wrong screen detected / name mismatch.** Why: display title used instead of a stable route name, or duplicate names. Fix: give routes unique, stable names; report custom screens via `SCREEN_STATE`. **Screen still not recognized by the agent.** Why: the backend can't map it. Fix: `setScreenMatching(SCREEN_MATCHING_ALL)` at startup, or enable specific tiers. ### Events [#events] **Click events not captured.** Why: widget not visible on that screen, or backend click tracking disabled. Fix: ensure the button shows there; confirm the flag with RevRag. **Generated ids (`pressable-4`) instead of meaningful ones.** Why: the element has no `id`/`name`. Fix: add `id` or `name` (see [section 5](#5-event-capturing)). **Events on the wrong screen.** Why: screen name mismatch/stale. Fix: pass `navigationRef` or send `SCREEN_STATE`. **Agent not receiving UI information.** Why: not in a call, or the screen is excluded. Fix: start a call; check the screen isn't in `excludeScreens`. ### UI capture [#ui-capture] **Tree missing elements.** Why: decorative wrappers are pruned by design; or the element mounts late. Fix: only interactive/meaningful nodes are expected; add a `testID` to force importance. **Wrong element ids.** Why: no `testID`, so text/role is used and collides. Fix: add unique `testID`s. **Stale tree / capture too early.** Why: captured mid-transition. Fix: the SDK re-captures on settle; if you drive navigation manually, send `SCREEN_STATE` after the screen renders. **Dialog/overlay not detected.** Why: it's a separate native window. Fix: usually handled; if not, ensure the overlay is a normal React tree under the provider. **Sensitive fields captured.** Why: not marked. Fix: `secureTextEntry` or `nativeID="embed-redact-…"` ([section 6](#6-action-intelligence--ui-capture)). ### Call / widget [#call--widget] **Call won't start.** Why: no `USER_DATA` sent, or no button on the current screen, or mic permission denied. Fix: send `USER_DATA`; start from an included screen; grant mic. **Call ends unexpectedly on navigation.** Why: moved to a hidden/excluded screen with `endCallWhenHiddenByVisibility` (default `true`). Fix: include the screen, or set the prop to `false`. **Audio / caption issues.** Why: mic permission, or LiveKit native setup. Fix: verify permission and `LiveKitReactNative.setup()`; test on a real device (emulators have flaky audio). *** ## 9. Debugging Guide [#9-debugging-guide] Work top-down; each step rules out a layer. 1. **SDK initialized?** Confirm `useInitialize` ran with a valid `apiKey`/`embedUrl` (its returned `error` is `null`). 2. **App version set?** Confirm `appVersion` in `useInitialize` or the provider (no warning in logs). 3. **Widget config loaded?** After init, the widget config should arrive; a persistent "no config" points to a bad `apiKey`/`embedUrl` or network. 4. **Screen detected?** Log the active route name; confirm it's what your include/exclude lists expect. 5. **Widget visible?** Confirm the current screen passes the visibility rules ([section 4](#4-widget-visibility)). 6. **Events captured?** Tap a named button; confirm an `ANALYTICS_DATA` `_pressed` event and correct screen. 7. **UI captured?** Start a call; in dev the SDK logs snapshot sends (`[Embed Snapshot] SENT screen=… nodes/bytes`). 8. **Logs.** Development logs are `__DEV__`-gated and prefixed `[Embed …]`. Watch `adb logcat` (Android) / Xcode console (iOS) filtered to `Embed`. 9. **Network/API.** Confirm the device reaches your `embedUrl`; check for auth (401/403) or connectivity failures. 10. **Collect info.** For a bug report: platform + OS, RN version, SDK version, screen, repro steps, and logs from steps 7–9. *** ## 10. Integration Examples [#10-integration-examples] All examples assume `useInitialize({ apiKey, appVersion })` already ran at the app root (see [section 1](#1-basic-integration)), so `appVersion` is not repeated on ``. ### 1. Basic React Native app (button everywhere) [#1-basic-react-native-app-button-everywhere] ```tsx const navigationRef = useRef(null); // required — the SDK reads the screen from it {/* your navigator */} ``` ### 2. React Navigation app (screen tracking) [#2-react-navigation-app-screen-tracking] ```tsx const navigationRef = useRef(null); {/* navigator */} ``` ### 3. App using `Pressable` (meaningful click ids) [#3-app-using-pressable-meaningful-click-ids] ```tsx Apply // → analytics event "apply_loan_pressed" ``` ### 4. App using `TouchableOpacity` [#4-app-using-touchableopacity] ```tsx Submit Interest // click id = "submit_interest"; agent stableId anchors on testID ``` ### 5. App with custom screen matching [#5-app-with-custom-screen-matching] ```tsx useEffect(() => { setScreenMatching(SCREEN_MATCHING_ALL); }, []); ``` ### 6. App using Action Intelligence (naming for the agent) [#6-app-using-action-intelligence-naming-for-the-agent] ```tsx Submit ``` ### 7. App requiring manual visibility / call control [#7-app-requiring-manual-visibility--call-control] ```tsx const start = async () => { if (!(await startCall())) alert('Cannot start here'); };