How to use: paste this entire document into your AI coding agent (Copilot, Cursor,
Claude Code, etc.) inside the Flutter app you want to integrate. The agent will inspect
your app, install and configure embed_flutter, ask you the few decisions only you can
make, then validate the integration end-to-end and report.
Every rule below is derived from the SDK's source (v0.4.0). Where this prompt and a
generic Flutter tutorial disagree, this prompt is correct for this SDK. Flutter is
materially simpler to integrate than the React Native SDK: there is no native
bootstrap code (no MainApplication/AppDelegate edits, no registerGlobals, no
Babel/Metro/worklets/Gesture-Handler setup) — the livekit_client plugin registers its
own native side.
You are an expert Flutter integration agent. Take this application from
zero → working → validated RevRag SDK integration with minimal manual work.
Prime directives
Inspect first, modify second. Never assume Flutter/Dart version, navigation
library, state management, or native setup. Read the project.
Preserve host behavior. Merge into existing config; never blindly overwrite
AndroidManifest.xml, Info.plist, Podfile, main.dart, or the root widget.
Never claim success without running the validation in §16–§21 and observing the
log lines named there.
Ask the developer for the decisions in §13 — do not guess them.
Never hide a crash or error. The SDK ships its own crash-guard that suppresses
only LiveKit/WebRTC async errors (§21); everything else must surface. Report
SDK-side issues you cannot fix.
Do not add dependencies, permissions, or config the SDK does not require.
a build constant, --dart-define, CI-stamped value, or package_info — the SDK does NOT auto-derive it (§10)
Platform targets
android/app/build.gradleminSdkVersion; ios/Podfileplatform :ios; is web/desktop a target? (unsupported, §2)
VALIDATE: you can state, in one sentence each, the navigation type, the runApp
location, whether a login gate exists, and where the app version comes from.
Supported: Flutter ≥ 3.29 with Dart ≥ 3.7 (pubspec.yaml declares
sdk: ">=3.7.0 <4.0.0", flutter: ">=3.29.0"). The floor is real, not cosmetic: the SDK
reads SemanticsData.role, which first shipped in Flutter 3.29. Android
minSdkVersion ≥ 21 and compileSdk 34+ (floors from flutter_webrtc/livekit_client,
not the SDK's Dart), iOS deployment target 13.0. Web and desktop are not supported
(LiveKit mobile stack) — Android + iOS only. Reference-tested on Flutter 3.41.x.
Target
Floor
Notes
Flutter
≥ 3.29
Needs SemanticsData.role (3.29). Below this, flutter pub get refuses to resolve ^0.4.0.
Dart
≥ 3.7 (< 4.0)
Bundled with Flutter 3.29+.
Android
minSdk 21, compileSdk 34+
Non-negotiable WebRTC floor.
iOS
13.0
platform :ios, '13.0' in the Podfile.
Web / desktop
unsupported
Android + iOS only.
If any platform target is below floor: explain, raise it, re-validate. Do not
upgrade/downgrade unrelated dependencies.
If the host's Flutter is older than 3.29 (check flutter --version and the
environment: block): do not silently upgrade the host's Flutter toolchain — that is a
project-wide change. Ask the developer which they want:
Upgrade Flutter to ≥ 3.29 (recommended; 0.3.0+ has the result-accuracy, screen-naming
and release-build fixes in §14/§17/§19–§21, and 0.4.0 the call-safety fixes), then use
embed_flutter: ^0.4.0; or
Stay on embed_flutter: 0.2.0 (pin exactly — embed_flutter: 0.2.0, not ^0.2.0,
which can never resolve to 0.4.0 anyway). 0.2.0 supports Flutter ≥ 3.0 / Dart ≥ 3.0 and
the same integration API (embedInitialize, EmbedWidget, EmbedNavigatorObserver,
USER_DATA). The 0.3.0+ behaviour described in this prompt (route-name-first
screen naming, the new failure codes, the new wire fields, and everything new in 0.4.0)
does not apply on 0.2.0 —
say so in your report.
VALIDATE:flutter pub get resolves with no version-solve error. If it fails on
livekit_client / flutter_webrtc / permission_handler / lottie, an existing host
pin is conflicting — reconcile the range (§4); do not fork the plugin.
There is nothing to configure.embed_flutter pulls livekit_client
(→ flutter_webrtc), permission_handler, shared_preferences, path_provider, http,
and lottie transitively; pub's solver shares one copy of each. Unlike React Native,
there is no Babel plugin, no Metro config, no worklets, no Gesture-Handler root view,
and no registerGlobals.
Verify a single resolved copy of the native stack:
Expected: exactly one resolved version of each. A second copy of livekit_client
pulled in by the host is a version-solve failure, not a silent double-runtime — reconcile
the range.
Permissions: add nothing. From 0.4.0 embed_flutter is a Flutter plugin whose own
AndroidManifest.xml declares every permission a call needs (INTERNET,
ACCESS_NETWORK_STATE, RECORD_AUDIO, MODIFY_AUDIO_SETTINGS, WAKE_LOCK,
BLUETOOTH_CONNECT, and BLUETOOTH up to Android 11); it is merged into the host at build
time. Do not add them to the host manifest (prime directive 6). Entries the host
already has are harmless — leave them. The SDK requests the runtime ones itself: the mic at
call start, and BLUETOOTH_CONNECT (Android 12+) only when a Bluetooth headset is
connected; a Bluetooth denial never blocks the call.
No MainApplication / MainActivity code is required. The plugin auto-registers;
MainActivity stays a plain FlutterActivity. (There is no
audioRecordSamplesDispatcher-style manual-setup trap as in RN.)
android/app/build.gradle: minSdkVersion 21 (or higher), compileSdkVersion at
the Flutter default (34+). The mic runtime prompt is done by the SDK during
startCall via permission_handler.
Build: flutter build apk --debug. On failure decide SDK-related vs pre-existing;
fix; rebuild. Never suppress errors.
VALIDATE: APK builds; the merged manifest
(build/app/intermediates/merged_manifests/.../AndroidManifest.xml) contains
RECORD_AUDIO and BLUETOOTH_CONNECT (from embed_flutter) without any host edit.
ios/Runner/Info.plist — NSMicrophoneUsageDescription is mandatory (iOS
kills the app on a mic access without it):
<key>NSMicrophoneUsageDescription</key><string>This app needs microphone access for voice calls.</string>
ios/Podfile — set the platform and add the permission_handler macro. The SDK
requests the mic through permission_handler (Permission.microphone.request()), so
PERMISSION_MICROPHONE=1 is required — without it the dialog never appears and the
permission is always reported denied:
platform :ios, '13.0'post_install do |installer| installer.pods_project.targets.each do |target| flutter_additional_ios_build_settings(target) target.build_configurations.each do |config| config.build_settings['GCC_PREPROCESSOR_DEFINITIONS'] ||= [ '$(inherited)', 'PERMISSION_MICROPHONE=1', ] end endend
Then cd ios && pod install.
No AppDelegate.swift edits are required — there is no LivekitReactNative.setup
analog.
The bundled examples/BajajFinance/ios/Podfile omits both the macro and the
platform :ios, '13.0' line — do not copy it verbatim, or iOS mic requests are
auto-denied and the WebRTC pods may fail to build below iOS 13.
flutter pub getdart analyze # host app must stay at its own baselineflutter build apk --debugflutter build ios --no-codesign
VALIDATE: both platform builds compile. A red analyzer error introduced by the
integration (e.g. a wrong embedInitialize signature) must be fixed here, not deferred.
Call embedInitializeonce, before runApp. All parameters are named; apiKey
and appVersion are required:
import 'package:embed_flutter/embed_flutter.dart';void main() { embedInitialize( apiKey: API_KEY, // REQUIRED (named), from the app's config/env appVersion: APP_VERSION, // REQUIRED (named) — see §10 embedUrl: EMBED_URL, // optional; RevRag-provided host, defaults to prod onResult: (bool success, String? error) { if (!success) debugPrint('Embed init failed: $error'); }, ); runApp(const MyApp());}
embedInitialize is the only place credentials and app version enter the SDK.
There is no EmbedProvider widget and no positional apiKey —
embedInitialize('key', ...) will not compile.
It is synchronous and fail-open: it never throws at your call site. Do NOT block the
app on init; render the app — the widget simply won't appear until config arrives.
It installs the host-crash safety net (EmbedCrashGuard, §21) before any LiveKit work.
Then wrap the tree — EmbedWidget must be the outermost widget (above MaterialApp,
above any portal/overlay/theme wrapper, or the FAB and highlights end up underneath):
VALIDATE: the call compiles with named apiKey: / appVersion:; onResult fires.
On the first run with a valid key you see [ApiService] ← GET /initialize status=200 and
(debug builds only) [Embed][ui_config] … full payload: {…}.
The key is sent as the X-Revrag-Embedded-Key header on GET /embedded-agent/initialize. A wrong or missing key still compiles and runs — the fetch
returns non-200, so the SDK logs [ApiService] ← GET /initialize status=<code> then
[ApiService] ⚠️ /initialize returned non-200 (<code>) — UI config is NULL, widget will use SDK defaults; onResult fires (false, "Failed to fetch widget configuration and no cached configuration was available.") (when nothing is cached), and the agent button
never appears.
Do not hard-code secrets in source if an env system exists. If missing → stop,
report "API key missing", ask the developer.
VALIDATE: with the real key you see [ApiService] ← GET /initialize status=200 and
onResult(success=true). A non-200 (401 / 403) means the key or header is wrong — do
not proceed to §11 until fixed.
appVersion is required in embedInitialize and is the only intake — there is
no auto-derivation from package_info and no post-init setter. It is sent as
?app_version=<value> on /initialize and attached to every outbound event in the
sdk block as app_version.
Derive it from a single source of truth (a build constant, a --dart-define, or the
value your CI stamps). Never pass "" or a stale literal.
VALIDATE: the /initialize URL query shows ?app_version=<your value>, and outbound
event payloads carry the same value in the sdk block.
The agent stays hidden and no call can connect until the SDK knows the user. After
login, send:
import 'package:embed_flutter/embed_flutter.dart';embedEvent( EventKeys.USER_DATA, UserEventPayload( app_user_id: userId, // required — your stable user id data: {'name': 'Priya', 'plan': 'gold'}, // optional context ),);
Facts the integration must respect:
Place it immediately after auth succeeds. For a no-auth/demo app, send it in the first
screen's initState.
Without it, a call's token fetch is skipped — the SDK logs
[ApiService] token request skipped: no app_user_id yet. Send a USER_DATA event … after login, before the agent is expected to connect. — so the call cannot connect.
No need to wait for config: with a user set, events go out whatever the widget is doing
(§18). Before any user is set, events are not sent and embedEvent returns false.
The user is kept across app restarts (0.4.0) until logout. Wire
await EmbedWidget.clearUserSession(); into the host's logout flow — it ends any call,
wipes the SDK's screen cache and form values, and forgets the user (also with no widget
mounted). Skipping it leaves the next person identified as the previous user.
embedClearStorageCache() also logs out. Re-send USER_DATA for the next user.
embedEvent returns a Future<bool> and never throws.
Required order: embedInitialize → init success → USER_DATA → widget config → FAB shown
on an enabled route → interactions → call.
The FAB renders only after/initialize returns a widget_config (position, colors,
avatar, collapsed view, inactivity nudge). It is dashboard-driven — there are no
host props for position or colors. The SDK merges it and type-coerces every field, so a
malformed value falls back to a default (it never crashes a build).
Case
Expected
Observe
A. config exists
FAB appears on allowed routes; agent_visible fires
[Embed][ui_config] … full payload: {…}
B. config null / {}
default styling (black button, #B391F3 gradient), no crash
onResult(false, …); [ApiService] ⚠️ /initialize returned non-200
D. delayed config
app must not assume config is ready; FAB appears when it lands
on a cold start the FAB may need the next route change to mount
Also confirm the native cache store: on-device UI-graph writes log under the [EmbedCache]
tag; [EmbedCache] store unavailable, memory-only: … means the file store failed and the
cache is in memory only.
After inspecting, list the actual route names you found (the names the navigator
reports — e.g. home, checkout; not display titles), then ask:
Which routes should the RevRag widget appear on? (Offer: all-except via
showOnAllRoutes: true + disabledRoutes, or an explicit enabledRoutes list.)
Which routes must never show the agent / must end a call on entry (login, OTP,
payment, regulated data)? → disabledRoutes (entering one ends a live call). This
is also the only way to stop the agent's missions from looking at a screen (§17).
Ask too which dialogs / bottom sheets are sensitive (OTP, card entry): give them a
RouteSettings(name:) and list that name — it blocks capture while open but does not
hide the FAB.
Should a live call survive navigation between agent routes? →
continuity: EmbedButtonContinuity.keepVisible vs reset (default) (§15).
Do different route groups need different show/delay behavior? →
buttonVisibility groups (EmbedButtonVisibilityConfig, §15).
App version source if it can't be inferred; API key location.
Any WebView-only, native-picker, or custom-navigation flows (see §22).
Wait for answers before finalizing screen integration.
Register the observer; EmbedWidget wraps MaterialApp. Detection rules by navigator:
Named routes / onGenerateRoute — register EmbedNavigatorObserver() on
MaterialApp.navigatorObservers. Always pass settings: settings in onGenerateRoute
so routes have names.
GoRouter — put EmbedNavigatorObserver() in GoRouter(observers: [...]) and use
each route's name: (not path:) in enabledRoutes.
Tabs / ShellRoute / BottomNavigationBar / IndexedStack — tab bodies live in a
nested navigator the top observer can't see. Wrap each tab body in
EmbedRouteListener(routeName: 'home', child: …) whose routeName matches
enabledRoutes.
Anonymous pushes — a MaterialPageRoute with no settings reports a null name
and is invisible. Give it settings: RouteSettings(name: '…'), or supply a custom
EmbedNavigatorObserver(nameExtractor: (route) => …).
Matching — routeMatchMode is exact (default), startsWith, or contains
(RouteMatchMode). Leading slashes are normalized ('home' == '/home'). This is the
analog of RN's screen-matching tiers.
Named popups — dialogs / bottom sheets opened with
routeSettings: RouteSettings(name: '…') never change the page route. Listing such a
name in disabledRoutes blocks capture (missions and telemetry) while it is open,
but does not hide the FAB — the button follows the page underneath. Named popups
still emit SCREEN_STATE route-change analytics on open/close.
Separately from FAB visibility, every UI capture is filed under a screen name. It keys
the on-device UI-graph cache, prefixes every element's stable ID, and is how the agent
knows a tap changed screens. 0.3.0 resolves it in this order:
The current route name — RouteSettings.name, your nameExtractor result, or the
name set by EmbedRouteListener(routeName:). If a tab shell (IndexedStack /
PageView) is in front, the active tab is appended: route/tab (e.g.
home/portfolio), so every tab of a single-route shell gets its own screen.
Stable widget-tree signals only — used for the tab part, and as the whole name
when the top page is unnamed:
IndexedStack tab, in order: a ValueKey<String> on the tab's root widget, a
Semantics(identifier:) at the top of the tab, then the tab's first heading
(Semantics(header: true) — an AppBar title);
PageView(children: …) page: a ValueKey<String> on the page child only;
unnamed top page: its first heading.
A heading only counts when it is short (≤ 4 words, ≤ 32 chars) and digit-free. Names
are slugged ("My Orders" → my_orders).
Unnamed pushed pages with no stable signal are named by their depth above the last
named route: route/page_N (e.g. checkout/page_1) — they never inherit the page
below them.
Otherwise the active visibility group, then unknown_flow.
Class names are never used — not in debug, not in release — so debug, release and
--obfuscate builds produce identical screen names, stable-ID prefixes and cache keys.
What to ask / fix in the host (in priority order):
Name every route — settings: RouteSettings(name: '…') on every
MaterialPageRoute / onGenerateRoute result, name: on every GoRoute.
route/page_N in the logs means an unnamed push: name it.
Give each tab root a ValueKey<String> in IndexedStack / PageView shells
(IndexedStack(children: [HomeTab(key: ValueKey('home')), …])), or wrap tab bodies in
EmbedRouteListener. Do not rely on headings for tabs whose title carries a number,
a greeting or a user's name — those are rejected as names.
Never derive names from runtimeType / toString() in a custom nameExtractor —
release builds strip toString() of Flutter types and --obfuscate renames classes.
VALIDATE:EmbedWidget(debug: true) turns on the SDK's verbose logging (EmbedLog),
which narrates config load, route changes, and FAB-visibility decisions; navigate to an
enabled route and confirm the FAB appears (after USER_DATA). For screen names, set
WireLog.enabled = true in a debug build (it is forced off in release) and check that
each [Embed ▶ SEND] type=ui_diff … screen=<name> line shows a meaningful route name (or route/tab),
not unknown_flow or …/page_N. Switch every tab and confirm each gets a distinct name.
Always hidden; the call ends when the FAB hides here. Also the only list that keeps the agent's missions from looking at a screen (§17). A named popup listed here blocks capture while open but does not hide the FAB.
continuity: EmbedButtonContinuity
keepVisible = FAB (and call) stay up across enabled routes; reset (default) = re-show/re-delay each route.
Visibility groups — {defaultDelayMs, groups}, each group {id, screens, continuity, delayMs?, delayPolicy?}; first match wins (see INTEGRATION.md §7.2).
Behavior to verify:
Under showOnAllRoutes: true, a call is continuous across screens — it is not
disconnected on each navigation. Add a route to disabledRoutes to force a disconnect
there (RN's endCallWhenHiddenByVisibility is automatic in Flutter: leaving the
allow-list ends the call).
A group's screens join the allow-list — a route listed in a buttonVisibility
group shows the FAB even if it is not in enabledRoutes.
VALIDATE: navigate across two enabled routes with continuity: keepVisible during a
live call — the call persists; navigate to a disabledRoutes route — the call ends
(agent_conversation_ended).
Confirm this order in the logs on a healthy first session, and that nothing fires out of
order:
[ApiService] ← GET /initialize status=200 → [Embed][ui_config] … full payload
USER_DATA sent (after login) → token fetch unlocked
agent_visible on an enabled route
on FAB tap / startCall: [CallTiming] t=0ms startCall() invoked → [CallTiming] t=<ms>ms … → ROOM CONNECTED — call is live → agent_conversation_started (carries a
flat call_id)
on end: agent_conversation_ended (same call_id)
Invalid patterns to catch: a call before USER_DATA ([ApiService] token request skipped: no app_user_id yet …), agent_visible never firing (re-check §11 and §14), a second
EmbedWidget in the tree.
For the agent to act (tap/scroll/type), it reads your Flutter Semantics tree — the
same data screen readers use. Mission frames arrive on the LiveKit data channel; element
highlights log [Embed HIGHLIGHT] pulse stableId=…; the durable on-device UI-graph writes
under [EmbedCache]. To see the wire traffic itself, set WireLog.enabled = true in a
debug build (e.g. WireLog.enabled = kDebugMode; before runApp): you then get
[Embed ◀ RECV] type=mission …, [Embed ▶ SEND] type=ui_diff … screen=…, and
[EmbedWidget] 📩 LiveKit DataReceived … headers. WireLog is forced off in release
builds whatever the host sets, and inbound set_text values are redacted in the debug log.
Deep guidance lives in HOST_INTEGRATION.md; the essentials:
Icon-only buttons need Semantics(label: …, button: true, …). Without a label the
SDK falls back to a name from what the control draws — an image's asset file name
(ic_home_active.png → home), a common Material/Cupertino icon's literal name
(Icons.pie_chart → pie chart), or, for an unlabelled tab, Tab 2 of 4. These are
fallbacks: a real label always wins, and "pie chart" is not "Portfolio".
Chips / toggles / custom radios / custom tabs must surface selection
(Semantics(selected: …) / Semantics(checked: …), or use ChoiceChip / FilterChip /
Radio / Switch). Without it the SDK infers selection — a custom tab row reports
selected only when exactly one item is drawn differently (else from the tab the session
saw lead to the current screen); a custom option group whose items each draw a small
round indicator becomes role radio with checked only when exactly one has a filled
dot. Explicit semantics always win; inference returns nothing when unsure.
Custom bottom-navigation rows (≥ 3 short tappables aligned in the bottom ~20% of the
screen, not inside a scrollable, not a Material button bar, grid or keypad) are reported
with role tab and kept first under the 15 KB snapshot budget.
Critical CTAs (login, pay, submit) should carry
Semantics(identifier: 'checkout.pay') — the SDK matches identifier first.
Disabled controls (onPressed: null, TextField(enabled: false)) are captured with
disabled: true; a tap or set_text on them fails at once with target_disabled.
Secure fields (TextField(obscureText: true) → SemanticsFlag.isObscured) are
redacted at capture and stay redacted (sticky) — typed secrets never leave the
device in the UI snapshot (secure: true, no value).
Who may look (0.3.0). Agent missions (find / tap / scroll / read / verify and
their responses) are blocked only by explicit host exclusions: disabledRoutes
(matching the current route or the detected screen name) and an open named popup listed
there. action_config.flags.dataCapture: false, a hidden FAB, and a route missing from
enabledRoutes / the platform allowlist do not blind a running mission —
dataCapture: false stops telemetry only (live ui_diff pushes, the
mission_exploration audit, the UI-graph cache). Telemetry also continues while the FAB
is hidden (flags.transmitOnHiddenWidget defaults on; only an explicit false turns
it off). A refused mission response is replaced by a failure frame
(screen_not_capturable / payload_too_large) — never dropped silently. Host query:
EmbedWidget.isWidgetTreeSnapshotEnabled.
What a healthy node looks like on the wire (one entry of a ui_diff baseline's
nodes; keys are omitted when empty/false):
{ "id": "checkout.group_0.full_name", // stable ID, prefixed by the screen name "role": "input_field", // Android vocabulary, see below "text": "Full Name", "bounds": [16, 220, 376, 276], // logical px "visible": true, // ≥ 5% of the node on screen "visible_fraction": 1.0, "has_action": true, "parent_id": "checkout.group_0", // nearest kept ancestor "actions": ["tap", "setText"], // setText on every writable field "editable": true, "placeholder": "As on PAN card", // input hint (editable nodes only) "disabled": false // measured; absent when unknown}
Other 0.3.0 node keys: range: {min, max, step?} on sliders when the host provides
bounds, selected: true, checked: true|false|"mixed", secure: true, identifier,
layer (> 0 inside a modal / sheet), subtree_text. Roles use Android's vocabulary:
button, card, text, heading, input_field, checkbox, radio, switch,
slider, link, image, tab, navigation, listitem, row, group, scrollable,
modal, overlay, none — raw Flutter role names never appear. Snapshot frames also
carry app_user_id (after USER_DATA), app_version and sdk.
A healthy snapshot for a key screen: the screen name is the route name (or
route/tab); every tappable the user would name has a non-empty text (icon-only items
labelled); the active tab / chip / radio has selected / checked; text fields carry
setText and their placeholder; disabled CTAs carry disabled: true; password/OTP
fields carry secure: true and no value; nothing is called unknown_flow.
What the tree cannot see: anything with no Semantics (raw CustomPaint, some
third-party UI kits), platform views / WebViews, and native pickers.
VALIDATE: turn on TalkBack/VoiceOver and tab through a key screen — anything the reader
can't name is invisible to the agent. Missions failing with target_not_found point here;
ambiguous_target means two controls carry the same label (give them distinct labels or
identifiers). A screen file over the cache upload budget logs
[EmbedCache] screen "…" is <N> bytes, over the <M> batch budget; sending alone.
Automatic capture. The SDK reports taps on every tappable control (buttons,
GestureDetector, InkWell, ListTile, checkboxes, switches, radios, tabs) by finding the
tapped node in the Semantics tree, and watches TextField controllers for text changes — all
gated by the dashboard clickTracking flag (ActionGates.clickTrackingEnabled), only while
the widget is active (EmbedWidget.isActive) and only once a user is set. Click
event_names follow the convention shared with Android, RN and web (0.4.0):
<name>_<suffix> (pressed / toggled / selected / opened), no screen suffix (the
screen is in context.screen). <name> = the control's ValueKey<String> →
Semantics(identifier:) → tooltip / visible text → the image or icon it draws → role. Rapid
taps (3 within 1 s) emit a rage_click analytics event.
The agent's targeting id (stableId) is separate from analytics: it resolves
Semantics(identifier:) → text → role. Put a stable identifier on every meaningful
control:
Observe agent events with embedOnAgent(...) / SdkEventName (see INTEGRATION.md §9).
rage_click and screen_view are analytics only — they don't reach embedOnAgent; use
embedOn for them.
VALIDATE: with USER_DATA in place, events are sent immediately and return true /
2xx whatever the widget is doing (hidden, delayed, excluded route, config still loading).
Before any user is set they are not queued: embedEvent returns false. Failed
analytics are retried one by one from a persisted queue (at most 100).
Preconditions the SDK enforces: config loaded (FAB present), mic permission granted (the
SDK prompts via permission_handler; denied → EmbedWidget(onPermissionStatusChanged:)
gets false + microphone_permission_denied), and USER_DATA stored (else the token
fetch is skipped, §11). Call start is single-flight (startCall no-ops while already
active/connecting).
Programmatic API (see INTEGRATION.md §10):
await EmbedWidget.startCall(); // co-pilot (default)await EmbedWidget.startCall(agentTriggerMode: AgentTriggerMode.WORKFLOW);await EmbedWidget.endCall();EmbedWidget.isCallActive; // boolEmbedWidget.collapseWidget(); EmbedWidget.expandWidget(); // bar <-> FABawait EmbedWidget.pauseAgent(); await EmbedWidget.resumeAgent(); // keep the call, mute the agentEmbedWidget.isAgentPaused; // boolawait EmbedWidget.clearUserSession(); // on logout
startCall() does not auto-expand the bar; pair with expandWidget() if you start
from your own UI.
AgentTriggerMode: CO_PILOT (default — agent drives your screens, panels
dropped) vs WORKFLOW (Revrag-owned panel/avatar surface).
pauseAgent() / resumeAgent() / isAgentPaused (0.4.0; also embedPauseAgent() /
embedResumeAgent() / embedIsAgentPaused()): the agent stops speaking and listening and
the mic is muted; resume restores the mic. They return false with no call or no agent
answer.
Not present in the Flutter SDK (unlike RN / Android): isWidgetExpanded. Do not
reference it.
A user touch on the host app while the agent is acting cancels that action
(failure_reason: user_interrupted).
VALIDATE (real device):startCall() → [CallTiming] t=0ms startCall() invoked →
ROOM CONNECTED — call is live → agent_conversation_started → isCallActive true →
endCall() → agent_conversation_ended. Then a second call starts cleanly.
During a co-pilot call the backend sends mission frames (one or more steps: click,
navigate, set_text, scroll_to, check, select, set_slider, back, …) and the SDK
answers on the data channel. To read them, run a debug build with
WireLog.enabled = true and filter on [Embed (bodies longer than ~900 bytes are split
into numbered [Embed body#N i/n] chunks — reassemble them). Per step you will see
mission_phase (searching → executing → verifying, plus disambiguating with
candidates), one verification_result, and runner lines such as
[Embed VERIFY mid=… step=… success=… evidence=[…] reason=…]. Every mission ends with
exactly one terminal mission_status: succeeded, failed or cancelled.
0.3.0 result honesty. A step is succeeded only when the intended target was acted
on and verification supports it; otherwise it is failed with a specific
failure_reason (the terminal vocabulary itself is unchanged). How to read the frames:
Field
Where
Meaning
step_index
every terminal mission_status (on succeeded: the last step)
Which chained step ended the mission.
steps_completed / steps_total
terminal mission_status
How far a multi-step mission got.
steps_skipped
terminal failed / cancelled
Steps that never ran.
steps_failed
terminal, continue_on_failure missions
Steps that failed while the mission kept going (each also emits step_failed).
requested_stable_id / requested_text
verification_result; mirrored on failure frames
What the step asked for (requested_text is the label, ≤ 60 chars, never typed text, omitted for obscured fields).
acted_stable_id / acted_role
verification_result (acted_stable_id mirrored on failure)
What was actually acted on — absent when nothing was. Compare with requested_*.
match_source
verification_result
How the target was found (stable_id, identifier, text_exact, text_token, redirect_label, twin, role, bounds, …).
verified
verification_result
The verdict rests on an observation of the intended target.
value_matched
verification_result, set_text only
The field's value matches the request — a boolean, never the value itself.
redirected
verification_result, set_text only
The text went to a field bound to the matched label.
value / strategy
verification_result, set_slider
The achieved slider value and how it was set.
Second look (observation only). When a dispatched tap / click / navigate / back
cannot be confirmed right away, the SDK waits ~1.5 s, re-captures once and re-runs
the verifier (log: [Embed SECOND_LOOK mid=… confirmed=… (no re-dispatch)]). A late,
network-driven update then becomes a verified success. The action is never pressed a
second time; still unconfirmed → the original failure is reported. A tap with no visible
effect is likewise never retried, so there are no double side effects (double payments,
double submits).
Optional step param params.expect_screen (navigate): when the backend sends it, the
landed screen must match it or the step fails target_mismatch — which makes stable
screen names (§14.1) matter.
VALIDATE (real device, debug build): ask the agent to do one tap, one navigation and
one text entry on a key screen. Each must end succeeded with verified: true and
acted_stable_id equal to (or clearly the same control as) requested_*. Any failed
frame: read its failure_reason in the §20 mission table and fix the host side it points
to before declaring success.
Non-200 (e.g. 401/403), UI config NULL, onResult(false, …), FAB hidden, no crash.
appVersion blank/stale
/initialize still fires with whatever was passed; supply a real value (§10).
USER_DATA never sent (and no user kept from a previous session)
FAB never shows; token fetch skipped (token request skipped: no app_user_id yet); events are not sent and embedEvent returns false.
Mic permission denied
onPermissionStatusChanged(false) / microphone_permission_denied; no crash.
Token fetch fails when the user taps Start
The FAB stays and shows "Couldn't connect — tap to retry"; tapping again retries. A failed pre-call prep at sign-in is retried automatically (backoff, up to 5 times).
User leaves for a hidden screen while a call is still connecting
The call is cancelled; it never goes live without a visible End button.
Network switch mid-call (full LiveKit reconnect)
The call continues; a muted or paused mic stays muted, and a running mission is not dropped.
Bluetooth headset connected / disconnected mid-call
Audio follows the headset, then returns to the user's speaker choice; the SDK never forces the loudspeaker over a headset.
Backend widget_config = null/{}
SDK uses default styling; FAB still works.
Wrong-typed backend config field
Coerced to a default; no CastError.
Navigate to disabledRoutes mid-call
Call ends cleanly.
Bad embedEvent payload type
Returns false (does not throw).
Forced LiveKit renegotiation error
Suppressed by the crash-guard; host process survives (§21).
Backend sends action_config.effects.*Ms: 0
Clamped to 1 ms — highlight / border / awakening animations run imperceptibly, no crash.
Backend sends flags.nativeBridge: false
Only the synthetic-pointer tap fallback is off; accessibility-action taps still work.
Agent asked to act on a disabledRoutes screen / an open excluded named popup
Nothing on that screen is captured or sent: the mission sees no screen, and a response that would carry its content is replaced by a screen_not_capturable failure frame. The mission still ends with a terminal status — it is never left hanging.
Mission failure codes — how to read them (0.3.0). These arrive as failure_reason on a
failedverification_result / terminal mission_status (§19.1). Most point at a host
fix; none is a crash.
failure_reason
What happened
Nothing acted on?
Host-side fix
target_not_found
No element matched, even after scroll-and-recapture.
yes
Label the control (Semantics(label:)), surface it in Semantics, check the screen is the one the agent expects (§17).
ambiguous_target
Two comparable controls match equally well (candidates ride the disambiguating phase). Side-effect verbs refuse to guess.
yes
Give duplicates distinct labels or Semantics(identifier:).
target_mismatch
The only reachable element names something else — a field labelled "PIN" for "Full Name", "Nominee Name" for "Name", a card whose only button isn't the request, or expect_screen not reached after a navigate.
yes (except the navigate expect_screen case)
Make each input's label sit in its own container / directly above or left of the field; label fields explicitly; name routes (§14.1).
target_disabled
The target is measured disabled (onPressed: null, enabled: false).
yes
Expected behaviour — the control really is disabled.
verification_unavailable
The action could not be verified (no capture after it, an unreadable obscured field, or — evidence unattributed_change — something changed elsewhere on screen that can't be tied to the target). Not success and not "nothing happened".
no (dispatched)
Make the effect visible on the target: a state change on the tapped control, its option group or its card, a route change, or an overlay. Surface selection with Semantics(selected:/checked:).
Dispatched; even after the ~1.5 s second look nothing tied to the target changed. Never retried.
no (dispatched)
Check the handler actually fires for semantics taps (a bare GestureDetector needs Semantics(button: true, onTap: …)); a change only in a counter / badge / status line outside the tapped card does not count.
back_had_no_observable_effect
Back was reported handled but nothing popped (e.g. a PopScope veto).
no
Wire EmbedWidget(onBackRequested:) for custom back handling.
check_not_applied / select_not_applied
The checkbox/switch or chosen option is not shown in the new state afterwards.
no
Expose Semantics(checked:) / Semantics(selected:) on the custom control.
field_has_no_text
read_field on an empty field (no longer answered with the label).
The SDK is engineered so nothing it does can crash the host (INTEGRATION.md §12):
public API returns safe defaults, every timer/listener/post-frame is guarded, config is
type-coerced, and a chained PlatformDispatcher.onError (installed by embedInitialize)
swallows onlylivekit_client/flutter_webrtc async errors (e.g. a renegotiation
NegotiationError), delegating everything else to the host's handler untouched.
Grep your crash reporting (Sentry / Crashlytics / logs) for these real SDK lines:
[EmbedSDK] suppressed voice-stack async error — the guard working as intended
(informational; a LiveKit/WebRTC error was kept from crashing the host).
[ApiService] ⚠️ /initialize returned non-200 / [ApiService] ⚠️ /initialize fetch FAILED — config load failed; FAB won't show (key/URL/network).
[ApiService] token request skipped: no app_user_id yet — USER_DATA not sent (§11).
[EmbedCache] sync rejected status=…; pass stopped (401/403 — credentials),
[EmbedCache] "…" rejected 3 times (…); quarantined until it changes (one screen file
the backend keeps refusing — the rest still sync), [EmbedCache] store unavailable, memory-only — UI-graph sync/store problems (non-fatal).
Embed init failed (from your onResult) — surfaces config-load failures.
Release builds are quiet by design (0.3.0):WireLog and inbound-frame logging are
forced off in release whatever the host sets, the /initialize response body and the
[Embed][ui_config] … lines are printed in debug builds only, and debug logs redact
secrets (API keys, tokens, PINs/OTPs, passwords, phone/email) and inbound set_text
values. Mission-level lines ([Embed VERIFY …], [Embed SECOND_LOOK …], ▶ SEND /
◀ RECV) therefore exist only in a debug build with WireLog.enabled = true — do not
expect them in Crashlytics / Sentry from production. Tracker IDs no longer collapse to
Instance of … in release.
Native (platform) failures to classify — not SDK crashes: attempted to access privacy-sensitive data without a usage description (iOS NSMicrophoneUsageDescription
missing), a mic prompt that never appears (iOS PERMISSION_MICROPHONE=1 missing, §6),
uses-sdk:minSdkVersion … cannot be smaller than version 21 (Android floor, §5). Never
swallow a native crash; classify app-side vs SDK-side.
Work through every row. "Ask" = a developer decision.
UI / layout
EmbedWidget not outermost → a global overlay/portal/theme wrapper above it can
occlude the FAB and highlights; keep EmbedWidget above MaterialApp; verify on a
device.
Bottom sheets / dialogs / overlays drawn above the FAB → the agent's highlight and
the FAB can end up under them; verify z-order; surface the sheet with
Semantics(container: true) so the agent sees it as its own subtree.
SafeArea / keyboard insets → the FAB position and highlight bounds can drift near
the bottom while the keyboard is open; verify.
Components / design system (Semantics)
Icon-only buttons / icon-only nav items without Semantics(label:) → 0.3.0 names
them from what they draw (asset file name, a common Material/Cupertino icon's literal
name, or Tab i of n), which is the icon, not the meaning ("pie chart", not
"Portfolio"; "Tab 3 of 4", not "Orders"). Label them and set button: true.
Custom GestureDetector tappables with no semantics → not tappable by the agent;
wrap in Semantics(button: true, onTap: …) or use a Material button.
Selection shown by style only (custom radios/chips/tabs) → 0.3.0 infers it only
when exactly one item is drawn differently (radios: exactly one filled dot) or from tab
taps seen this session, and returns nothing when unsure — so check / select / tab
taps can end verification_unavailable / select_not_applied. Add
Semantics(selected:) / Semantics(checked:) (or use ChoiceChip/Radio/Switch).
Custom bottom-navigation bars (a Row of GestureDetectors) → reported as role
tab; without Semantics(selected:) the active tab is inferred as above. Mark it
explicitly: Semantics(label: 'Portfolio', selected: i == current, child: …).
Input labels far from their field (label in another column, or two labelled fields
stacked with one caption) → set_text refuses to guess and fails target_mismatch
rather than typing into the wrong field. Put each label in the field's own container
(or use InputDecoration(labelText:)).
CustomPaint / canvas UIs / third-party kits with no Semantics → invisible to the
agent (§17); wrap the meaningful bits.
Inputs / forms
Uncontrolled TextField (no controller) → harder to verify typed text and to detect
changes for auto-capture; prefer a TextEditingController.
Password / OTP fields → mark them obscureText: true so SemanticsFlag.isObscured
redacts them at capture (sticky). Redaction is per node — mark each secret field,
not the wrapper.
Read-only field used as a picker trigger → the agent may try to type into it; use a
tappable + Text instead.
Lists / performance
Lazy lists (ListView.builder, Sliver…) → off-screen rows aren't in the tree
(not captured) until scrolled into view; the find ladder scrolls-and-recaptures, but
give rows a stable Semantics(identifier:) so they're targetable when visible.
Very heavy screens → capture measures the tree; large trees can hit the cache byte
budget ([EmbedCache] screen "…" is <N> bytes, over the …) and get trimmed.
Navigation-adjacent
Anonymous routes — MaterialPageRoute without settings ⇒ no route name ⇒
invisible to the observer for FAB visibility, and its screen name falls back to its
heading or parentRoute/page_N (§14.1). Give every route a name.
--obfuscate / --split-debug-info release builds → 0.3.0 never uses class names,
so screen names, stable IDs and cache keys are identical to debug. Verify any custom
nameExtractor doesn't use runtimeType / toString() (obfuscated or stripped in
release).
GoRouter observer omitted — must be on GoRouter(observers: […]), not only on an
inner MaterialApp; use route name: not path:.
Tabs via IndexedStack / PageView with no ValueKey<String> on the tab roots, no
Semantics(identifier:) and no short heading → every tab shares the shell's route name
(one screen for all tabs, cache and stable IDs collide). Give each tab root a
ValueKey<String> (→ route/tab), or wrap each tab body in EmbedRouteListener (also
needed for per-tab FAB visibility).
Route names differing between enabledRoutes and EmbedRouteListener — they must be
identical strings.
disabledRoutes intent — a disabled route ends an active call; confirm that's
desired (e.g. a payment WebView screen).
Native / platform
Backgrounded calls (iOS) → the verified setup covers foreground calls. If a call
must keep running while the app is backgrounded, iOS additionally needs audio under
UIBackgroundModes in Info.plist; the SDK and example do not set it — confirm the
requirement and add it deliberately (don't assume it's handled). Ask.
Other audio-session users (audio players, video, VoIP) → the SDK drives the audio
session per call; a host that also owns it may conflict. Ask.
onPermissionStatusChanged not wired → an EmbedWidget callback prop; if the host
doesn't pass it, a mic denial is silent to the host UI (the SDK still doesn't crash and
still emits microphone_permission_denied).
Build / tooling
Transitive-dependency version clash — the SDK pins livekit_client,
flutter_webrtc, permission_handler, shared_preferences, path_provider, http,
and lottie. A conflicting host major (especially lottie / permission_handler)
fails the pub version solve; reconcile the range, don't fork.
Existing livekit_client / flutter_webrtc at a conflicting version → reconcile the
pub range; do not vendor a second copy.
Web/desktop build target → unsupported (§2); guard the widget out of those flavors.
Runtime / lifecycle
Host overriding PlatformDispatcher.onErrorafterembedInitialize → the host's
handler wins; make sure it re-chains, or the LiveKit crash-guard (§21) is lost.
Sentry/Crashlytics console capture → the guard's informational
[EmbedSDK] suppressed voice-stack async error line is expected, not a fault; don't
page on it.
Security / privacy
disabledRoutes ≠ no capture of other screens — the durable UI-graph cache is
written per screen while capture is enabled (dataCapture flag). Mark sensitive fields
(obscureText for passwords/OTP — a hard guarantee) and ask the developer about
regulated screens.
Store-listing disclosure — the app requests microphone (and Bluetooth) permissions;
disclose in the listing.