Skip to Content
🚀 Orga AI is in open beta.
ExplanationConceptsWebRTC model

WebRTC model

Orga’s real-time experiences ride on WebRTC because it provides sub‑second audio/video latency, resilient NAT traversal, and bi‑directional data channels without forcing you to run SFU infrastructure. This page explains how your backend, client SDKs, and Orga Cloud cooperate to make that possible.

Three-party architecture

ActorResponsibilitiesKey artifacts
Your backend (Node SDK or REST proxy)Keeps the long-lived API key, exchanges it for ephemeral credentials, and hands ICE servers to clients.API key (sk_orga_ai_*), ephemeral token, ICE server list
Client SDK (React / React Native)Requests session config from your backend, opens the local media devices, and negotiates the peer connection.fetchSessionConfig, RTCPeerConnection, media tracks, DataChannel events
Orga CloudIssues short-lived credentials, hosts TURN infrastructure, and runs the multimodal AI model that joins the call.Ephemeral token validator, TURN/STUN cluster, AI session runtime

The clean separation ensures API keys never leave the server while still letting clients negotiate directly with Orga for media transport.

Credential lifecycle

  1. Permanent API key – Stored only in your backend environment (ORGA_API_KEY).
  2. Ephemeral token – Backend calls POST /v1/realtime/client-secrets with the API key. Token TTL is minutes, single-session.
  3. ICE configuration – Backend immediately calls GET /v1/realtime/ice-config with the ephemeral token to fetch TURN/STUN servers that align with the token’s lifespan.
  4. Session config response – Backend returns { ephemeralToken, iceServers } to the browser/mobile client.
  5. Client usage – Client uses the token as Authorization: Bearer for POST /v1/realtime/calls when submitting its SDP offer.

Because tokens expire quickly, most apps request them on demand (right before startSession()), not at login time.

Media negotiation timeline

  1. Local device prep – Client SDK prompts for mic/camera access and gathers ICE candidates.
  2. Offer creation – Browser/mobile builds an SDP offer containing codecs, media lines, and gathered candidates.
  3. Call creation – Client POSTs { offer, params } to /v1/realtime/calls using the ephemeral token.
  4. Answer & conversation id – Orga returns an SDP answer plus conversation_id. Client sets the answer as the remote description.
  5. Data & media flow – Audio/video RTP flows peer-to-peer (with TURN fallback). Text and telemetry events travel over the WebRTC data channel.
  6. Session updates – While connected, clients may call updateParams (SDK) or send DataChannel messages to adjust model, voice, or instructions without renegotiating.

Reliability patterns

  • TURN-first fallback – Always include the TURN credentials from /v1/realtime/ice-config; corporate firewalls often block direct UDP.
  • Token freshness – If negotiation fails with invalid_ephemeral_token, fetch a new token before retrying; tokens cannot be reused across concurrent attempts.
  • Observability – Enable logLevel: 'debug' in development to correlate SDK logs with backend logs. In production, log the conversation_id so you can trace issues inside Orga’s session tooling.
Last updated on
ExplanationSDKs