How Rapport Works (At a Glance)

The technical architecture behind Rapport for developers — how Rapport Studio, Session Runners, the processing pipeline, and deployment targets work together to deliver real-time AI character experiences.

How Rapport Works (At a Glance)

Rapport operates as two connected systems — a creative workspace for building and a real-time engine for delivery.

This split allows teams to collaborate efficiently while keeping live interactions fast, reliable, and scalable.

Where Rapport Comes From

Rapport was developed by Speech Graphics, a company renowned for its award-winning facial animation technology used by leading game studios including EA, Sony, and 2K. Originally a spin-out from the University of Edinburgh's School of Informatics, Speech Graphics pioneered audio-driven facial animation through detailed muscle simulation and phoneme-based lip sync. That same technology now powers Rapport.

The Two Main Components

1. Rapport Studio (Browser)

A browser-based environment where creators design and configure characters with a live preview.

From Rapport Studio, you can:

• Start from templates or build from scratch

• Pick characters, AI models, languages, and voices

• Define prompts, behaviour, and layout

• Manage projects, assets, and workspaces

💡 Runs entirely in your browser — no installation required.

When you first sign in, Rapport Studio automatically creates your Organisation and Workspace, taking you directly to the Template Grid — a visual starting point where you can launch a preconfigured project or begin from the developer path. Templates provide working setups that can be customised immediately within Studio.

2. Session Runners (Cloud / On-Prem)

The Session Runners form the real-time engine that powers every live interaction. They orchestrate communication between AI, voice, and animation services, ensuring each response feels immediate and natural.

Session Runners handle:

• Streaming audio and data in real time

• Connecting to AI, STT, TTS, and animation providers

• Sending back synchronised voice and animation for rendering

• Managing reliability and fallback in case of service interruption

Optimised for:

• Low latency — fast, conversational response times

• High concurrency — supports many users at once

• Reliability — automatic recovery and error handling

The Processing Pipeline

When someone interacts with a Rapport character, multiple cloud services work together in real time:

1. Voice or Text Input — the user speaks or types to the character

2. Speech-to-Text (STT) — speech is converted into text using providers such as Whisper, Azure, Google, or AWS

3. AI Response Generation — AI models (OpenAI, Gemini, Groq, or custom LLMs) generate a contextual reply

4. Text-to-Speech (TTS) — replies are voiced through speech engines including Rapport Voice Pack, ElevenLabs, Azure, Google, and AWS Polly

5. Facial Animation & Emotion — Speech Graphics' engine drives detailed facial animation and emotional expression in sync with the voice

6. Character Response Rendered — the result streams back instantly with voice, lip-sync, and expression

Technology Stack Overview

Rapport integrates text, speech, animation, and emotion into a seamless, real-time performance — delivering high-quality character interaction across devices and environments.

Deployment Targets

Rapport experiences can be deployed anywhere your users are:

• Web Page — hosted experience, instantly shareable by link

• Web Widget — embeddable component for websites or e-learning platforms

• Unreal Plugin — native integration for 3D and immersive applications

• Unity Plugin — for real-time simulation and game projects

Every deployment connects to the same expressive Rapport engine — ensuring consistent realism and responsiveness across all environments.

Why This Split Works

• Creators gain a clean, no-code workspace for designing and testing interactive characters.

• Developers can extend or integrate Rapport into existing pipelines using SDKs and APIs.

• End-users experience fast, expressive, and reliable interactions at scale.

This architecture makes Rapport flexible enough for solo creators, production teams, and enterprise-scale deployments alike.