Main Features
-
Web component or iframe based integration.
-
Initiating the Rapport Cloud Processing Infrastructure (CPI) to prepare a room with the AI specified in your Rapport account and manages sessions.
-
Establishing a WebRTC connection between a browser and CPI room.
-
Handling of the streaming connection that provides audio and animation data which then gets buffered, synchronized, and turned into a Three.js animation of your AI talking.
-
Resizing the rendering canvas to fit the parent element or set style.
-
Targeting a rendering rate of 30 fps.
-
Caching of 3D (.glb) models.
-
Sending commands to CPI modules, e.g. send-text, set-mood, set-scale, etc.
-
Receiving messages from CPI modules, e.g. for AI, TTS, ASR, AC (animation controller) modules.
-
Disconnecting sessions on page termination events where possible.
-
Info, Warning, sessionDisconnected callback and event listener notification system.
-
Providing acoustic echo cancellation (AEC) for microphone conversations where possible.
Optional Features
-
Rendering on colored or transparent canvas.
-
Providing orbital controls for allowing the user to move around the animation.
-
3 point lighting.
-
Opening text to initiate the AI conversation on joining the room.
-
Inactivity timers for automated disconnecting sessions.
-
Displaying a loading image while a room is being prepared.
-
Callbacks and events for session connected and disconnected.
-
Configurable network signal quality indicators.
-
Configuring audio for changing audio inputs and outputs (work in progress).
-
Seamlessly switching between 'no microphone' and 'microphone required' modes.
-
Footer for displaying various status indicators.
-
Audio controls for speaker volume and muting microphone.
-
Status bar for displaying room preparation steps.
-
Progress bar for displaying estimation of background work completed before joining the room.
-
Icon showing detected emotion in user's voice.
-
Mic icon indicates if voice activity is detected.
-