04 Jun 2021
Minute to read
Updated on 04 Jun 2021
Minute to read
- Web component or iframe based integration.
- Initiating the Rapport Cloud Processing Infrastructure (CPI) to prepare a room with the AI specified in your Rapport account and manages sessions.
- Establishing a WebRTC connection between a browser and CPI room.
- Handling of the streaming connection that provides audio and animation data which then gets buffered, synchronized, and turned into a Three.js animation of your AI talking.
- Resizing the rendering canvas to fit the parent element or set style.
- Targeting a rendering rate of 30 fps.
- Caching of 3D (.glb) models.
- Sending commands to CPI modules, e.g. send-text, set-mood, set-scale, etc.
- Receiving messages from CPI modules, e.g. for AI, TTS, ASR, AC (animation controller) modules.
- Disconnecting sessions on page termination events where possible.
- Info, Warning, sessionDisconnected callback and event listener notification system.
- Providing acoustic echo cancellation (AEC) for microphone conversations where possible.
- Rendering on colored or transparent canvas.
- Providing orbital controls for allowing the user to move around the animation.
- 3 point lighting.
- Opening text to initiate the AI conversation on joining the room.
- Inactivity timers for automated disconnecting sessions.
- Displaying a loading image while a room is being prepared.
- Callbacks and events for session connected and disconnected.
- Configurable network signal quality indicators.
- Configuring audio for changing audio inputs and outputs (work in progress).
- Seamlessly switching between 'no microphone' and 'microphone required' modes.
- Footer for displaying various status indicators.
- Audio controls for speaker volume and muting microphone.
- Status bar for displaying room preparation steps.
- Progress bar for displaying estimation of background work completed before joining the room.
- Icon showing detected emotion in user's voice.
- Mic icon indicates if voice activity is detected.