Configuration

General Setup

projectId: Your project’s ID. (String, required)

projectToken: Your project’s token. (String, required)

lobbyZoneId: ID of the lobby zone that your AI is added to. (String, required)

aiUserId: Your AI user’s ID associated with this project. (String, required)

Rendering

backgroundColor: Hex value of a color or 'transparent'. (String, default ‘transparent’) eg.: #fcba03

loadingImage: URL of the image shown until the AI is fully loaded. (String, default null)

orbitalControls: Allow camera rotation around the avatar. (Boolean, default true)

ocAngle: Maximum angle of rotation around the avatar. (Number, default 45 degrees)

ocZoom: Allow zoom in and out on the avatar. (Boolean, default false)

cameraPosition: Allow adjustment of default calculated camera position. Eg. use param object cameraPosition = { x: 0, y: 0.5, z: 2 } or camera-position attribute as a string '{"x":0,"y":0.5,"z":2}'

signalIndicator: A green / yellow / red border indicating network signal quality. Activate with ‘all’ for a border on all sides of the rendering canvas or  ‘bottom’ for a border at the bottom of the canvas. (Defaults to ‘none’ / no border indicator.)

progressBar: A 0-100% loading bar indicating progress of room session setup. (Boolean, default false)

statusBar: A text notification bar indicating steps taken during a session request. Log in, create a room, download a model, communicate with CPI tower, etc. (Boolean, default false)

showLogo: Controls the visibility of the Rapport logo and link rendered on the scene. (Boolean, default true)

Advanced

micRequired: When set to false microphone permission request is skipped. RWV won’t use the microphone in this mode and won’t start the session until there is a user interaction on the page. (Boolean, default true)

micMuted: When set to true the session is started with muted microphone. (Boolean, default false)

speakerMuted: When set to true the session is started with muted speakers. (Boolean, default false)

micDelay: Session is started with a muted microphone and unmutes after a delay. The delay value is in milliseconds. Setting the delay to 0 disables this behavior. (Number, default 300)

ttsOpeningText: The first thing the AI will say when a session is connected. This can be used to trigger a response from a user. (String, default null)

enableCookies: If true, a localStorage key rapport_client_display_name_${projectId} will store for a generated username. If false, a new username gets registered for each session. (Boolean, default true) 

aecRequired: Echo cancellation should be available for most conversations with a microphone.  If echo cancellation cannot be established and if aecRequired is set to true, the session will terminate with sessionDisconnected notifications. Warning messages will be sent if AEC can not be established and a microphone is not required. 

emotions: If true, icons will appear in the footer indicating the emotion (positive, negative, neutral) detected in stream from the user's microphone. (Boolean, default false)

micControl: If enabled, a microphone for muting / unmuting the user’s microphone will appear. (Boolean, default false)

micDebug: If enabled, the microphone will turn green on Lex AI voice activity detections (VAD). (Boolean, default false)

volumeControl: If true, a slider will appear to allow the user to control the output volume, and a speaker for muting unmuting output volume. (Boolean, default false)

timeout: max duration of session. After this time, the session will disconnect and a sessionDisconnected info notification will be sent. (Seconds, default 600)

inactivity: an inactivity timer to disconnect a session. Inactivity is reset by messages received from a CPIM module, eg aiMessage, ttsMessage, asrMessage. (Seconds, default 180)

logLevel: level used for setting Status Code notifications integrator wishes to receive.  All notifications will be sent if set to ‘info’. Set to ‘warning’ to receive ‘warning’ and ‘error’ notifications. Set this to ‘error’ to only receive notification of error / terminating sessionDisconnected events. (String, default ‘info’)

Manage scene lights

The lights module provides methods to read, create, update and delete lights on the scene during runtime. Multiple different type of lights are supported. Each type has its own properties that can be modified.

Example light object: 

{
  type: "DirectionalLight",
  intensity: 1,
  color: "#ffffff",
  shadows: true,
  rotation: {
    x: 0,
    y: 0
  }
}

Type of lights:
AmbientLight:

{
  type: 'AmbientLight',
  intensity: 1,
  color: '#ffffff',
}

Reference: https://threejs.org/docs/?q=light#api/en/lights/AmbientLight

DirectionalLight:

{
  type: 'DirectionalLight',
  intensity: 1,
  color: '#ffffff',
  shadows: true,
  rotation: {
    x: 0,
    y: 0,
  },
}

Reference: https://threejs.org/docs/?q=light#api/en/lights/DirectionalLight

SpotLight:

  type: 'SpotLight',
  intensity: 1,
  color: '#ffffff',
  distance: 0,
  angle: Math.PI / 3,
  penumbra: 0,
  decay: 1,
  rotation: {
    x: 0,
    y: 0,
  }
}

Reference: https://threejs.org/docs/?q=light#api/en/lights/SpotLight

HemisphereLight:

{
  type: 'HemisphereLight',
  intensity: 1,
  skyColor: '#bae8ff',
  groundColor: '#ffedd2',
}

Reference: https://threejs.org/docs/?q=light#api/en/lights/HemisphereLight

Create:

Create a light object based on the given properties and attach it to the scene.
Returns the new light object.

myRapportScene.lights.create({
  type: 'DirectionalLight',
  intensity: 1,
  color: '#ffffff',
  shadows: true,
  rotation: {
    x: 0,
    y: 0,
  },
});

Read:

Returns all light objects attached to the scene in an array.

myRapportScene.lights.read();

Update:
Update the given light object with the new properties.

myRapportScene.lights.update(lightObject, {
  intensity: 1,
  color: '#ffffff',
  shadows: true,
  rotation: {
    x: 0,
    y: 0,
  },
});

Delete:
Delete the given light from the scene.

myRapportScene.lights.delete(lightObject);

Delete all:
Delete all lights from the scene.

myRapportScene.lights.delete();

Light tester playground:
https://demos.rapport.cloud/demos/tech/light/

Events
The <rapport-scene> element emits events during lifecycle points. You can listen to these events in the following ways.

myRapportScene.addEventListener(‘sessionDisconnected’, (reason) => {});

myRapportScene.addEventListener(‘sessionConnected’, () => {});

You can also pass callback functions to the configuration object.

myRapportScene.sessionRequest({
  sessionDisconnected: (reason) => {},
  sessionConnected: () => {},
});

sessionDisconnected(reason)

Called after the session ends because of an error or timeout. Doesn’t invoke after disconnect triggered with rapportScene.sessionDisconnect(). The argument reason can be used to determine the reason for the disconnect. Learn more at the Status codes section.

sessionConnected()

Called after the scene and the AI is fully ready for starting the conversation. Useful if you want to show a loader until the session is fully connected.
 

Methods

sessionRequest() 
> myRapportScene.sessionRequest(configuration = {}); 

The session requestMethod is used to request a Rapport session. Returns a Promise and resolves if a Rapport session has successfully been requested and rejects with a reason if the Promise fails. Parameters for starting the session need to be obtained from either <rapport-scene> attributes or from the given configuration parameter object. If the configuration parameter is omitted then the element's attributes will be used. Element attributes always have priority over the configuration parameters. A resolved sessionRequest does not mean the AI is ready for a conversation yet. Use the sessionConnected event to determine when the AI is fully ready to start the conversation. For configuration parameters see Configuration.

sessionDisconnect

> myRapportScene.sessionDisconnect(); 

The synchronous sessionDisconnect method is used to disconnect the session with the AI. After disconnecting the scene will show the loading image. Returns after the session has made a best effort at disconnecting. You can call sessionRequest() again in a disconnected state. Removing myRapportScene from the DOM automatically calls sessionDisconnect(). If the network connection is lost mid stream, it will take up to 30 seconds for the session to be fully disconnected by CPI.

getMic()

> myRapportScene.getMic();

The getMic method is a promise which resolves and returns true if microphone access was received. Should be used in conjunction with mic-required = false, and once sessionConnected. Rejects if the call fails to get access to the user’s microphone..

muteSpeaker(mute = Boolean)

> myRapportScene.muteSpeaker(true);

Mutes output speaker if mute boolean is true. Unmutes output Speaker if mute boolean is false. Throws an error if trying to mute before sessionConnected.

CPI (Cloud Processing Infrastructure) modules

CPI modules are a way to configure aspects of your AI session. The main modules currently are; ASR (Automatic Speech Recognition), AI, TTS (Text-To-Speech), AC (Animation Controller). You can send commands to specific modules and you can listen to responses from specific modules. Also, echo cancellation should be available for most conversations with a microphone. If echo cancellation cannot be established and aecRequired is set to true, the session will terminate with a sessionDisconnected notification. Warning messages will be sent if AEC can not be established and microphone is not required.

Modules and their capabilities will continue to expand in the future, including language translations between steps.

The general flow of data is:
- User speaks into the microphone :

• ASR module converts microphone audio into text (if no microphone ASR is bypassed, and integrator would use myRapportScene.modules.ai.sendText(text)).
• ASR text is sent to AI module to get a text response.
• AI text response is converted into audio by TTS module.
• TTS audio is sent to AC module to generate animation.

You can listen for responses from specific CPI modules with event listeners:

// moduleName = 'asrMessage' || 'aiMessage' || 'ttsMessage' || 'acMessage' 

rapportScene.addEventListener(moduleName, (ev) => { }); // see ev.detail.params

or as sessionRequest callback parameters:

myRapportScene.sessionRequest({
   asrMessage: (ev) => { }, // see ev.params;

   aiMessage: (ev) => { },

   ttsMessage: (ev) => { },

   acMessage: (ev) => { },

})

Listening to the asrMessage will give a transcript of what the user said, while listening to the aiMessage (for audio-stream driven AIs) or the ttsMessage (for text driven / transactional AIs) will give the transcript of what the AI said.

If a command is sent, a message will be sent back to the modules event listener and callback. These messages will need to be checked for results; indicating success or potential error notifications. 

ASR (Automatic Speech Recognition):

This module captures what the user says into a microphone. Useful if you want to integrate with an accessibility dual chat and speech widget. 

Future implementations will include configuring the source and destination language of the ASR module and if using Google or Amazon ASR services. No commands can presently be sent to the ASR module. Default source and destination ASR language is English and default ASR provider is Amazon.

TTS (Text-To-Speech):

TTS can be configured in setting up a pandorabot AI on accounts, for voice, language, and for using Google or Amazon TTS services. Lex AI’s default to the Lex configuration.

You can listen for the TTS responses in textual form with the ttsMessage event listener or callback. Useful if you want to integrate with a chat widget.

Future implementations will include setting voice, language and translator.

sendText(text)

> myRapportScene.modules.tts.sendText(text);

The TTS sendText method is used to directly get the AI to say something to the user. Used by ‘tts-opening-text’. Throws an error if text is not a string.

AC (Animation Controller):

myRapportScene.modules.ac.setMood(mood);

Sets the mood for the avatar. Possible moods are: positive, acknowledge, negative, and neutral.

The mood is set until a new mood overwrites the currently set mood. Setting mood will overwrite the default AC which has automatic mood detection that switches between the moods depending on audio detected. (Is this still bugged?)
 

myRapportScene.modules.ac.setScale(number);

Sets the amount the animation is exaggerated. Usually a number between 0 and 2. The Closer to zero the smaller the mouth movements. The default value is 1.

AI (Artificial Intelligence):

This module is the AI you configured in accounts. Useful if you have a session with no microphone access and want to send text to an AI.

sendText(text)

> myRapportScene.modules.ai.sendText(text);

The AI sentText mothod is used to send a string directly to the AI. The String can be in SSML format. (Insert SSML documentation here.) Useful if you want to create a button driven or hybrid AI.  Throws an error if text is not a string.

setLexUserId(userId)

> myRapportScene.modules.ai.setLexUserId(userId);

The AI setLexUserId method is used for setting the Lex AI userId. Throws an error if userId is not a string.

getUserId();

> myRapportScene.modules.ai.getUserId();

The AI getUserId() method is used to get current Lex AI userId. Returns current Lex AI userId.

iframe Based Integration:

The Iframe based integration approach is good for quick prototyping and basic integration. We suggest using the web component based integration, however, in order to have faster initial loading speed, smaller download size, and more overall flexibility.

Browser permissions:

By default, you need to set the allow="microphone" attribute to the iframe in order to give microphone access permission for the iframe. The allow="microphone" attribute can be omitted if the micRequired RWV param is set to false. In this case the iframe content will automatically show a button that the user needs to press in order to start the session. User interaction on the iframe is necessary to give audio context for the iframe if microphone permission is not provided.

API:

You can pass parameters to the Rapport scene via the iframe src attribute value as url query parameters. The base url is https://accounts.rapport.cloud/avatar-iframe/. Make sure you pass the query parameter values in url encoded format.

projectId: 

Your project's id. (String, required) 

projectToken:

Your project's token. (String, required)

aiUserId:

Your AI user id associated with the given project. (String, required)

lobbyZoneId:

Lobby zone id you added your AI user to. (String, required)

micRequired:

When set to false microphone permission request is skipped. RWV won’t use the microphone in this mode and won’t start the session until there is a user interaction on the iframe.  (Boolean, default true)

openingText:

A string sent to the AI via the ai module sendText method after the session connected. (String, default null)

enableCookies:

Allows the iframe to set a local storage entry on the user's browser to speed up connection for revisiting users. This falls under a preference cookie category and it's the integrator's responsibility to handle this cookie by the host page's privacy policy. (Boolean, default false)

orbitalControls:

Allow camera rotation around the avatar. (Boolean, default true)

ocZoom:

Allow zoom in and out on the avatar with the mouse scrollwheel. (Boolean, default true)

ocAngle:

Maximum angle of rotation around the avatar. (Number, default 45)

backgroundColor:

Background color of the scene. (String, default 'transparent')

initialMood:

Sets initial mood for the avatar via the ac module setMood method after session connected. (String, default null)

buttonLabel:

Sets the label of the button shown when 'micRequired' is false. (String, default 'Start conversation')

autoConnect:

Setting this to false won't start the session. Useful for experimenting with styling during integration. (Boolean, default true)

showLogo:

Controls the visibility of the Rapport logo and link rendered on the scene. (Boolean, default true)

Styling:

It’s possible to remove the default iframe border and scrollbar with the following attributes.

<iframe
  scrolling="no"
  style="border: 0;"
></iframe>

Example utilizing an <iframe>:

<iframe
  src="https://accounts.rapport.cloud/avatar-iframe/?projectId=yourProjectId&projectToken=yourProjectToken&aiUserId=yourAiUserId&lobbyZoneId=yourLobbyZoneId&buttonLabel=Start%20demo"
  title="Rapport" 
  allow="microphone"
  scrolling="no"
  style="border: 0; width: 600px; height: 600px; max-width: 100%; max-height: 100%;"
></iframe>