RapportModules

Description

The RapportModules API provides a robust and centralized set of tools for creating interactive, AI-driven characters. By acting as a central hub, it streamlines the process of integrating core functionalities like facial animation, text-to-speech, speech recognition, and AI interaction.

These modules empower you to build a character that can understand language, generate dynamic speech, and react in real-time, bringing a new level of engagement and realism to your applications.

Nested Classes

AcModule

The AcModule class controls the facial animations of a character. It provides methods to set the character's mood, the intensity of their facial movements, and the speed and frequency of those animations.

Methods

Name	Return Type	Description
SetMood(string mood)	void	Sets the character's mood. Parameters: `string mood`: A string representing the desired mood. Allowed values are 'neutral', 'positive', 'negative', 'acknowledge', and 'thinking'.
SetScale(float scale)	void	Adjusts the scale of the facial animations. A higher value results in more pronounced movements. Parameters: `float scale`: The animation scale, typically a value between 0 and 2.
SetFrequency(float frequency)	void	Sets the frequency of animation changes. A higher value causes animations to change more rapidly. Parameters: `float frequency`: The animation frequency, typically a value between 0 and 2.
SetSpeed(float speed)	void	Controls the playback speed of animations. A higher value makes animations play faster.

AiModule

The AiModule class is responsible for sending text to an AI system and managing the session state.

Methods

Name	Return Type	Description
SendText(string text, string language)	void	Sends a text string to the AI for processing. Parameters: `string text`: The text to send. `string language`: The language of the text.
GetSessionId()	void	Requests the current session ID from the AI module.
SetSessionId(string sessionId)	void	Sets the session ID for the AI module. Parameters: `string userId`: The new session ID.
SetUserId(string userId)	void	Sets the user ID for the AI module, which can change the AI character within a project. Parameters: `string userId`: The new user ID.

AsrModule

The AsrModule class handles speech recognition features, including setting the spoken language and attempting to identify a user's language.

Methods

Name

Return Type

Description

SetLanguage(string language)

void

Sets the expected language for speech recognition. This uses the "amazon" ASR service by default.

Parameters:
- string language: The language code (e.g., 'en-US').

IdentifyLanguage(string languageHints, int? timeout)

void

Tries to identify the speaker's language within a specified timeframe. The system requires at least 5 seconds of constant voice input. A successful identification or a timeout will be reported via the AsrMessage event.

Parameters:
- string languageHints: A comma-separated list of potential language codes (e.g., 'en-US,de-DE,fr-FR').
- int? timeout: The maximum duration in seconds for the identification process.

TtsModule

The TtsModule class manages text-to-speech functionality, allowing the system to generate spoken audio from text.

Methods

Name

Return Type

Description

SendText(string text, string language)

void

Sends text to be converted into speech.

Parameters:
- string text: The text to be spoken.
- string lang: An optional language code. If not provided, it uses the currently set language.

SetLanguage(string language, string voice, string tts, bool neural)

void

Sets the language and voice for the text-to-speech system. This method internally makes two separate calls: one to set the language and another to configure the TTS provider, voice, and neural setting.

Parameters:
- string language: The language code (e.g., 'en-US').
- string voice: The name of the specific voice to use (e.g., 'Matthew').
- string tts: The text-to-speech provider. Defaults to "polly".
- bool neural: A flag indicating whether to use a neural voice. Defaults to true.

VadModule

The VadModule class is responsible for configuring voice activity detection settings. This helps distinguish between speech and silence.

Methods

Name

Return Type

Description

SetMode(string mode)

void

Sets the aggressiveness mode for the VAD.

Parameters:
- string mode: The VAD mode.

SetThreshold(float trueMs, float falseMs)

void

Sets the time thresholds in milliseconds for switching between speech and silence classifications.

Parameters:
- float trueMs: The minimum duration in milliseconds to classify a sound as voice.
- float falseMs: The minimum duration in milliseconds to classify a lack of sound as silence.

Properties

Name	Type	Description
AC	AcModule	Animation Controller
AI	AiModule	Artificial Intelligence
ASR	AsrModule	Automatic Speech Recognition
TTS	TtsModule	Text to Speech
VAD	VadModule	Voice Auto Detect