Extension - Behaviour Mode

Overview

The Behavior Mode Extension enhances Rapport characters by synchronizing AI-driven emotional intent with visual behavior in real time.

When enabled, the extension:

• Interprets emotion and animation annotations from AI responses

• Applies corresponding facial expressions and body language

• Sends clean dialogue only to Text-to-Speech (TTS)

This results in more expressive, believable character interactions — without emotion tags being spoken aloud.

⚠️ Important

Behavior Mode Extensions are created and maintained by Rapport Admins.
End users do not create extensions, but can enable and use them when included in a template or copied project.

Where This Feature Lives

• Configured in:
  Project Setup → Advanced Settings → Developer Settings

• Authored by:
  Rapport Admins only

• Used by:
  Any project that includes the extension (via templates or copied projects)

Using the Extension in Your Project

Templates & Copied Projects

Some templates include a preconfigured Behavior Mode Extension.

If you:

• Use a template that includes the extension, or

• Copy a project that already has it enabled

…the extension can be reused immediately — no additional setup is required beyond configuration.

Required Configuration (Critical)

Disable “Send AI to TTS”

When the Behavior Mode Extension is enabled, it takes full control of routing AI output to TTS.

Configuration rule:

If both are enabled, you may experience:

• ❌ Duplicate audio playback

• ❌ Emotion tags spoken aloud

• ❌ Broken emotion synchronization

• ❌ TTS ordering issues

🔗 Configure this in: Advanced Settings → Developer Settings

How It Works (At Runtime)

Processing Flow

1. AI Response
   AI generates text with inline emotion/animation tags
   Example:
   {happy/Jump}: I'm excited to help!

2. Extension Parsing
   The extension extracts:

   • Emotion

   • Animation (optional)

   • Dialogue text

3. Visual Behavior Update
   Character expression and body language update immediately

4. Clean TTS Output
   Only the dialogue text is sent to TTS (tags removed)

5. Synchronized Playback
   Each emotion is applied in sequence as speech progresses

AI Prompt Requirements

To work correctly, your AI must be instructed to format responses using emotion and animation tags.

Required Format

{Emotion/Animation}: Text

Rules:

• Each sentence starts with a tag

• One tag per sentence is recommended

• Two sentences = two tagged segments

Example Output

{positive/Idle}: Welcome! I'm glad you're here.
{happy/Jump}: This is going to be great!

Supported Emotions

The AI must only use emotions from this list:

⚠️ If the AI wants to express something outside this list, it must choose the closest match.

Best practice: Avoid repeating the same emotion consecutively.

Supported Animations

Example Prompt Template

# Response Format
Format responses with emotion/animation tags: {Emotion/Animation}: Text

Rules:
- Start each sentence with a tag
- Use ONLY approved emotions
- Vary emotions between sentences
- Match animation to sentiment

Available Emotions:
neutral, negative, angry, acknowledge, idle, positive, happy, sad, surprise

Available Animations:
Idle, Walk, Run, Point, Jump, Dance

Examples:
User: I want to go on a crazy journey!
Assistant:
{negative/Dance}: I'm not sure about this plan.
{happy/Jump}: But I'm still excited to go!

What the Extension Does (Technical Summary)

1. Annotation Parsing

• Detects {emotion/animation}: text patterns

• Supports multiple tagged segments per response

2. Emotion Application

• Adjusts facial animation, blinking, and eye movement

• Intensifies motion for strong emotions (e.g., angry, negative)

3. TTS Processing

• Removes all tags before speech

• Sends dialogue segments with staggered timing

• Prevents TTS mis-ordering

4. Real-Time Synchronization

• Applies emotion before speech begins

• Advances emotion on TTS completion

• Returns character to idle/listening state at the end

Setup Checklist

☐ Extension enabled in Advanced Settings → Developer Settings
☐ “Send AI to TTS” disabled
☐ AI prompt updated with emotion tags
☐ Tested with multiple responses
☐ Prompt refined for natural emotion flow

Troubleshooting

Best Practices

• ✅ Test multiple emotional scenarios

• ✅ Use clear prompt examples

• ✅ Encourage emotion variety

• ✅ Monitor logs during development

• ✅ Observe user reactions and refine prompts