Using Pixel Streaming

In order to use pixel streaming with the Rapport Unreal Viewer plugin, a few steps should be taken.

Enabling Pixel Streaming

Ensure your project is set up for pixel streaming. The setup process depends on how you plan to host your project, either on your own server or by using a third-party provider.

• Self-hosted - If you are hosting the project yourself, refer to the official Unreal Engine Pixel Streaming documentation for setup instructions

• Third-party solution - If you are using a third-party pixel streaming provider, follow their guide or documentation to prepare your project

Adding the Pixel Streaming Audio Component

This component allows in-engine playback of incoming audio from a particular pixel streaming player using their mic in the browser.

The component can be added to your character blueprint using the Add button in the Components panel of the blueprint. By default, the component will listen to the first non-listened to player that it finds.

Adding the Rapport Pixel Streaming Input Component

Next, we need to add a Rapport Pixel Streaming Input component as the Rapport audio source to use. Adding the component to the blueprint can be done in the same manner as the Pixel Streaming Audio component; select Add in the Component panel and choose Rapport Pixel Streaming Input.

Starting the Rapport Session

When starting a Rapport Session, you are required to pass in a Rapport Audio Source. In this case, the source will be the Rapport Pixel Streaming Input component.

Starting the Pixel Streaming Audio Capture

To start capturing audio received via pixel streaming, call Start Capture on the Rapport Pixel Streaming Input component. This will set a custom submix on the Pixel Streaming Audio component which then automatically passes the audio to Rapport as it is received from the browser.