3D Spatial Audio
3D Spatial Audio brings theater-like effects to Interactive Live Streaming, making it seem as if the sound originates from all around the user. Video SDK provides spatial audio effects that give users an immersive audio experience in scenarios such as e-sports competitions, and online conferences.
You can configure the following spatial audio effects:
Spatial audio effects for users:
By setting the local and remote user's spatial positions, you can create an environment that enables users to experience changes in the distance, position, and orientation of other users in real-time. You can also enable your users to experience multiple audio effects such as audio blurring and air absorption.
Spatial audio effects for media player:
By updating the spatial position of the media player, you can add a sense of space to media resources such as background sounds and musical accompaniment. Agora provides local cartesian coordinate system calculation solution for the media player. This solution calculates the relative positions of the local user and the media player through the Video SDK. You update the spatial coordinates of the local user and the media player, respectively, so that the local user can hear the spatial audio effect of the media player.
Understand the tech
The following figure shows the workflow you need to integrate spatial audio into your app.
In order to follow this procedure you must have:
- Implemented the SDK quickstart project for Interactive Live Streaming.
- Android Studio 4.1 or higher.
- Android SDK API Level 24 or higher.
- A mobile device that runs Android 4.1 or higher.
A computer with Internet access.
Ensure that no firewall is blocking your network communication.
- To ensure a true spatial experience, Agora recommends using an audio device that supports true binaural playback.
To create the environment necessary to implement spatial audio into your app, do the following:
Add spatial audio to your app
This section shows how to use the Video SDK to implement spatial audio into your app, step-by-step.
Implement a user interface
In a real-word application, you report your local spatial position to a server in your environment and retrieve positions of remote users in the channel from your server. In this simple example, you use a
SeekBar to set the spatial position of a remote user.
To add a
SeekBar to the UI, in
/app/res/layout/activity_main.xml, add the following code before
Handle the system logic
Add the required libraries
To set up and configure an instance of the spatial audio engine and use the
SeekBar, import the corresponding Agora and Android libraries. In
/app/java/com.example.<projectname>/MainActivity, add the following to the list of
Implement spatial audio
To implement spatial audio features in your app, take the following steps:
Declare the variables you need
You create an instance of
ILocalSpatialAudioEngineto configure spatial audio parameters and set self and remote user positions. In
/app/java/com.example.<projectname>/MainActivity, add the following declarations to the
Instantiate and configure the spatial audio engine
To create an instance of
ILocalSpatialAudioEngineat startup, take the following steps:
When a user launches the app, you create an instance of
ILocalSpatialAudioEngine, configure it and update the user's self position. To do this, add the following method to the
To execute this method at startup, add the following line to the
Set the spatial position of a remote user
To update the spatial position of a remote user:
updateSpatialAudioPositionwhen a user moves the slider, add the following code to
To update the spatial position of a specific remote user, you need the
uidof that user. Add the following line to the
onUserJoinedcallback, to store the
Spatial audio functionality is currently available as beta.
Test your implementation
To ensure that you have implemented spatial audio features into your app:
Generate a temporary token in Agora Console .
In your browser, navigate to the Agora web demo and update App ID, Channel, and Token with the values for your temporary token, then click Join.
In Android Studio, open
tokenwith the values from Agora Console.
Connect an Android device to your development device.
In Android Studio, click Run app. A moment later, you see the project installed on your device.
If this is the first time you run your app, grant camera and microphone permissions.
- Press Join to connect to the same channel as your web demo.
Test spatial audio effects for remote users
Put on earphones connected to the Android test device.
Speak into the microphone connected to the web demo app. Note the quality of audio playing through your earphones.
Move the slider to the right. Your app updates the position of the remote user in the spatial audio engine by increasing the distance.
Speak into the microphone again and listen to the audio through your earphones. You feel that the remote user has moved away from you.
Test spatial audio effects for media player
To setup spatial audio position of your media player, add Media Playing to your app.
Replace the call to
updateSpatialAudioPositionwith the following:
Play the media file in your Android app.
Connect your earphones to the web demo and note the quality of audio playing.
Move the slider to the right. Your app updates the position of the media player in the spatial audio engine.
Note the change in quality of media playing through your earphones. You feel that the position of the media player has moved away from you.
This section completes the information on this page, or points you to documentation that explains other aspects about this product.