Skip to main content

You are viewing Agora Docs forBeta products and features. Switch to Docs

3D Spatial Audio

3D Spatial Audio brings theater-like effects to Voice Calling, making it seem as if the sound originates from all around the user. Voice SDK provides spatial audio effects that give users an immersive audio experience in scenarios such as e-sports competitions, and online conferences.

You can configure the following spatial audio effects:

  • Spatial audio effects for users:

    By setting the local and remote user's spatial positions, you create an environment that enables users to experience changes in the distance, position, and orientation of other users in real time. This includes:

    • Audio range: Enable users in the same group and within the range to hear each other.

    • Voice blur: Enable users to hear sounds outside of the selected range and distance, or sounds of specific objects, as blurred whispers.

    • Air attenuation: Create a more natural and realistic audio experience, making it easier for people to have natural conversations. The audio engine models how sound behaves as it travels through the air and adjusts the volume based on the distance of the audio source from the listener.

    • Local sound source playback: Customize the background sound and other audio effects.

  • Spatial audio effects for media player:

    Update the spatial position of the media player to add a sense of space to media resources such as background sounds and musical accompaniment. Agora provides local cartesian coordinate system calculation solution for the media player. This solution calculates the relative positions of the local user and the media player through Voice SDK. You update the spatial coordinates of the local user and the media player, respectively, so that the local user can hear the spatial audio effect of the media player.

Understand the tech

The following figure shows the workflow you need to integrate spatial audio into your app.

Spatial Audio

Prerequisites

  • Android Studio 4.1 or higher.
  • Android SDK API Level 24 or higher.
  • A mobile device that runs Android 4.1 or higher.
  • An Agora account and project.

  • A computer with Internet access.

    Ensure that no firewall is blocking your network communication.

  • To ensure a true spatial experience, best practice is to use an audio device that supports true binaural playback.

Add spatial audio to your app

This section shows how to use the Video SDK to implement spatial audio into your app, step-by-step.

Implement a user interface

In a real-word application, you report your local spatial position to a server in your environment and retrieve positions of remote users in the channel from your server. In this simple example, you use a SeekBar to set the spatial position of a remote user.

To add a SeekBar to the UI, in /app/res/layout/activity_main.xml, add the following code before </RelativeLayout>:


_18
<TextView
_18
android:id="@+id/seekBarText"
_18
android:layout_width="wrap_content"
_18
android:layout_height="wrap_content"
_18
android:layout_below="@id/JoinButton"
_18
android:text="Change spatial position"
_18
android:layout_alignStart="@id/JoinButton"/>
_18
_18
<SeekBar
_18
android:id="@+id/distanceSeekBar"
_18
android:layout_width="match_parent"
_18
android:layout_height="wrap_content"
_18
android:layout_below="@id/seekBarText"
_18
android:layout_alignStart="@id/JoinButton"
_18
android:layout_alignEnd="@id/LeaveButton"
_18
android:min="0"
_18
android:max="10"
_18
android:progress="0" />

Handle the system logic

  1. Add the required libraries

    To set up and configure an instance of the spatial audio engine and use the SeekBar, import the corresponding Agora and Android libraries. In /app/java/com.example.<projectname>/MainActivity, add the following to the list of import statements:


    _5
    import io.agora.spatialaudio.ILocalSpatialAudioEngine;
    _5
    import io.agora.spatialaudio.LocalSpatialAudioConfig;
    _5
    import io.agora.spatialaudio.RemoteVoicePositionInfo;
    _5
    _5
    import android.widget.SeekBar;

Implement spatial audio

To implement spatial audio features in your app, take the following steps:

  1. Declare the variables you need

    You create an instance of ILocalSpatialAudioEngine to configure spatial audio parameters and set self and remote user positions. In /app/java/com.example.<projectname>/MainActivity, add the following declarations to the MainActivity class:


    _3
    private ILocalSpatialAudioEngine localSpatial;
    _3
    private int remoteUid = 0; // Stores the uid of the remote user
    _3
    private SeekBar distanceSeekBar;

  2. Instantiate and configure the spatial audio engine

    To create an instance of ILocalSpatialAudioEngine at startup, take the following steps:

    1. When a user launches the app, you create an instance of ILocalSpatialAudioEngine, configure it and update the user's self position. To do this, add the following method to the MainActivity class:


      _24
      private void configureSpatialAudioEngine() {
      _24
      // Enable spatial audio
      _24
      agoraEngine.enableSpatialAudio(true);
      _24
      _24
      // Create and initialize the spatial audio engine
      _24
      LocalSpatialAudioConfig localSpatialAudioConfig = new LocalSpatialAudioConfig();
      _24
      localSpatialAudioConfig.mRtcEngine = agoraEngine;
      _24
      localSpatial = ILocalSpatialAudioEngine.create();
      _24
      localSpatial.initialize(localSpatialAudioConfig);
      _24
      _24
      // Set the audio reception range of the local user in meters
      _24
      localSpatial.setAudioRecvRange(50);
      _24
      _24
      // Set the length of unit distance in meters
      _24
      localSpatial.setDistanceUnit(1);
      _24
      _24
      // Define the position of the local user
      _24
      float[] pos = new float[]{0.0F, 0.0F, 0.0F};
      _24
      float[] forward = new float[]{1.0F, 0.0F, 0.0F};
      _24
      float[] right = new float[]{0.0F, 1.0F, 0.0F};
      _24
      float[] up = new float[]{0.0F, 0.0F, 1.0F};
      _24
      // Set the position of the local user
      _24
      localSpatial.updateSelfPosition(pos, forward, right, up);
      _24
      }

    2. To execute this method at startup, add the following line to the onCreate method after setupVideoSDKEngine();:


      _1
      configureSpatialAudioEngine();

  3. Set the spatial position of a remote user

    To update the spatial position of a remote user:

    1. Define the RemoteVoicePositionInfo and call updateRemotePosition.


      _12
      public void updateSpatialAudioPosition(float sourceDistance){ //View view){
      _12
      // Define a remote user's spatial position
      _12
      RemoteVoicePositionInfo positionInfo = new RemoteVoicePositionInfo();
      _12
      // The three values represent the front, right, and top coordinates
      _12
      positionInfo.position = new float[]{sourceDistance, 0.0F, 0.0F};
      _12
      positionInfo.forward = new float[]{sourceDistance, 0.0F, 0.0F};
      _12
      _12
      // Update the spatial position of the specified remote user
      _12
      localSpatial.updateRemotePosition(remoteUid, positionInfo);
      _12
      _12
      showMessage("Spatial position updated");
      _12
      }

    2. To call updateSpatialAudioPosition when a user moves the slider, add the following code to onCreate:


      _15
      distanceSeekBar = (SeekBar)findViewById(R.id.distanceSeekBar);
      _15
      _15
      distanceSeekBar.setOnSeekBarChangeListener(new SeekBar.OnSeekBarChangeListener() {
      _15
      public void onProgressChanged(SeekBar seekBar, int progress, boolean fromUser) {
      _15
      //Required to implement OnSeekBarChangeListener
      _15
      }
      _15
      _15
      public void onStartTrackingTouch(SeekBar seekBar) {
      _15
      //Required to implement OnSeekBarChangeListener
      _15
      }
      _15
      _15
      public void onStopTrackingTouch(SeekBar seekBar) {
      _15
      updateSpatialAudioPosition((float) seekBar.getProgress());
      _15
      }
      _15
      });

    3. To update the spatial position of a specific remote user, you need the uid of that user. Add the following line to the onUserJoined callback, to store the remoteUid:


      _1
      remoteUid = uid;

Test your implementation

To ensure that you have implemented 3D Spatial Audio features into your app:

  1. Generate a temporary token in Agora Console .

  2. In your browser, navigate to the Agora web demo and update App ID, Channel, and Token with the values for your temporary token, then click Join.

  1. In Android Studio, open app/java/com.example.<projectname>/MainActivity and update appId, channelName and token with the values from Agora Console.

  2. Connect an Android device to your development device.

  3. In Android Studio, click Run app. A moment later, you see the project installed on your device.

    If this is the first time you run your app, grant camera and microphone permissions.

  1. Press Join to connect to the same channel as your web demo.
  1. Test spatial audio effects for remote users

    1. Put on earphones connected to the Android test device.

    2. Speak into the microphone connected to the web demo app. Note the quality of audio playing through your earphones.

    3. Move the slider to the right. Your app updates the position of the remote user in the spatial audio engine by increasing the distance.

    4. Speak into the microphone again and listen to the audio through your earphones. You feel that the remote user has moved away from you.

  2. Test spatial audio effects for media player

    1. To setup spatial audio position of your media player, add Media Playing to your app.

    2. Replace the call to localSpatial.updateRemotePosition in updateSpatialAudioPosition with the following:


      _2
      // Update spatial position of the media player
      _2
      localSpatial.updatePlayerPositionInfo(mediaPlayer.getMediaPlayerId(), positionInfo);

    3. Play the media file in your Android app.

    4. Connect your earphones to the web demo and note the quality of audio playing.

    5. Move the slider to the right. Your app updates the position of the media player in the spatial audio engine.

      Note the change in quality of media playing through your earphones. You feel that the position of the media player has moved away from you.

Reference

This section completes the information on this page, or points you to documentation that explains other aspects about this product.

Voice Calling