Integrate an extension

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

To test the code used in this page you need to have:

• Implemented either of the following:
  • UI Kit quickstart
  • SDK quickstart

• An Agora account and project.

• A computer with Internet access.

  Ensure that no firewall is blocking your network communication.

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.

   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

      You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

3. Download the extension

   In the extension detail page, click Download, then unzip the extension in a local directory.

4. Install the extension in your project

   • Android Archive file (.aar)

     1. Save the extension .aar file to /app/libs in your project.

     2. In /Gradle Scripts/build.gradle(Module: <ProjectName> app), add the following line under dependencies:

        
        

        _1

        implementation fileTree(include: ['*.jar', '*.aar'], dir: 'libs')

        
        

   • Shared Library (.so)

     Save the .so file to the following paths in your project:

     1. /app/src/main/jniLibs/arm64-v8a

     2. /app/src/main/jniLibs/armeabi-v7a

You are now ready to integrate the extension in your app.

The watermark extension adds a watermark on video streamed to your local client. This section shows you how to implement the watermark extension in your Agora project:

1. Import the necessary classes

   1. Download the watermark extension and follow the steps for .aar files in setup.

   2. In app/src/main/java/com.example.<projectname>/MainActivity:

      1. Add the following lines to import the Android classes used by the extension:

         
         

         _2

         import org.json.JSONException;

         _2

         import org.json.JSONObject;

         
         

      2. Add the following lines to import the Agora classes used by the extension:

         
         

         _3

         // ExtensionManager is used to pass in basic info about the extension

         _3

         import io.agora.extension.ExtensionManager;

         _3

         import io.agora.rtc2.IMediaExtensionObserver;

         
         

2. Add the extension and register the event handler

   In setupVideoSDKEngine, add the following code before agoraEngine = RtcEngine.create(config);:

   
   

   _26

   config.addExtension(ExtensionManager.EXTENSION_NAME);

   _26

   _26

   // Register IMediaExtensionObserver to receive events from the extension.

   _26

   config.mExtensionObserver = new IMediaExtensionObserver() {

   _26

   @Override

   _26

   public void onEvent(String vendor, String extension, String key, String value) {

   _26

   // Add callback handling logics for extension events.

   _26

   showMessage("Extension: " + extension + "\n Key: "

   _26

   + key + " \nValue:" + value);

   _26

   }

   _26

   _26

   @Override

   _26

   public void onStarted(String provider, String extension) {

   _26

   showMessage("Extension started");

   _26

   }

   _26

   _26

   @Override

   _26

   public void onStopped(String provider, String extension) {

   _26

   showMessage("Extension stopped");

   _26

   }

   _26

   _26

   @Override

   _26

   public void onError(String provider, String extension, int error, String message) {

   _26

   showMessage(message);

   _26

   }

   _26

   };

   
   

3. Enable the extension

   Call enableExtension to enable the extension. To enable multiple extensions, call enableExtension as many times. The sequence of enabling multiple extensions determines the order of these extensions in the transmission pipeline. For example, if you enable extension A before extension B, extension A processes data from the SDK before extension B.

   In setupVideoSDKEngine, add the following code before agoraEngine = RtcEngine.create(config);:

   
   

   _2

   agoraEngine.enableExtension(ExtensionManager.EXTENSION_VENDOR_NAME,

   _2

   ExtensionManager.EXTENSION_VIDEO_FILTER_NAME, true);

   
   

4. Set extension properties

   In the joinChannel(View view) method, add the following code after agoraEngine.joinChannel:

   
   

   _12

   JSONObject o = new JSONObject();

   _12

   try {

   _12

   // Pass in the key-value pairs defined by the extension provider to configure the feature you want to use.

   _12

   o.put("plugin.watermark.wmStr", "Agora");

   _12

   o.put("plugin.watermark.wmEffectEnabled", true);

   _12

   _12

   // Call setExtensionProperty to use the watermark feature.

   _12

   agoraEngine.setExtensionProperty(ExtensionManager.EXTENSION_VENDOR_NAME,

   _12

   ExtensionManager.EXTENSION_VIDEO_FILTER_NAME, "key", o.toString());

   _12

   } catch (JSONException e) {

   _12

   e.printStackTrace();

   _12

   }

   
   

To ensure that you have integrated the extension in your app:

1. Connect the Android device to the computer.

2. Click Run app on your Android Studio. A moment later you will see the project installed on your device.

3. When the app launches, you can see yourself and the watermark Agora on the local view.

This section contains information that completes the information in this page, or points you to documentation that explains other aspects to this product.

• RtcEngineConfig.addExtension
• enableExtension
• getExtensionProperty
• setExtensionProperty
• IMediaExtensionObserver