Skip to main content

You are viewing Agora Docs forBetaproducts and features. Switch to Docs

Android
iOS
Web
macOS
Windows
Flutter
React Native

BytePlus Effects

The BytePlus Effects extension encapsulates the core APIs of the BytePlus Effects SDK. This guide is provided by BytePlus. Agora is planning a documentation upgrade program for all extensions on the marketplace. Please stay tuned.

Understand the tech

The BytePlus Effects extension encapsulates the core APIs of the BytePlus Effects SDK. By calling the setExtensionProperty or setExtensionPropertyWithVendor method of the Agora Video SDK v4.x and passing in the corresponding key and value, you can quickly integrate BytePlus's capabilities.

Taking setExtensionProperty as an example, the key is named after the BytePlus API, and the value wraps either certain or all of the parameters of that API in JSON. When you call setExtensionProperty and pass in the pair of key and value, it is equivalent to calling the corresponding BytePlus API. The case is the same for setExtensionPropertyWithVendor.

Note

Currently, the extension encapsulates only a part of the APIs of the BytePlus Effects SDK. For details, see the key-value overview.

Prerequisites

The development environment has to meet the following requirements:

  • Android Studio 4.1 or later.
  • A physical device (not an emulator) running Android 5.0 or later.

Preparation

The BytePlus Effects extension works together with the Video SDK v4.x. Refer to the following doc to integrate the SDK and start a basic video call:

Integrate the extension

  1. Download the Android package of BytePlus Effects.

  2. Unzip the package, and save all .aar files to the /app/libs path of your project folder.

  3. Contact Agora to get the resource package of the BytePlus extension. Save the files you need to the /app/src/main/assets path of the project folder. For details, see Resource package structure.

  4. In the app/build.gradle file, add the following line in dependencies:


    _1
    implementation fileTree(dir: "libs", include: ["*.jar", "*.aar"])

  5. Import the required classes:


    _7
    import io.agora.rtc2.Constants;
    _7
    import io.agora.rtc2.IMediaExtensionObserver;
    _7
    import io.agora.rtc2.IRtcEngineEventHandler;
    _7
    import io.agora.rtc2.RtcEngine;
    _7
    import io.agora.rtc2.RtcEngineConfig;
    _7
    import io.agora.rtc2.video.VideoCanvas;
    _7
    import io.agora.rte.extension.bytedance.ExtensionManager;

Call sequence

This section describes the call sequence of using the extension. For a detailed parameter description, see the API reference.

Enable the extension

When initializing RtcEngine, call enableExtension to enable the extension:


_7
RtcEngineConfig config = new RtcEngineConfig();
_7
// Listen to extension events to receive onEvent callbacks
_7
config.mExtensionObserver = this;
_7
config.addExtension("AgoraByteDanceExtension");
_7
mRtcEngine = RtcEngine.create(config);
_7
// Enable the plugin
_7
mRtcEngine.enableExtension("ByteDance", "Effect", enabled);

Initialize the extension

To initialize the extension, call setExtensionProperty, and pass in the corresponding keys and values:

  1. Check the license Set the key as bef_effect_ai_check_license and the value as the path of the license file LicenseBag.bundle.

  2. Initialize the effects Set the key as bef_effect_ai_init and the value as the path of the model file ModelResource.bundle and the device name.


    _39
    private void initExtension() {
    _39
    String[] resources = new String[] {"byte_dance/LicenseBag.bundle", "byte_dance/ModelResource.bundle"};
    _39
    loadBundle(resources, () -> {
    _39
    File destFile = getExternalFilesDir(null);
    _39
    _39
    // Get the path of the license file
    _39
    File licensePath = new File(
    _39
    destFile,
    _39
    "byte_dance/LicenseBag.bundle/" + io.agora.rte.extension.bytedance.example.Constants.mLicenseName);
    _39
    _39
    // Check the license
    _39
    try {
    _39
    JSONObject jsonObject = new JSONObject();
    _39
    jsonObject.put("licensePath", licensePath.getPath());
    _39
    setExtensionProperty("bef_effect_ai_check_license", jsonObject.toString());
    _39
    } catch (JSONException e) {
    _39
    Log.e(TAG, e.toString());
    _39
    }
    _39
    _39
    // Get the path of the algorithm model files
    _39
    File strModelDir = new File(destFile, "byte_dance/ModelResource.bundle");
    _39
    _39
    // Initialize the effects
    _39
    try {
    _39
    JSONObject jsonObject = new JSONObject();
    _39
    jsonObject.put("strModelDir", strModelDir.getPath());
    _39
    jsonObject.put("deviceName", "");
    _39
    setExtensionProperty("bef_effect_ai_init", jsonObject.toString());
    _39
    } catch (JSONException e) {
    _39
    Log.e(TAG, e.toString());
    _39
    }
    _39
    });
    _39
    }
    _39
    _39
    _39
    // Convenient for multiple calls to setExtensionProperty
    _39
    private void setExtensionProperty(String key, String property) {
    _39
    mRtcEngine.setExtensionProperty("ByteDance", "Effect", key, property);
    _39
    }

Configure effects

Call setExtensionProperty, and pass in the corresponding keys and values. You can implement the following functions:

  • Set the orientation of the mobile phone
  • Set overlay effects (face beautification, shape beautification, and beauty makeup) and their intensity
  • Set stickers
  • Set filters and their intensity

You can combine calls according to your needs. For the corresponding key and value, refer to the API reference. To use superimposed special effects in an environment that does not use stickers, initialize the composer first. To remove an effect, remove the composer node. See the key-value overview for details.

Run the sample project

The complete sample project is available on GitHub:

PlatformLanguageSample project
AndroidJavaByteDance/android

Take the following steps to run the sample project:

  1. Clone the repository using the following command:


    _1
    git clone https://github.com/AgoraIO-Community/AgoraMarketPlace.git

  2. Open the the Agora Console > Cloud Market > BytePlus Effects page, click to download the Android extension package, and then click Contact Us to obtain an exclusive certificate file.

  3. Unzip the downloaded Android extension package and save all .aar files to the project folder ByteDance/android/app/libs.

  4. Click to download the material package of the extension and save all files to ByteDance/android/app/src/main/assets/Resource.

  5. Open the sample project ByteDance/android in Android Studio.

  6. Sync project with Gradle files.

  7. Save your certificate file with the extension .licbag to the path ByteDance/android/app/src/main/assets/LicenseBag.bundle.

  8. Open the Config.java file and make the following modifications:

    • Replace <YOUR_APP_ID> with your App ID.

    • Replace <YOUR_LICENSE_NAME> with your certificate file name. For example, bytedance_4.2.3.licbag.


    _5
    public interface Config {
    _5
    String mAppId = "<YOUR_APP_ID>";
    _5
    String mToken = null;
    _5
    String mLicenseName = "<YOUR_LICENSE_NAME>";
    _5
    }

  9. Connect a physical Android device (not an emulator), and run the project.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

API reference

This section lists the APIs related to using extensions with the Agora SDK.

BytePlus key-value overview

This section might not provide the latest information. You can find a more detailed explanation from Effects Interface - C, where you use the name of the key to search the corresponding API.

To implement the BytePlus Effects extension in your app, pass in the corresponding key-value pair when calling setExtensionProperty or setExtensionPropertyWithVendor:

Check the license
  • key: bef_effect_ai_check_license
  • value: licensePath. String. The path of the license file.

See Special effects handle license.

Initialize effects
  • key: bef_effect_ai_init
  • value: The value includes:
    • strModelDir: String. The path of the resource file.
    • deviceName: String. The device name. Generally you can enter an empty string ("").

See Initialize special effects handle.

Initialize composer
  • key: bef_effect_ai_set_composer
  • value: strPath. String. The path to composer resource files.

See Initialize composer.

Remove composer
  • key: bef_effect_ai_composer_remove_nodes
  • value: The value includes:
    • nodePaths: String. The array containing effect resource paths.
    • nodeNum: Integer. The length of the array containing effect resource paths.

See Remove composer.

Set the phone orientation
  • key: bef_effect_ai_set_orientation
  • value: orientation. The parameter type is bef_ai_rotate_type. You can set it to 0 (no rotation), 1(rotate 90 degrees clockwise), 2(rotate 180 degrees clockwise), or 3(rotate 270 degrees clockwise.

See Set the phone angle.

Set superimposed effects
  • key: bef_effect_ai_composer_set_nodes
  • value: nodePaths. String. The array containing effect resource paths.

See Set superimposed effects.

Set the intensity of superimposed effects
  • key: bef_effect_ai_composer_update_node
  • value: The value includes:
    • nodePath: String. The path of the effect resource.
    • nodeTag: String. The key of the effect. See Functions of Resource Keys.
    • value: Float. The intensity of a single node for the combined effects.

See Set the intensity of superimposed effects.

Set stickers
  • key: bef_effect_ai_set_effect
  • value: strPath. String. The path of the effect resource.

See Set stickers.

Set filters
  • key: bef_effect_ai_set_color_filter_v2
  • value: strPath. String. The path of the effect resource.

See Set filters.

Set filter intensity
  • key: bef_effect_ai_set_intensity
  • value: fIntensity. Float. The value ranges between [0.0, 1.0], where 0 means disabling the filter.

See Set filter intensity.

vundefined