HTEffect 3D Avatar

The HTEffect 3D Avatar extension enables you to quickly and seamlessly integrate 3D portrait special effects capabilities into your app. It encapsulates the Hongtu HTAvatar core API. You call API functions by setting the appropriate extension parameters.

This page shows you how to integrate the HTEffect 3D Avatar extension into your app.

You use methods provided by Video SDK to pass in appropriate key and value parameters to the extension. The key parameter corresponds exactly to the Hongtu API name, and the value wraps some or all of the API parameters in JSON format.

The development environment requirements are as follows:

• Android Studio 4.1 or higher.
• Android SDK API Level 24 or higher.
• A mobile device that runs Android 4.1 or higher.

• Implemented SDK quickstart for Video Calling.

To implement HTEffect 3D Avatar features in your app, open the SDK quickstart for Video Calling project you created previously, and take the following steps:

1. Download the HTEffect 3D Avatar Android package.

2. Copy all the .aar files from the package to the /app/libs folder of your Android project.

3. Contact Agora to obtain the HTEffect 3D Avatar resource package. Save the required model and prop files to the /app/src/main/assets folder of your project.

4. Open the app/build.gradle file, and under dependencies, add the following:

   
   

   _1

   implementation fileTree(dir: "libs", include: ["*.jar", "*.aar"])

   
   

5. Import the classes you need:

   
   

   _6

   import io.agora.rtc2.Constants;

   _6

   import io.agora.rtc2.IMediaExtensionObserver;

   _6

   import io.agora.rtc2.IRtcEngineEventHandler;

   _6

   import io.agora.rtc2.RtcEngine;

   _6

   import io.agora.rtc2.RtcEngineConfig;

   _6

   import io.agora.rtc2.video.VideoCanvas;

   
   

6. Add in the xml layout file corresponding to the Java file that uses the 3D effects:

   
   

   _5

   <RelativeLayout

   _5

   android:id="@+id/unityView"

   _5

   android:layout_width="1dp"

   _5

   android:layout_height="1dp">

   _5

   </RelativeLayout>

   
   

7. In the Java file that uses 3D effects, add:

   
   

   _2

   unityView = findViewById(R.id.unityView);

   _2

   unityView.addView(mUnityPlayer);

   
   

Take the following steps:

1. Enable the plugin

   When initializing the RtcEngine, call enableExtension to enable the plug-in.

   
   

   _3

   private void enableExtension(boolean enabled) {

   _3

   agoraEngine.enableExtension("Texeljoy", "HTAvatar", enabled);

   _3

   }

   
   

2. Initialize the plugin

   To initialize the plug-in, call setExtensionProperty and pass in the corresponding key and value.

   
   

   _11

   private void initExtension() {

   _11

   // Initialization

   _11

   try {

   _11

   // Offline authentication method

   _11

   JSONObject jsonObject = new JSONObject();

   _11

   jsonObject.put("license", HTKey);

   _11

   setExtensionProperty("htARInitOffline", jsonObject.toString());

   _11

   } catch (JSONException e) {

   _11

   Log.e(TAG, e.toString());

   _11

   }

   _11

   }

   
   

3. Set up a 3D effect

   To implement a 3D effect, call setExtensionProperty and pass in the corresponding key and value. The specific sequence is as follows:

   1. Pass in the key to htARRenderEnable to enable AR special effects.

      
      

      _3

      JSONObject jsonObject = new JSONObject();

      _3

      jsonObject.put ("enable", enable);

      _3

      agoraEngine.setExtensionProperty ("htARRenderEnable", jsonObject.toString());

      
      

   2. Pass in a key to htARSetModel to set the AR model. For example:

      
      

      _2

      agoraEngine.setExtensionProperty("Texeljoy", "HTAvatar",

      _2

      "htARSetModel", "ht_3d_effect_tiger");

      
      

   You can combine multiple calls according to the needs of your scene. Please note that HTEffect 3D Avatar does not support landscape adaptation.

   For the complete key and value descriptions, refer to the Method key overview.

This section contains content that completes the information in this page, or points you to documentation that explains other aspects to this product.

To run the complete sample project available on Github, take the following steps:

1. Clone the sample code repository:

   
   

   _1

   git clone https://github.com/Texeljoy/AgoraHTAvatarExtensionDemo/tree/master/Android

   
   

2. Download the Android plug-in package for HTEffect 3D Avatar. Save all .aar files to the agora-htavatar-extension-demo/Android/app/libs folder.

3. Contact Agora to obtain a certificate file and the HTEffect 3D Avatar resource package.

4. Save the required model and prop files from the resource package to the agora-htavatar-extension-demo/Android/app/src/main/assets folder.

5. Open the project folder agora-htavatar-extension-demo/Android in Android Studio and sync project with Gradle files.

6. Open the file agora-htavatar-extension-demo/Android/app/src/main/java/io/agora/rte/extension/hteffectar/example/Constants.java:

   
   

   _5

   public interface Constants {

   _5

   String mAppId = "<YOUR_APP_ID>";

   _5

   String mToken = null;

   _5

   String mLicenseName = "<YOUR_LICENSE_NAME>";

   _5

   }

   
   

   Replace <YOUR_APP_ID> with your Agora app ID. You don't need to specify mToken and mLicenseName values when running the demo.

7. Open the file agora-htavatar-extension-demo/Android/app/src/main/java/io/agora/rte/extension/hteffectar/example/HtKey.java:

   
   

   _3

   public interface HtKey {

   _3

   String HTKey = "<YOUR_HT_SDK_KEY>";

   _3

   }

   
   

   Replace <YOUR_HT_SDK_KEY> with the test key provided by Agora

8. Connect a physical Android device and run the project.

9. Click htARSetModelto button to set the 3D AR prop effect.

• RtcEngineConfig.addExtension
• enableExtension
• getExtensionProperty
• setExtensionProperty
• IMediaExtensionObserver

When calling the setExtensionProperty (Android) or setExtensionPropertyWithVendor (iOS) method of Video SDK, the following keys and their corresponding functions are supported:

Key	Description
htARInitOffline	Initialization, offline authentication.

Key	Description
htARReleaseBufferRenderer	Resource release function.

Key	Description
htARRenderEnable	Turn on/off the AR special effects function.
htARSetModel	Switch AR model function.
htARMirror	Set whether the image has a mirror function
htARImageFormat	Set image format function
htARRotation	Set image rotation angle function
htARMaxFaces	Set the maximum number of supported faces function
htARCopyResource	Copy the resource path to the sandbox function

value contains the following parameters:

value parameter	description
license	String type, AR offline authentication key.

value contains the following parameters:

value parameter	description
enable	BOOL type, whether to confirm the release of resources.

value contains the following parameters:

value parameter	description
enable	BOOL type, whether to enable the AR function.

value contains the following parameters:

value parameter	description
name	String type, the name of the model.

value contains the following parameters:

value parameter	description
isMirror	BOOL type, whether to set the mirror display of prop special effects, the default is true

value contains the following parameters:

value parameter	description
imageFormat	Int type, enumeration value of image format. For details on the specific value, see HTEARImageFormat.

value contains the following parameters:

value parameter	description
rotation	Int type, enumeration value of image rotation angle. For details on the specific value, see HTEAROrientation.

value contains the following parameters:

value parameter	description
maxFaces	Int type, the maximum number of faces that can be supported. The value is [1, 5]

value contains the following parameters:

value parameter	description
path	String type, the path that needs to be copied to the sandbox.