Record captions

Real-Time STT can record the caption text it produces, generate a .vtt file, and then store it in the cloud storage as configured. Recorded captions are designed for playback with audio and video recordings. The .vtt file can also be used to generate meeting minutes and topic summaries, conduct sentiment analyses and content moderation, and so on. Caption recording entails no additional fees.

Since cloud recording and transcription tasks are running on different servers, enable the ntpTimestamp parameter for cloud recording to sync the time with transcription.

This page explains how to record captions and sync them with the audio or video files generated by Cloud Recording.

Prerequisites

To follow this procedure, you must:

Have a valid Agora Account.
Have a valid Agora project with an app ID and a temporary token or a token server. For details, see Agora account management.
Have a computer with access to the internet. If your network has a firewall, follow the steps in Firewall requirements.
Join a Video SDK channel as a host and start streaming. Refer to the Voice SDK quickstart guide.

Make sure Real-Time STT is enabled for your app.
Enable ntpTimestamp for cloud recording.

Implementation

Record the captions

To record the captions, follow the API call sequence from the REST Quickstart and modify the start request to include caption recording parameters as follows:

curl --location -g 'https://api.agora.io/v1/projects/{{appId}}/rtsc/speech-to-text/tasks?builderToken={{tokenName}}' \
--header 'Content-Type: application/json' \
--data '{
{{
     "languages": [
         "<YourTranscribeLanguages>"
     ],
     "maxIdleTime": 50,
     "rtcConfig": {
         // No change on rtcConfig.
     },
     "captionConfig": {                               // The caption recording configurations.
         "storage": {
             "accessKey": "<YourOssAccessKey>",       // The OSS access key.
             "secretKey": "<YourOssSecretKey>",       // The OSS secret key.
             "bucket": "<YourOssBucketName>",         // The storage bucket name.
             "vendor": <YourOssVendor>,               // The OSS vendor.
             "region": <YourOssRegion>,               // The OSS region.
             "fileNamePrefix": ["<YourOssPrefix>"]    // Optional, an array of strings. Sets the path of the recorded files in the third-party cloud storage.
         }
     },

Sync the files

The m3u8+vtt file generated by Real-Time STT and the m3u8+ts file generated by Cloud Recording are two independent files, with different time stamps. The Cloud Recording time stamp starts at 0, while Real-Time STT uses the system time stamp. If either process starts abnormally, the media files generated by the two services may be out of sync during playback.

Agora provides a post-processing script that lets you sync the m3u8+ts and m3u8+vtt files. To run this script, take the following steps:

Unzip the post-processing script to a local folder.
Run the script on your transcription files:

_21 _2python3 insert_subtitle.py --av audio_dir/audio_ts.m3u8 --subtitle subtitle_dir/subtitle.m3u8 --output output_dir/ --overwrite

If ffmpeg/ffprob are not in your PATH, use -ffmpeg_path to specify the path.
Play the synchronized files:
1. Run the following command to start the HTTP server:
  
  _1python3 -m http.server --bind 127.0.0.1 -doutput_dir
2. In your browser, enter the following URL:
  
  _1http://127.0.0.1:8000/player_demo.html

Supported OSS vendors

vendor: Number. The third-party cloud storage vendor. The following are supported:

region: Number. The region information specified for the third-party cloud storage. The service only supports regions in the following lists:

Amazon S3 (vendor = 1):
- 0: US_EAST_1
- 1: US_EAST_2
- 2: US_WEST_1
- 3: US_WEST_2
- 4: EU_WEST_1
- 5: EU_WEST_2
- 6: EU_WEST_3
- 7: EU_CENTRAL_1
- 8: AP_SOUTHEAST_1
- 9: AP_SOUTHEAST_2
- 10: AP_NORTHEAST_1
- 11: AP_NORTHEAST_2
- 12: SA_EAST_1
- 13: CA_CENTRAL_1
- 14: AP_SOUTH_1
- 15: CN_NORTH_1
- 16: CN_NORTHWEST_1
- 17: US_GOV_WEST_1
- 20: AP_NORTHEAST_3
- 21: EU_NORTH_1
- 22: ME_SOUTH_1
- 23: US_GOV_EAST_1
- 24: AP_SOUTHEAST_3
- 25: EU_SOUTH_1
- 28: IL_CENTRAL_1
Alibaba Cloud (vendor = 2):
- 0: CN_Hangzhou
- 1: CN_Shanghai
- 2: CN_Qingdao
- 3: CN_Beijing
- 4: CN_Zhangjiakou
- 5: CN_Huhehaote
- 6: CN_Shenzhen
- 7: CN_Hongkong
- 8: US_West_1
- 9: US_East_1
- 10: AP_Southeast_1
- 11: AP_Southeast_2
- 12: AP_Southeast_3
- 13: AP_Southeast_5
- 14: AP_Northeast_1
- 15: AP_South_1
- 16: EU_Central_1
- 17: EU_West_1
- 18: EU_East_1
- 19: AP_Southeast_6
- 20: CN_Heyuan
- 21: CN_Guangzhou
- 22: CN_Chengdu
- 23: CN_Nanjing
- 24: CN_Fuzhou
- 25: CN_Wulanchabu
- 26: CN_Northeast_2
- 27: CN_Southeast_7
For details, see Alibaba Cloud documentation.
Tencent Cloud (vendor = 3):
- 0: AP_Beijing_1
- 1: AP_Beijing
- 2: AP_Shanghai
- 3: AP_Guangzhou
- 4: AP_Chengdu
- 5: AP_Chongqing
- 6: AP_Shenzhen_FSI
- 7: AP_Shanghai_FSI
- 8: AP_Beijing_FSI
- 9: AP_Hongkong
- 10: AP_Singapore
- 11: AP_Mumbai
- 12: AP_Seoul
- 13: AP_Bangkok
- 14: AP_Tokyo
- 15: NA_Siliconvalley
- 16: NA_Ashburn
- 17: NA_Toronto
- 18: EU_Frankfurt
- 19: EU_Moscow
Kingsoft Cloud (vendor = 4):
- 0: CN_Hangzhou
- 1: CN_Shanghai
- 2: CN_Qingdao
- 3: CN_Beijing
- 4: CN_Guangzhou
- 5: CN_Hongkong
- 6: JR_Beijing
- 7: JR_Shanghai
- 8: NA_Russia_1
- 9: NA_Singapore_1
Microsoft Azure (vendor = 5): The region parameter has no effect, whether set or not.
Google Cloud (vendor = 6): The region parameter has no effect, whether set or not.
Huawei Cloud (vendor = 7):
- 0: CN_North_1
- 1: CN_North_4
- 2: CN_East_2
- 3: CN_East_3
- 4: CN_South_1
- 5: CN_Southwest_2
- 6: AP_Southeast_1
- 7: AP_Southeast_2
- 8: AP_Southeast_3
- 9: AF_South_1
- 10: SA_Argentina_1
- 11: SA_Peru_1
- 12: NA_Mexico_1
- 13: SA_Brazil_1
- 14: LA_South_2
- 15: SA_Chile_1

You are viewing Agora Docs forBetaproducts and features. Switch to Docs

Prerequisites​

Implementation​

Record the captions​

Sync the files​

Supported OSS vendors​

Prerequisites

Implementation

Record the captions

Sync the files

Supported OSS vendors