Core concepts

RTC (Real-Time Communication) refers to real-time communication technology, which allows almost instant exchange of audio, video, and other data between the sender and the receiver.

Agora SDKs provide real-time audio and video interaction services, with multi-platform and multi-device support. This includes high-definition video calls, voice-only calls, interactive live streaming, as well as one-on-one and multi-group chats.

This article introduces the key processes and concepts you need to know to use Agora SDKs.

To use Agora SDKs, create an audio and video project in the Agora Console first. See Agora account management for details.

Agora Console is the main dashboard where you manage your Agora projects and services. Agora Console provides an intuitive interface for developers to query and manage their Agora account. After registering an Agora Account, you use the Agora Console to perform the following tasks:

• Manage the account
• Create and configure Agora projects and services
• Get an App ID
• Manage members and roles
• Check call quality and usage
• Check bills and make payments
• Access product resources

Agora also provides RESTful APIs that you use to implement features such as creating a project and fetching usage numbers programmatically.

Agora uses the following basic concepts:

The App ID is a random string generated within Agora Console when you create a new project. You can create multiple projects in your account; each project has a different App ID. This App ID enables your app users to communicate securely with each other. When you initialize Agora Engine in your app, you pass the App ID as an argument. The App ID is also used to create the authentication tokens that ensure secure communication in a channel. You retrieve your App ID using Agora Console.

Agora uses this App ID to identify each app, provide billing and other statistical data services.

For applications requiring high security in a production environment, you must choose a App ID + Token mechanism for user authentication when creating a new project. Without an authentication token, your environment is open to anyone with access to your App ID.

An App certificate is a string generated by Agora Console to enable token authentication. It is required for generating a Video SDK or Signaling authentication token.

To use your App certificate for setting up a token server, see Create and run a token server.

A token is a dynamic key that is used by the Agora authentication server to check user permissions. You use Agora Console to generate a temporary token for testing purposes during the development process. In a production environment, you implement a token server in your security infrastructure to control access to your channels.

For more information, see Secure authentication with tokens.

After obtaining the App ID, App Certificate, and Token in the Agora Console, you can start implementing basic audio and video communication in your app.

Agora uses the channel name to identify a channel. Users who specify the same channel name join a common channel and interact with each other. A channel is created when the first user joins. It ceases to exist when the last user leaves.

You create a channel by calling the methods for transmitting real-time data. Agora uses different channels to transmit different types of data. The Video SDK channel transmits audio or video data, while the Signaling channel transmits messaging or signaling data. The Video SDK and Signaling channels are independent of each other.

Additional components provided by Agora, such as On-Premise Recording and Cloud Recording, join the Video SDK channel and provide real-time recording, transmission acceleration, media playback, and content moderation.

The SDK applies different optimization methods according to the selected channel profile. Agora supports the following channel profiles:

Channel profile	Description
COMMUNICATION	This profile is suitable for one-on-one or group calls, where all users in the channel talk freely.
LIVE_BROADCASTING	In a live streaming channel, users have two client roles: host and audience. The host sends and receives audio or video, while the audience only receives audio or video with the sending function disabled.

A stream is a sequence of digitally-encoded coherent signals that contains audio or video data. Users in a channel publish local streams and subscribe to remote streams from other users.

Publishing is the act of sending a user’s audio or video data to the channel. Usually, the published stream is created by the audio data sampled from a microphone or the video data captured by a camera. You can also publish media streams from other sources, such as an online music file or the user’s screen. After successfully publishing a stream, the SDK continues sending media data to other users in the channel. By publishing the local stream and subscribing to remote streams, users communicate with each other in real-time.

Subscribing is the act of receiving media streams published by remote users to the channel. A user receives audio and video data from other users by subscribing to one or more of their streams. You either directly play the subscribed streams or process incoming data for other purposes such as recording or capturing screenshots.

A User ID (UID) identifies a user in a channel. Each user in a channel should have a unique user ID. If you do not specify a user ID when the user joins a channel, a UID is automatically generated and assigned to the user.

A user role is used to define whether users in the channel have permission to publish streams. There are two user roles:

• Host: A user who can publish streams in a channel.
• Audience: A user who cannot publish streams in a channel. Users with this role can only subscribe to remote audio and video streams.

The connection between the SDK and the channel. When you need to publish or receive multiple streams in multiple channels, a connection is used to specify the target channel.

Agora's core engagement services are powered by its Software-Defined Real-time Network (SD-RTN™) that is accessible and available anytime, anywhere around the world. The software-defined network isn’t confined by device, phone numbers, or a telecommunication provider’s coverage area like traditional networks. Agora SD-RTN™ has data centers globally that cover over 200+ countries and regions. The network delivers sub-second latency and high availability of real-time video and audio anywhere on the globe. With Agora SD-RTN™, Agora can deliver live user engagement experiences in the form of real-time communication (RTC) with the following advantages:

• Unmatched quality of service
• High availability and accessibility
• True scalability
• Low Cost

The following diagram shows the basic workflow of using the Agora SDK to implement basic audio and video interaction.

Agora uses the following basic concepts:

In audio interaction, the main functions of the audio module are as shown in the figure below:

After you call registerAudioFrameObserver, you can obtain the raw audio data at the following observation points in the audio transmission process:

1. Obtain the raw audio data of ear monitoring through the onEarMonitoringAudioFrame callback.
2. Obtain the captured raw audio data through the onRecordAudioFrame callback.
3. Obtain the raw audio playback data of each individual stream through the onPlaybackAudioFrameBeforeMixing callback.
4. Obtain the raw audio playback data of all mixed streams through the onPlaybackAudioFrame callback.
5. Obtain the raw audio data after mixing the captured and playback audio through the onMixedAudioFrame callback.
   (5) onMixedAudioFrame = (2) onRecordAudioFrame + (4) onPlaybackAudioFrame

The audio output device used by the app when playing audio. Common audio routes include wired headphones, earpieces, speakers, Bluetooth headphones, and others.

The APIs used by the audio module are as follows:

• Enable local audio collection: enableLocalAudio
• Set local playback device: setPlaybackDevice
• Set up audio routing: setDefaultAudioRouteToSpeakerphone

The following diagram shows the main functions of the video module in video interaction:

The figure shows the following observation points:

1. POSITION_POST_CAPTURER_ORIGIN.
2. POSITION_POST_CAPTURER, corresponds to the onCaptureVideoFrame callback.
3. POSITION_PRE_ENCODER, corresponds to the onPreEncodeVideoFrame callback.
4. POSITION_PRE_RENDERER, corresponds to the onRenderVideoFrame callback.

The APIs used by the video module are as follows:

• Enable local video collection: enableLocalVideo
• Local preview: setupLocalVideo → startPreview
• Video rendering shows: setupRemoteVideo