This guide covers all concepts and features to develop your communication client application with ApiRTC.

apiKey

The apiKey is a 32 characters string associated with an Enterprise. It allows associating ApiRTC usage with your project for quota and billing.

An apiKey is required to start using the client ApiRTC APIs.

UserAgent

The UserAgent open_in_new is an entrypoint to ApiRTC. This is the first object to instantiate. It represents the local user that will participate to communications.

There are two ways to instantiate a UserAgent:

with apiKey

This method is useful to use ApiRTC anonymously, or with external user management.

In this case the uri is apiKey prefixed with "apiKey:" string:

with username

This method is useful to further authenticate with ApiRTC User management.

In this case the uri is username prefixed with "apirtc:" string:

UserData

UserData is a class that holds a data object to store values associated to a UserAgent. Make sure to call UserData.setProp(key, value) to set a property.

Once connected to Session, call userData.setToSession() to make other connected peers notified of UserData properties change through the Session.contactListUpdate event.

For that purpose, the data object associated to Session.contactListUpdate event has a userDataChanged property which is an array of Contacts for which UserData has changed.

MediaDevices

UserAgent's mediaDeviceChanged event can be listened to in order to get notified of the list of devices available to the browser:

Here is what the mediaDevices object looks like:

This is useful to propose a list of available media devices to the user.

Session

A Session open_in_new instance is obtained when connecting to the ApiRTC platform.

A session with ApiRTC platform needs to be established in order to access connected features of ApiRTC such as chat or video conferencing.

This is done through the UserAgent.register(options) open_in_new method. Some options (type RegisterInformation open_in_new) parameters control the authentication mechanisms.

Authentication

Getting a session supports the following authentication mechanisms:

No auth


on a UserAgent created with apiKey

External user management JWT (JSON Web Token)

on a UserAgent created with apiKey

ApiRTC user management

on a UserAgent created with username

Presence

In an ApiRTC client application, remote users are Contactopen_in_new.

The presence feature allows to get notified to all users currently connected with same apiKey. It notifies on contacts, connected with same Enterprise(apiKey).

Groups

A notion of groups comes with this feature for better control. To make a user connect within some group, set the RegisterInformation.groups in the UserAgent.register(options) options.

The data object associated to Session.contactListUpdate event has joinedGroup and leftGroup properties to carry information on which Contact joined of left which group:

Data exchange

Data can be exchanged with Contacts by using the Contact.sendData(object) open_in_new method:

To receive the data, listen on the session contactData open_in_new event:

Conversation

A conversation is the way to gather participants to exchange medias. It can be text message, audio/video streams, files...

Whenever there are 2 participants or more, a conversation takes place.

getOrCreate

To get a Conversation open_in_new instance, the Session method getOrCreateConversation(name, options) open_in_new shall be used.

The name is a free string.

meshModeEnabled enables the mesh mode (otherwise SFU star topology is used by default).

The stream exchanges will remain in this mode until:

  • the number of participant goes over 4,
  • or too many packet loss is detected for one participant.

Then the conversation will automatically switch to star topology mode using ApiRTC SFUs infrastructure.

Setting both meshModeEnabled and meshOnlyEnabled to true forces the conversation to remain mesh only.

moderationEnabled enables moderation on the Conversation to create. This option may change the behavior of the joining process depending on whether moderator option is true or false. Note that every party must enable this option to have consistent moderation.

moderator adds the local user to the list of moderators for the conversation.

Join

Conversation.join() makes the local user enter the conversation. Note that this method returns a Promise and one must wait for it to be fulfilled before doing anything else on the conversation.

A good practice is to register all required Conversation event listeners before calling the join method:

Leave

Conversation.leave() makes the local user leave the conversation. All ongoing calls will be automatically closed.

A good practice is to destroy the Conversation after leaving it. (Except if you wish to be able to join it back)

Moderation

When enabled, moderation allows a group of users (moderators) to control who is allowed to participate to the conversation.

Activation

In order to activate the moderation for a conversation, every party (moderator or not) must explicitly set the moderationEnabled option to true when calling getOrCreateConversation open_in_new.

Additionally, a party that shall be a moderator must set the moderator option to true as well.

Joining process

If the local user is moderator, then the join() will resolve immediately.

But if the local user is not moderator, then the join() will only resolve when a moderator allows it. In the meanwhile, the user will be put in a waiting room.

Waiting Room

The waiting room is a presence group associated to the conversation and it allows to know which participants are currently waiting for a moderation answer.

Events contactJoinedWaitingRoom and contactLeftWaitingRoom will be triggered respectively upon the arrival and departure of a user to/from the waiting room:

Then the moderator can allow or deny a contact to enter the conversation:

Eject

Moderators have the ability to eject another participant from the conversation.

To get notified of a participant ejection, listen on the participantEjected event. The event data object wears a self boolean set to true if the current local user is the ejected participant.

Local Streams

Camera

Acquiring camera local stream is done through the UserAgent.createStream(options) method. The browser asks the user to choose among available media devices. The Promise resolves with a Stream open_in_new instance.

Possible options (CreateStreamOptions open_in_new) are:

constraints is of type MediaStreamConstraints open_in_new

filters is an array of type FilterDescriptor open_in_new

Screen sharing

Acquiring screen sharing local stream is done through a Stream static method:

Publish/Unpublish

Publishing a local stream makes it available for remote peer participants to subscribe and view it.

The local user (UserAgent) must have joined the conversation to publish a stream.

PublishOptions

Conversation.publish(localStream, options) can take optionally take PublishOptions second parameter object to control publication. Please check reference for details on PublishOptions open_in_new.

Unpublishing a local stream makes it unavailable for remote peer participants to subscribe and stops sending media stream to peers.

Remote Streams

Handle remote streams availability

ApiRTC notifies when stream availability changes through the Conversation.streamListChanged event.

This event is triggered once for each existing stream, and then again every time a new stream is published to the conversation.

The streamInfo.contact.getId() and streamInfo.streamId can be useful to identify remote which peer published one stream.

Subscribe

A remote stream is subscribed to using the Conversation.subscribeToStream(streamId) method. It takes the id of stream provided in the StreamInfo data object:

SubscribeOptions

Conversation.subscribeToStream(streamId, options) can take optionally take SubscribeOptions open_in_new second parameter object to control subscription.

Unsubscribe

Unsubscribing to a remote stream is done by the Conversation.unsubscribeToStream(streamId) method.

Manage media streams

Once a stream has been subscribed, ApiRTC notifies with an actual Stream instance through the streamAdded event.

This event is triggered every time the actual media stream is available to be displayed.

Whenever a media stream becomes unavailable, ApiRTC notifies the conversation with a streamRemoved event.

Stream display

In order to display or remove media element in DOM, you can use our helpers:

  • Stream.addInDiv() and Stream.removeFromDiv() to add/remove <video> element within an existing <div>
  • Stream.attachToElement(domElement) to directly attach to a <video> element.

We manage some devices specificities in our helpers that can help avoid media plays issue. (for instance with Safari iOS)

Stream constraints

Constraints are camera properties that can be set: resolution, brightness, contrast, frameRate, saturation, torch, zoom.

Capabilities are supported properties and value ranges. Settings are the current properties values.

ApiRTC allows to access constraints, capabilities, settings on both local and remote streams, using the same methods. This means you can easily control both local and remote devices.

Stream.applyConstraints(constraints:MediaStreamConstraints open_in_new) method returns a Promise resolved when all constraints are applied:

Stream.getConstraints() returns in a Promise all properties that were modified and their current values:

Example of a constraints data object:

In this example, only the video.frameRate was modified from it's default value to be at 10.

In addition, supported properties can be queried using Stream.getCapabilities() that returns a Promise with accepted values ranges:

Example of a capabilities data object:

In this example video.frameRate property may be set between 0 and 30.

Finally, properties values can be checked with Stream.getSettings() that returns in a Promise all current settings:

Example of a settings data object:

In this example video.frameRate is a supported property and it's actual value is 30.

video.zoom is not a supported property for this combination of device/camera/navigator as it is not present in the returned object.

QoS statistics

Conversation event callStatsUpdate provides statistics information on media stream exchanges quality of service.

Depending on whether the stream is sent or received, the event data object (callStats) holds the following information:

For a local stream, qos info on sent media

For a remote stream, qos info on received media

The callStats.streamId is useful to associate data to corresponding stream.

Speaker detection

To display which participant is currently talking in a Conversation, enable the feature:

Then, listen on the audioAmplitude event:

the event data object (amplitudeInfo) holds the following information:

When the participant speaks and amplitude goes over the threshold configured during feature enabling, event with isSpeaking set to true is fired.

When the participant does not speak anymore, the event is fired again with initial amplitude value that triggered the event, but this time isSpeaking is false.

Audio/Video Mute

To control local or remote stream audio/video mute, use the following Stream methods:

Whiteboard

Will be documented soon.

Recording

ApiRTC platform allows to record a composite video of a conversation. The video will be composed of all streams exchanged in the conversation and will be stored in ApiRTC's database.

To start recording a conversation:

Refer to startRecording(options) open_in_new for options details.

Example of recordingInfo (RecordingInfo open_in_new):

To stop recording a conversation:

Once the recording is stopped, the ApiRTC platform wil process it and make it available for download. To get notified when a record is available:

When the video is available, you can use the RecordingInfo.mediaURL to download it.

Chat messages

Will be documented soon.

Background subtraction

ApiRTC allows to create a background blurred stream based on an original stream using Stream.blur().

To start the blur process on a stream:

The returned blurredStream is a an instance of Stream and is a copy of the base stream with a blur process applied on it. It means that there is a strong dependance between the child and his parent (e.g. If the parent audio is muted the child audio will be too, if the parent is released the child also). Both streams needs to be handled by the application to keep the blur process going on.

Use the blurred stream as a local stream:

To stop the blur process:

Both methods have the same result, the only difference is the object on which you call the method.

Good practices

  • Register events required for a feature BEFORE calling the methods involving the feature.
  • Always implement error cases
  • Always implement objects cleanup