Table of Contents. WebRTC SDK. Introduction 4. Purpose 4. Scope 4. WebRTC Technology Concept 5. CodeLabs Video 5. Getting started-javascriptsdk 5

(1)

(2)

2

1. Introduction

WebRTC (Web Real-Time Communication) is a technology that enhances the user experience by adding voice, video and data communication to the browser, as well as to mobile applications.

WebRTC is an open-source standard that helps you to incorporate communications into web applications for a fully customizable experience.

WebRTC is a set of standards that enable users to share data and do peer-to-peer video conferencing.

Users can join voice or video calls with a single click, and relevant data can be provided through integrations into the operational systems.

WebRTC solution consists of a WebRTC Gateway and a client application running on a browser or a mobile app.CodeLabs WebRTC client SDK allows developers to integrate WebRTC functionality into Android/iOs and web applications.

1.1. Purpose

This SDK documentation defines the Application Programming Interface (API) use of the Web RealTime Communications (RTC) SDK.

1.2. Scope

This document describes the API that must be implemented to use Codelabs WebRTC SDK to build an Android/iOs or web application that will interact with a server to establish video calls.

The document may be used by developers who want to use the CodeLabs WebRTC provided SDK to build Web RTC clients.

1.3. Benefits

● Simple deployment

● Strong Security and interoperability

● Add real-time video communication to your web applications

(5)

5

2. WebRTC Technology Concept

As you get started with CodeLabs WebRTC, you can also receive support from knowing about the various interrelated protocols and APIs that form the WebRTC technology.

Session Traversal Utilities for NAT (STUN)

A protocol used to discover your public address and identify any restrictions in your router that would prevent you from connecting directly to a peer.

Traversal Using Relays around NAT (TURN)

A protocol to bypass the Symmetric NAT restriction by opening a connection to a TURN server and relaying all information through a server.

Secure Real-time Transport Protocol (SRTP)

A protocol to add encryption, message authentication and integrity, and replay attack protection to RTP data

Stream Control Transmission Protocol (SCTP)

A protocol used for peer-to-peer arbitrary data delivery across browsers.

Interactive Connectivity Establishment (ICE)

A framework that enables the web browser to connect with peers.

ICE Candidate

A method that the sending peer can use to communicate.

3. CodeLabs Video

You'll find everything you need to build video experiences from here. Our getting started guide is for developers who just want to start building.

● Getting started-JavaScriptSDK

● Getting started-iOs SDK

● Getting started-Android SDK

4. Getting started-JavaScriptSDK

The key artefacts you'll use in the Video API to create your video application with the CodeLabs Video JavaScript SDK are described in this document.

You'll find that Video offers a basic cover of WebRTC APIs to make rich audio and video applications easier to develop. From this documentation you can take advantage of the features provided by CodeLabs and the functionality made available by its plugins.

(6)

6

A JavaScript library (assets/js/codelabsRTC_core.js) that can use all interfaces using the same API is available to make it simpler for web developers. The process of developing sessions with the CodeLabs base, adding WebRTC users to plugins, sending and receiving requests and events to the plugins

themselves, and so on is simplified with this library. Since the assets/js/codelabsRTC_core.js library allows use of the webRTC-adapter shim's functionality, it should always be used as a dependency in the web application.

To make it easy to connect with other JavaScript libraries and frameworks, the CodeLabs.js library helps you to provide custom implementations of such dependencies. This feature ensures that CodeLabs.js does not (implicitly) depend on global variables. By default CodeLabs.useDefaultDependencies will be used, that relies on native browser APIs that might require more modern browsers.

In particular, you can do the following when using the CodeLabs features:

1. Add the CodeLabs JavaScript library in your web page;

2. Initialize the CodeLabs JavaScript library and (optionally) passing its dependencies;

3. Establish a session by connecting to the server.

4. Create one or more handles to attach to a plugin (e.g., echo test and/or streaming);

5. Interact with the plugin (sending/receiving messages, negotiating a PeerConnection);

6. Eventually, close all the handles and shutdown the related PeerConnections;

7. Terminate the session.

The measures above will be described in sequence, with descriptions of how to execute them using the low-level API.Consider that we might be providing higher levels of the wrapper to this API in the future in order to address specific requirements, e.g. with each plugin, a higher level API: This would make it much simpler to use the server functionality as a high-level streaming plugin API might, for example, simply ask for the server address and <video> element ID to view the stream, and take care on your behalf of all the above measures.

4.1. Using as Javascript Module

Initially, the CodeLabs library should be included in your project.

The codelabs object is the core of the JavaScript API. When this object is used for the first time in a page, it must be initialized. This can be accomplished by calling the object's static init function, which accepts the following parameters:

● debug: whether or not the JavaScript console should be allowed for debugging, and at what levels

○ true or "all": all debuggers enabled

○ array (e.g., ["trace", "warn"]): only enable selected debuggers (allowed tokens: trace, debug, log, warn, error)

○ false: disable all debuggers

(7)

7

● callback: a user provided function activated when the initialization has been completed

● dependencies: a user provided implementation of CodeLabs library dependencies Consider the following scenario:

Codelabs.init({

debug: true,

dependencies: codelabs.useDefaultDependencies(), callback: function() {

// Done!

});

You can start to create sessions once the library is initialized. In general, any Codelabs session will have several different handles for the plugin, meaning you can run multiple different WebRTC sessions with one and the same plugin with the same user using the same Codelabs session. The same codelabs session is usually needed in every browser tab.

It is quite easy to create a session. You need only to use the new constructor to create a new codelabs object which manages your server interaction. Taking into account the dynamic and asynchronous aspect of the codelabs sessions(events may occur at any time), you may setup several more properties for creating a session and callbacks:

● server: the server's address as a specific address or as an array of addresses to use consecutively during setup to enable automatic fallback/failover;

● iceServers: a list of STUN/TURN servers to use (if this property is omitted, a default STUN server would be used);

● ipv6:whether or not IPv6 candidates should be accumulated;

● withCredentials: if XHR requests should be allowed withCredentials' property or not (false by default, and only valid when using HTTP as a transport, ignored for WebSockets);

● max_poll_events:when polling, the maximum number of events that should be returned; The default is 1 (polling returns an object); forwarding a higher number causes the backend to return an array of objects (again, only true for HTTP use as this is specifically related to long polling;

overlooked for WebSockets);

● destroyOnUnload: whether we should destroy automatically try and destroy this session via Codelabs API when onbeforeunload is called (true by default);

● token , apisecret: optional parameters only needed in case you're Authenticating the Codelabs API ;

● a set of callbacks to be informed of events, namely:

○ success:The session was effectively developed and is now available for use.

○ error: the session was NOT successfully created;

○ destroyed: the session was destroyed and can't be used again.

The method receives these properties and callbacks as properties of a single parameter object: the codelabs constructor only accepts one parameter, which serves as a container for all of the given options.

The success callback is where most program logic begins, such as connecting the peer to a plugin and initiating a media session.

(8)

8

Consider the following scenario:

var codelabs = new codelabs(

{

server: 'http://yourserver:8088/codelabs', success: function() {

// Done! attach to plugin XYZ },

error: function(cause) {

// Error, can't go on...

},

destroyed: function() {

// I should get rid of this }

});

As expected, the server may be a specific address, for example.

{

server: 'http://yourserver:8088/codelabs', // or

server: 'ws://yourserver:8188/', [..]

Alternatively, an array of addresses may be used. Such an array is especially useful if you want the library to check if the WebSockets server is reachable first and fallback to plain HTTP if it isn't, or if you just want to provide a link to multiple instances to try for failover. An example of how to pass a 'try websockets and fallback to HTTP' array is as follows:

{

server: ['ws://yourserver:8188/','http://yourserver:8088/codelabs'], [..]

This object represents your server session once it has been established. A codelabs object can be interacted with in a number of ways. The following properties and methods are described in detail:

● getServer(): returns the address of the server;

● isConnected(): returns true if the codelabs instance is connected to the server, false otherwise;

● getSessionId(): returns the unique codelabs session identifier;

● attach(parameters):tends to create a handle by attaching the session to a plugin; At the same time, more handles to the same or different plugins can be created;

● destroy(parameters): destroys the session with the server, and closes all the handles (and related PeerConnections) the session may have with any plugin as well.

The attach() method is clearly the most significant property, since it's what allows you to use a plugin's functionality to control the media sent and/or obtained through a PeerConnection in your web page.

(9)

9

This method would build a plugin handle for you to use, which you will then use to customize properties and callbacks while calling the attach() method. The attach() method of the codelabs constructor takes a single parameter that can include all of the properties and callbacks mentioned below:

● plugin: the unique package name of the plugin (e.g., codelabs.plugin);

● opaqueId: an optional opaque string meaningful to your application (e.g., to map all the handles of the same user);

● a collection of callbacks for receiving event notifications, namely:

○ success: The handle has been effectively developed and is now ready for use.

○ error: the handle was NOT successfully created;

○ consentDialog: this callback is triggered just before getUserMedia is called (parameter=true) and after it is completed (parameter=false); this means it can be used to modify the UI accordingly, e.g., to prompt the user about the need to accept the device access consent requests;

○ webrtcState: this callback is triggered with a true value when the PeerConnection

associated to a handle becomes active (so ICE, DTLS and everything else succeeded) from the codelabs perspective, while false is triggered when the PeerConnection goes down instead; useful to figure out when WebRTC is up and running between you and codelabs (e.g., to notify a user they're now active in a conference); notice that in case of false a reason string may be present as an optional parameter;

○ iceState: this callback is triggered when the ICE state for the PeerConnection associated to the handle changes: the argument of the callback is the new state as a string (e.g.,

"connected" or "failed");

○ mediaState: this callback is triggered when codelabs starts or stops receiving your media:

for instance, a mediaState with type=audio and on=true means codelabs started receiving your audio stream (or started getting them again after a pause of more than a second); a mediaState with type=video and on=false means codelabs hasn't received any video from you in the last second after a start was detected before; useful to figure out when codelabs started handling your media, or to detect problems on the media path (e.g., media never started, or stopped at some time);

○ slowLink: this callback is triggered when codelabs reports trouble either sending or receiving media on the specified PeerConnection, typically as a consequence of too many NACKs received from/sent to the user in the last second: for instance, a slow link with uplink=true means you notified several missing packets from codelabs, while uplink=false means codelabs is not receiving all your packets; useful to figure out when there are problems on the media path (e.g., excessive loss), to possibly react accordingly (e.g., decrease the bitrate if most of our packets are getting lost);

○ onmessage: a message/event has been received from the plugin;

○ onlocalstream: a local MediaStream is available and ready to be displayed;

○ onremotestream: a remote MediaStream is available and ready to be displayed;

○ ondataopen: a Data Channel is available and ready to be used;

○ ondata: data has been received through the Data Channel;

○ oncleanup: the WebRTC PeerConnection with the plugin was closed;

○ detached: the plugin handle has been detached by the plugin itself, and so should not be used again

(10)

10

/ Attach to echo test plugin, using the previously created codelabs instance codelabs.attach(

{

plugin: "codelabs.plugin.echotest", success: function(pluginHandle) {

// Plugin attached! 'pluginHandle' is our handle },

error: function(cause) {

// Couldn't attach to the plugin },

consentDialog: function(on) {

// e.g., Darken the screen if on=true (getUserMedia incoming), restore it otherwise

},

onmessage: function(msg, jsep) {

// We got a message/event (msg) from the plugin

// If jsep is not null, this involves a WebRTC negotiation },

onlocalstream: function(stream) {

// We have a local stream (getUserMedia worked!) to display },

onremotestream: function(stream) {

// We have a remote stream (working PeerConnection!) to display },

oncleanup: function() {

// PeerConnection with the plugin closed, clean the UI

// The plugin handle is still valid so we can create a new one },

detached: function() {

// Connection with the plugin closed, get rid of its features // The plugin handle is not valid anymore

} });

Consequently, the attach() method enables you to attach to a plugin and define the callbacks to call when something specific occurs. You will use the handle object returned by the success callback to actively communicate with the plugin (pluginHandle in the example).

This Handle object contains several methods for interacting with the plugin or monitoring the status of the handle:

● getId(): returns the unique handle identifier;

● getPlugin(): returns the unique package name of the attached plugin;

● send(parameters): sends a message to the plugin;

● createOffer(callbacks): asks the library to create a WebRTC compliant OFFER;

● createAnswer(callbacks): asks the library to create a WebRTC compliant ANSWER;

● handleRemoteJsep(callbacks): asks the library to handle an incoming WebRTC compliant session description;

● data(parameters): Sends data, where available, via the Data Channel;

(11)

11

● getBitrate(): gets a verbose description of the stream bitrate currently received;

● hangup(sendRequest): tells the library to close the PeerConnection; if the optional sendRequest argument is set to true, then a hangup codelabs API request is sent to codelabs as well (disabled by default, codelabs can usually figure this out via DTLS alerts and the like but it may be useful to enable it sometimes);

● detach(parameters): Dismisses the plugin and removes the handle and, if it persists, dismisses PeerConnection.

The Handle API can sound confusing, but when you get the idea it is pretty straightforward. The first step to be taken to learn the WebRTC API might be the interactions with Peer Connection, but once again the handle would make things much simpler if you understand the WebRTC API.

This is the concept behind its use:

1. To create a Handle object, you use attach();

2. Your application logic can step in with the success callback: You may want to send a message (send({msg}), negotiate with a Plugin for PeerConnection (createOffer, then send({msg, jsep})) at once and wait for anything to happen.

3. The onmessage callback tells you if you have plugin messages.; if the jsep parameter is not null, just pass it to the library, which will take care of it for you; if it's an OFFER use createAnswer (followed by a send({msg, jsep}) to close the loop with the plugin), otherwise use handleRemoteJsep ; 4. If you have taken the initiative to establish a peer Connection or have done so, you will be given a

stream that can be seen in the onlocalstream and/or the onremotestream callbacks on your page.

5. Any plugin will allow you to control the PeerConnection channel: the send method and onmessage callback will allow you to handle this interaction (e.g., to tell the plugin to mute your stream, or to be notified about someone joining a virtual room), while the ondata callback is triggered whenever data is received on the Data Channel if available (and the ondataopen callback will tell you when a Data Channel is available).

In specific, the following paragraphs describe the properties and callbacks involved in the negotiation stage established throughout the Handled API. For the first negotiating phase to build a new PeerConnection out of scratch, the following paragraphs address: knowing how to create or manage renegotiation instead (i.e.

add/delist/replace a source or reboot the ICE) Instead, search for an update to the integrates seamlessly of PeerConnection (renegotiations).

● createOffer takes only one parameter, which can contain any of the properties and callbacks mentioned below:

● media: you can use this property to tell the library which media (audio/video/data) you're

interested in, and whether you're going to send and/or receive any of them; by default audio and video are enabled in both directions, while the Data Channels are disabled; this option is an object that can take any of the following properties:

■ audioSend: true/false (do or do not send audio);

■ audioRecv: true/false (do or do not receive audio);

■ audio: true/false (do or do not send and receive audio, takes precedence on the above);

(12)

12

■ audio: an object with deviceId property (specify the ID of the audio device to capture, takes precedence on the above;

■ videoSend: true/false (do or do not send video);

■ videoRecv: true/false (do or do not receive video);

■ video: true/false (do or do not send and receive video, takes precedence on the above);

■ video: "lowres"/"lowres-16:9"/"stdres"/"stdres-16:9"/"hires"/"hires-16:9" (send a 320x240/320x180/640x480/640x360/1280x720 video, takes precedence on the above; default is "stdres" ) this property will affect the resulting getUserMedia that the library will issue; please notice that Firefox doesn't support the "16:9" variants, which will fall back to the ones; besides, "hires" and "hires-16:9" are currently synonymous, as there's no 4:3 high resolution constraint as of now;

■ video: "screen" (use screen sharing for video, disables audio, takes precedence on both audio and video);

■ video: object with deviceId , width and/or height properties (specify ID of the video device to capture and optionally resolution to use, takes precedence on the above;

■ data: true/false (do or do not use Data Channels, default is false)

■ failIfNoAudio: true/false (whether a getUserMedia should fail if audio send is asked, but no audio device is available, default is false)

■ failIfNoVideo: true/false (whether a getUserMedia should fail if video send is asked, but no video device is available, default is false)

■ screenshareFrameRate: in case you're sharing a screen/application, allows you to specify the frame rate (default=3);

○ trickle: true/false, to tell the library whether you want Trickle ICE to be used (true, the default) or not (false);

○ stream: optional, only to be passed in case you obtained a MediaStream object yourself with a getUserMedia request, and that you want the library to use instead of having it get one by itself (makes the media property useless, as it won't be read for accessing any device);

○ a set of callbacks to be notified about the result, namely:

■ success: the session description was created (attached as a parameter) and is ready to be sent to the plugin;

■ error: the session description was NOT successfully created;

■ customizeSdp: you can modify the SDP generated by the webrtc engine if you need;

● createAnswer takes the same options as createOffer, but requires an additional one as part of the single parameter argument:

○ jsep: the session description sent by the plugin (e.g., as received in an onmessage callback) as its OFFER.

Regardless of whether you use createOffer or createAnswer, the success callback should return a valid jsep object. You can pass this jsep object to the plugin by attaching it to a message in a send request and letting codel abs negotiate a PeerConnection with your application.

Here's an example of how to use createOffer:

(13)

13 // Attach to echo test plugin

codelabs.attach(

{

plugin: "codelabs.plugin.echotest", success: function(pluginHandle) { // Negotiate WebRTC echotest = pluginHandle;

var body = { "audio": true, "video": true };

echotest.send({"message": body});

echotest.createOffer(

{

// No media property provided: by default, // it's sendrecv for audio and video success: function(jsep) {

// Got our SDP! Send our OFFER to the plugin echotest.send({"message": body, "jsep":

jsep});

},

error: function(error) {

// An error occurred...

},

customizeSdp: function(jsep) {

// if you want to modify the original sdp, do as the following

// oldSdp = jsep.sdp;

// jsep.sdp = yourNewSdp;

} });

}, [..]

// Handle msg, if needed, and check jsep if(jsep !== undefined && jsep !== null) { // We have the ANSWER from the plugin echotest.handleRemoteJsep({jsep: jsep});

} },

[..]

onlocalstream: function(stream) { // Invoked after createOffer // This is our video

},

// Invoked after handleRemoteJsep has got us a PeerConnection // This is the remote video

}, [..]

Here's an example of how to use createOffer:

// Attach to echo test plugin codelabs.attach(

{

(14)

14 plugin: "codelabs.plugin.streaming", success: function(pluginHandle) { // Handle created

streaming = pluginHandle;

[..]

}, [..]

// Handle msg, if needed, and check jsep if(jsep !== undefined && jsep !== null) { // We have an OFFER from the plugin streaming.createAnswer(

{

// We attach the remote OFFER jsep: jsep,

// We want recvonly audio/video

media: { audioSend: false, videoSend: false }, success: function(ourjsep) {

// Got our SDP! Send our ANSWER to the plugin

var body = { "request": "start" };

streaming.send({"message": body,

"jsep": ourjsep});

},

// An error occurred...

} });

} },

[..]

onlocalstream: function(stream) {

// This will NOT be invoked, we chose recvonly },

// Invoked after send has got us a PeerConnection // This is the remote video

}, [..]

A more complicated example (such as a Video Call plugin) will combine the two audio and video, allowing you to either send or receive offers from the plugin.It's as simple as determining the type of the jsep object and responding accordingly.

4.2. Updating a PeerConnection

Although the JavaScript APIs mentioned above would serve for the majority of popular situations, changes to a PeerConnection may be necessary for certain instances. This can happen anytime you try to add a new media channel (for example, adding video to an audio-only call), replace an existing one (for example, switching from recording the camera to sharing the screen), or force an ICE restart due to a network update. All of these acts necessitate renegotiation, which necessitates a fresh SDP offer/answer round to upgrade the original PeerConnection.

(15)

15

For createOffer and createAnswer, there are additional properties you can pass to: As is explained in the next paragraphs, most of the properties added in the last section are still accessible.

In createOffer and createAnswer, you can pass to the media the latest features:

● addAudio: if set, start capturing audio if you weren't (will fail if you're sending audio already);

● addVideo: if set, start capturing video if you weren't (will fail if you're sending video already);

● addData: if set, negotiate a datachannel if it didn't exist (is just a synonym for data:true );

● removeAudio: if set, stop capturing audio and remove the local audio track;

● removeVideo: if set, stop capturing video and remove the local video track;

● replaceAudio: if set, stop capturing the current audio (remove the local audio track), and capture a new audio source;

● replaceVideo: if set, stop capturing the current video (remove the local video track), and capture a new video source.

These properties are processed only when a renegotiation is attempted and ignored when a new

PeerConnection is developed. These properties don't substitute the existing media properties but go along with them. It is important, though, to point out that the negotiations that lead to the establishment of a new PeerConnection in the first place usually differ based on which plugin you are trying to do so. Some plugins can make it possible for you to renegotiate, while others may force you to submit another request to initiate a renegotiation. This is especially important for ICE restarts which will be made clearer later.

To provide more detail about how to conduct renegotiations in special situations, in addition to the general and core concepts introduced in this section, please refer to the documents of each plugin.

Here's a small example of how to delete the local video capture in a session using removeVideo:

// Remove local video echotest.createOffer(

{

media: { removeVideo: true }, success: function(jsep) { codelabs.debug(jsep);

echotest.send({message: {audio: true, video: true}, "jsep": jsep});

},

bootbox.alert("WebRTC error... " + JSON.stringify(error));

} });

Alternatively, this other example demonstrates how to add a new video stream to an audio-only PeerConnection:

// Add local video echotest.createOffer(

{

media: { addVideo: true }, success: function(jsep) { codelabs.debug(jsep);

(16)

16

},

} });

This example demonstrates how to replace the video track by combining it with one of the properties we discussed earlier:

// Replace local video echotest.createOffer(

{

media: { video: {

deviceId: "44f4740bee234ce6ddcfea8e59e8ed7505054f75edf27e3a12294686b37ff6a7"

},

replaceVideo: true },

success: function(jsep) { codelabs.debug(jsep);

},

} });

Note that renegotiations involving media improvements (either local or remote), are likely to lead to the new class to the onlocalstream and onremotestream callbacks: as such, be prepared to see those callbacks more than once during a media session for the same peer connection.

4.3. ICE Restarts

ICE restarts don't comply with media shifts, nor in the transportation underlying itself. For example, when a network transition takes place (e.g., the IP address changed, or the user switched from WiFi to 4G). To do this, it is essential to switch new candidates and to reboot connection scans, to find the new best route.

When submitting a new offer, you can only force an ICE restart. To do this, simply add iceRestart:true to your createOffer call, and an ICE restart will be requested.

echotest.createOffer({

iceRestart: true, media: { data: true }, success: function(jsep) {

echotest.send({message: {audio: true, video: true}, jsep: jsep});

} });

(17)

17

We're not looking for any modifications to the media streams in this situation, just an ICE restart. If successful, the client and codelabs will restart the ICE loop and find a new route for the media packets as soon as the response is obtained.

5. Getting started-iOs SDK

The CodeLabs Video iOS SDK helps you to add real-time voice and video into your native iOS apps.

This section gives you an understanding of the key elements in the CodeLabs Video iOS SDK's Video API that you'll use to develop your video application.

5.1. Video API Overview

To begin, here's a concise overview of the CodeLabs Video API.

The code samples below show basic tasks that a developer would want to execute related to a room and its participants.

● A Room represents real-time audio, video, and/or screen-share session, and is the basic building block for a Programmable Video application.

● In a Group Room, media is routed through Codelabs Media Servers. Supports up to 20 participants.

● Participants represent client applications that are connected to a Room and sharing audio and/or video media with one another.

● Tracks represent the individual audio and video media streams that are shared within a Room.

● LocalTracks represent the audio and video captured from the local microphone and camera.

● RemoteTracks represent the audio and video tracks from other participants connected to the Room.

5.2. Requirements

Here's what you require before you start using the iOs SDK:

● Xcode 10.0+

● Swift projects must use Swift 4.0 or higher

● Support for iOS 10.0+ at build time, and iOS 10.0+ at run time

5.3. Prerequisites

You need to perform certain simple tasks before you can start using the iOS CodeLabs Video SDK in your applications.

CocoaPods

To install the dependencies in your project, run pod install.

(18)

18

pod 'CLWebRTC'

5.4. User Permissions

CodeLabs Video requires user consent for features such as sharing video from the camera or audio from the microphone. In case certain permissions (such as the cameras) are denied, consider how the

applications can perform with restricted functionality.

Friendly Name Privacy Usage Description Key

iOS Version

Recommendation

Camera NSCameraUsageD

escription

All Request permissions using

AVCaptureDevice.requestAccess(for:co mpletionHandler:)

Microphone NSMicrophoneUsa geDescription

All Request permissions using

AVAudioSession.requestRecordPermissi on()

5.5. Supported Devices

The iOS SDK supports iOS 10.0 or higher. It is built for armv7, arm64 architectures with Bitcode slices for armv7 and arm64 devices.

5.6. Supported Xcode Versions and Bitcode

For the current project setup, please set Bitcode to No in Build Settings.

For CLWebRTC in Pods, make sure Enable Bitcode is set to No in the Build Settings.

(19)

19

5.7. Code Snippets

Set up variables for the client, video server, and operator respectively.

var rtcManager:RTCClient?

var clientServer: RTCVideoServer?

var rtcOperator: WebRTCOperator?

5.8. Using VideoTrack API

In this section, we will demonstrate how to use the Video Track API to send messages between Participants connected to a Room. You will be able to create important collaboration features such as whiteboarding, screen descriptions, and more using the VideoTrack API.

In the following section, we will demonstrate how to use the VideoTrack API with the iOS SDK.

Set up a video track for a local video track (a video track made by the user).

var localVideoTrack:RTCVideoTrack?

Setup RTCMTLVideoView for a local video track (the user's video track).View as a placeholder

var rtcLocalView:RTCMTLVideoView?

Set up RTCMTLVideoView for remote video track(remote publishers video track).View as a placeholder

var rtcRemoteView:RTCMTLVideoView?

5.9. Set ICE Server

(20)

20

To improve the reliability of IP communications, WebRTC and other VoIP stacks implement support for ICE.

The Interactive Connectivity Establishment (ICE) protocol specifies how to coordinate STUN and TURN to establish a connection between hosts.

Each browser's ICE algorithm selects a candidate IP/port pair from the list received in the other party's SDP and sends a STUN request to it. If the other browser responds, the search is considered successful, and the IP/port pair is marked as a valid ICE candidate.

Once all of the IP/port pairs have been checked for connectivity, the browsers negotiate and select one of the remaining, valid pairs. Following the selection of a pair, media begins to flow between the browsers.

To set up ICE servers

var iceServers = [RTCIceServer]()

iceServers.append(RTCIceServer(urlStrings:["stun:stun.l.google.com:19302"])) iceServers.append(RTCIceServer(urlStrings:["stun:someotherserver:port"])) rtcManager?.defaultIceServer = iceServers

5.10. Setup Manager and Websockets

To communicate using the WebSocket protocol, you must first build a WebSocket object, that will attempt to connect to the server automatically.

The URL to which to connect this should be the URL to which the WebSocket server will respond. This should use the URL scheme wss://.

Setting Manager to control client and setting Websocket URL where the Codelabs engine is deployed.

rtcManager = RTCClient(videoCall: true)

clientServer = RTCVideoServer(url: "wss://address", client: rtcManager!)

5.11. Setup Display ID

Setting the display name (self) ID or username

clientServer?.display = myId

5.12. Setup Operator

Setting Operator and delegate to receive events related to WebRTC

rtcOperator = WebRTCOperator(delegate: self,clSocket: clientServer!)

(21)

21

rtcManager?.delegate = rtcOperator

5.13. Create A Room

CodeLabs Video reflects on the concept of a room. A Room is conceptually a virtual space wherein end- users communicate. A Room, in technical terms, is a computing resource that provides Real-time

Communications (RTC) services to client applications through a set of APIs. Developers can create a Room by following the reference details.

The following code snippet illustrates how to create a room by mentioning the type of connection (Join a room or create room), and roomName as the name of the room to be created.

clientServer?.type = .Create clientServer?.roomName = “Test Room”

5.14. Connect to a Room Using ID

Once a room is created you can connect a Room using an ID by following this code. Once connected, you can send and receive audio and video streams with other Participants who are connected to the Room.

clientServer?.registerMeetRoom(Int64(roomID))

5.15. Disconnect From the Room

To disconnect from the room follow this code.

clientServer?.disconnectMeeting()

5.16. Mute

To mute or unmute the current session, false mutes the connection by ending audio gathered from the user's microphone, true unmutes the connection.

To mute the session

rtcManager?.setAudioEnable(flag: false)

To unmute the session

rtcManager?.setAudioEnable(flag: true)

(22)

22

5.17. Video Feed

To turn off and switch on the video feed follow below code snippets.

To turn off the self video feed

clientServer?.unpublishMyself()

To switch on the video feed

clientServer?.publishMyself()

5.18. Switch Camera

In the case of mobile devices with a front and back camera, and now it’s time to put that into practice in an application. We use the CodeLabsVideo function LocalVideoTrack, passing the constraints for the video device we want.

Switch Camera (back/front camera)

if(localVideoTrack != nil && (localVideoTrack!.source as? RTCAVFoundationVideoSource)?.canUseBackCamera == true){

(localVideoTrack!.source as! RTCAVFoundationVideoSource).useBackCamera = !(localVideoTrack!.source as!

RTCAVFoundationVideoSource).useBackCamera }

5.19. Room Participants

The Video Rooms service represents an intercommunication session using one of CodeLabs' video SDKs.

Users who are connected (Participants) will send and receive video and audio Tracks in the Room. Use this API to get all data related to the participants who are currently in the room.

Contains all the data related to participants who are currently in the room

clientServer?.joinedParticipant?

5.20. Protocols

The following code snippet is a protocol you need to flow to get events for an ios application.

CodelabsRTCClientDelegate,RTCEAGLVideoViewDelegate, RTCVideoServerDelegate

(23)

23

5.21. Methods

If the server fails to connect the following event will be generated.

func serverDidNotConnect(error: Error?)

5.22. Events

The following code snippets for the event.

Event : When local (self) video feed is published:

func rtcClient(_ id: String, didReceiveLocalVideoTrack localVideoTrack:RTCVideoTrack)

Event : When remote video feed is received:

func rtcClient(_ id: String, didReceiveRemoteVideoTrack remoteVideoTrack: RTCVideoTrack)

Event : When remote video feed is removed:

func rtcClient(_ id: String, client: RTCClient, didRemoveRemoteVideoTrack remoteVideoTrack:

RTCVideoTrack)

Event: When there is an error received from the server:

func rtcClient(_ id: String, didReceiveError error: Error)

Event : When there is a change in connection state:

func rtcClient(_ id: String, didChangeConnectionState connectionState: RTCIceConnectionState)

6. Getting started-Android SDK

The CodeLabs Video Android SDK helps you to add real-time voice and video into your native android apps.

This section gives you an understanding of the key elements in the CodeLabs Video android SDK's Video API that you'll use to develop your video application.

6.1. Video API Overview

(24)

24