Multimedia Over IP

(1)

M

u

l

t

i

m

e

d

i

a

o

v

e

r

I

P

(RT

P

,

RT

CP

,

S

I

P

,

RSTP)

(2)

Multimedia Traffic

Production,

Production, transmissiontransmission

and use of data take place

at different times at different times (RealPlayer, Windows (RealPlayer, Windows Media Player) Media Player)

The production, transmission,

and use of data take

place at the same time

(NetMeeting).

Streaming Live A/V

(Like broadcast TV/radio but via Internet, (Like broadcast TV/radio but via Internet, Can’t pause, rewind. The time between Can’t pause, rewind. The time between request and display 1 to 10 seconds. request and display 1 to 10 seconds. Continuous playout)

Continuous playout)

Streaming Stored A/V

(Like DVD but via Internet (Like DVD but via Internet

Prerecorded multimedia content, user Prerecorded multimedia content, user may pause, rewind, forward,…The may pause, rewind, forward,…The time between

time between request and request and display 1 display 1 toto 10 seconds After display start the

10 seconds After display start the playout

playout must be continuous)must be continuous)

Real-Time Interactive A/V

(IP phone, video conferencing. Can’t pause, (IP phone, video conferencing. Can’t pause, rewind. Delay between initial request

rewind. Delay between initial request andand display small: video

display small: video <150ms, audio <400<150ms, audio <400 acceptable. Continuous playout)

(3)

Non-Real Time Multimedia Traffic

Example:

Example: downloading downloading of a video of a video from the Internet. from the Internet. The video has The video has already beenalready been made; it’s a finis

made; it’s a finished product. A hed product. A client HTTP is usclient HTTP is used to ded to downloadownload the video from the video from anan HTTP server and the user

HTTP server and the user views the video at a later time. The views the video at a later time. The production,production, transmission, and use all happen at different times.

transmission, and use all happen at different times.

TCP TCP (port 80) (port 80) TCP TCP (port 80) (port 80)

(4)

Real-Time Multimedia Traffic

Example: a video conference in which a camera is connected to a server that transmits Example: a video conference in which a camera is connected to a server that transmits video information as it is produced. Everything that happens at the server site can be video information as it is produced. Everything that happens at the server site can be displayed on the computer at the client site. This is both multimedia (video) and real-time displayed on the computer at the client site. This is both multimedia (video) and real-time traffic (production and use at the same

traffic (production and use at the same time).time).

Video frames are transmitted

as they are produced and displayed

Multimedia Traffic (cont.)

(5)

Time Relationship

Each packet holds 10 seconds of video information

Assumption: transfer delay is constant and equal 1 second (exaggerated)

The receiver sees the video with the same speed as it is created, the

constant transfer delay is immaterial.

(6)

Jitter

What happens if the transfer delay is not

constant?

T T h h e e s s e e g g a a p p s s b b e e t t w w e e e e n n r r e e c c e e i

i v v e e

d d p p a a c c k k e e t t s s a a r r e e c c a a l l l l e e d d j j i i t t t t e e r r . .

Real-Time Multimedia Traffic (cont.)

(7)

A solution to the jitter problem is use of

timestamps

-

each

packe

t ha

s a t

ime

of

the packet with re

spect to the fi

rst packet (ti

me is measure

d at

the sender’s si

te.)

(20)

The playback is deferred 7 The playback is deferred 7 seconds after the reception seconds after the reception of the first packet, all gaps of the first packet, all gaps eliminated

eliminated

Playback time is separated Playback time is separated

5 sec 5 sec 2 sec 2 sec 7sec 7sec

(8)

Playback Buffer

In order to separate playback time from the arrival time a

playback

buffer

is needed.

Playback Playback (constant (constant bit rate) bit rate) 2 2ndnd packet (TS=10)packet (TS=10) 3 3rdrd packet (TS=20)packet (TS=20) 00:00:01 00:00:01 00:00:15 00:00:15 00:00:27 00:00:27 00:00:11 00:00:11 00:00:25 00:00:25 00:00:37 00:00:37 PLAYBA

PLAYBACK CK BUFFERBUFFER

1 1stst_{packet (TS=0)}_{packet (TS=0)} Arrival Arrival (variable (variable bit rate) bit rate) Playback Playback (constant (constant bit rate) bit rate) 3 3rdrd packet (TS=20)packet (TS=20) 00:00:01 00:00:01 00:00:15 00:00:15 00:00:27 00:00:27 00:00:25 00:00:25 00:00:37 00:00:37 PLAYBACK BUFFER PLAYBACK BUFFER 1 1ststpacket (TS=0)packet (TS=0) Arrival Arrival (variable (variable bit rate) bit rate) 2 2ndnd packet (TS=10)packet (TS=10) Arrival times Arrival times

Arrival and playback pointers at

Arrival and playback

Arrival and playback pointers atpointers at

00:00:08 00:00:08 7*C bits in buffer 7*C bits in buffer 3*C bits in buffer 3*C bits in buffer

Real-Time Multimedia Traffic (cont.)

(9)

Playback Playback (constant (constant bit rate) bit rate) 00:00:01 00:00:01 00:00:15 00:00:15 00:00:27 00:00:27 00:00:11 00:00:11 00:00:37 00:00:37 PLAYBA

PLAYBACK CK BUFFERBUFFER

1 1ststpacket (TS=0)packet (TS=0) Arrival Arrival (variable (variable bit rate) bit rate) Playback Playback (constant (constant bit rate) bit rate) 3 3rdrd packet (TS=20)packet (TS=20) 00:00:01 00:00:01 00:00:15 00:00:15 00:00:27 00:00:27 00:00:25 00:00:25 PLAYBACK BUFFER PLAYBACK BUFFER 1 1ststpacket (TS=0)packet (TS=0) Arrival Arrival (variable (variable bit rate) bit rate) 2 2ndnd packet (TS=10)packet (TS=10)

Arrival and playback pointers at

00:00:28 00:00:28 3 3rdrd ppaacckkeet t ((TTSS==2200)) 22ndnd packet (TS=10)packet (TS=10) 1*C bits in buffer 1*C bits in buffer Buffer empty Buffer empty

(10)

Playback point

Playback point oror plplayayououtt dedelalayy

(constant

(constant offset offset between between datadata generation and playback) generation and playback)

Propagation Propagation delay + max delay + max jitter jitter Time Time B B y y t t e e s s ( ( s s e e q q u u e e n n c c e e n n u u m m b b e e r r s s ) ) Byte generation Byte generation (CBR) (CBR) Playback Playback (CBR) (CBR) Arrival of data Arrival of data (VBR) (VBR)

For most of the voice applications

the pl

the playouayoutt deladelay shouy should be lesld be lesss

than 300ms (for some applications

even less than 150 ms is required)

Network Network

delay delay

Playback Buffer (cont.)

Real-Time Multimedia Traffic (cont.)

(11)

How to

determi

ne the

size o

f playou

t

delay?

D = ED +

β

EV

where:

D

–

Playout

delay (se

t for each

talk spurt

talk spurt)

)

ED

–

Estima

ted average pac

ket delay

EV

–

Estimated

average p

acket delay

variati

on

β

–

safety fact

or (usua

lly

β

= 4)

The computation of

ED

and

EV

is done for each packet and is based on

timestamps:

ED

_i_i

:=

α

ED

_i-1_i-1

+ (1-

α

) (r

_i_i

–

t

)

_i_i

EV

_i_i

:=

α

ED

_i-1_i-1

+ (1-

α

) |r

_i_i

–

t

_i_i

−

ED

_i_i

|

where:

α

–

weighting factor (typically

αα

= 0.998)

r

_i_i

–

time the p

acket

i

is received

t

(12)

!

In order to ensure playback timing and jitter removal

timestamps

are

required.

!

In order to ensure the presence and order of data a

sequence number

sequence number is

is

required.

!

Most of the real-time multimedia applications are video conferencing

where several clients receive data. Therefore the

multicast

mode is

preferred.

!

In order to deal with congestion a mechanism for sender notification

and change of

encoding parameters

must be provided.

!

In order to display streams generated by different standards the

choice

of encoding

must be provided.

!

In order to display audio and video streams (and user data like titles)

within a single A/V session

mixers

are required.

!

In order to use high bit rate streams over a low-bandwidth network the

translators

are required (multimedia payload encoders and decoders)

Requireme

nts for

Real-Time Multimedia Traffic

Real-Time Multimedia Traffic (cont.)

(13)

!

TCP forces the receiver application to wait for

retransmission

(s) in case of

packet l

oss, which ca

uses larg

e delays.

!

TCP cannot support

multicast

.

!

TCP

congestion control

mechanisms decreases the congestion window

when packe

t losses ar

e detected ("

slow start

"). Audio a

nd vid

eo,

on the

other hand, have "natural" rates that cannot be suddenly decreased.

!

TCP

headers

are larger than a UDP header (40 bytes for TCP compared to

8 bytes for UDP).

!

TCP doesn’t contain the necessary

timestamp

and encoding information

needed by the receiving application.

!

TCP doesn’t

allow packet loss

. In A/V however loss of 1-20% is tolerable.

(This kind of loss can be compensated by FEC, see later)

Why real-time data can not use TCP?

(14)

There are several related protocols which support the real-time traffic over

Internet (here are listed the most important ones which will be discussed in

the following slides):

!

RTP

(Real-Time Protocol)

---- Used for realUsed for real-time dat-time data transpoa transport (extenrt (extends UDP, sits beds UDP, sits betweentween application and UDP)

application and UDP)

!

RTCP

(Real-time Control Protocol)

---- Used to exchaUsed to exchange contrnge control informatol information betweeion between sender and recein sender and receiver,ver, works in conjunction with RTP

works in conjunction with RTP

!

SIP

(Session Initiation Protocol)

---- ProvidProvides mechanes mechanisms for estisms for establisablishing calhing calls over IPls over IP

!

RTSP

(Real-Time Streaming Protocol)

---- Allows usAllows user to control diser to control display (rewinplay (rewind, FF, pausd, FF, pause, resume,e, resume,…)…)

!

RSVP

(Resource Reservation Protocol, discussed in next chapter)

---- Intended to Intended to add dadd determinism to eterminism to connectionless connectionless information, pinformation, providesrovides QoS

QoS to to somsome ee extextent.nt.

Protocols

Real-Time Multimedia Traffic (cont.)

(15)

A A p p p p_l_l i i c c a a t t _i_i o o_n_n d d a a e e _m_m o o_n_n k k e e _r_r n n e e _l_l MGCP MGCP /Megaco /Megaco

T

C

P

U

D

P

IPv4, IPv6

H H..332233 SSDDPP SIP SIP R RTTSSPP RRSSVVPP RRTTCCPP RTP RTP H.261, H.261, MPEG MPEG P PPPPP AAAALL33//44 AAAALL55 PPPPPP S Soonneett AATTMM EEtthheerrnneett VV..3344

Signaling

Quality of Service

Reservation

Reservation MeasurementMeasurement

Media

transport

Protocols (cont.)

SDP

(16)

Real Time Protocol (RTP)

(17)

There was a dil

emma whethe

r to implemen

t RTP as a sublayer

of transpor

t layer

or a s a part of application layer. At this point it is common that RTP is

implemented as an

application library

, which executes in user space rather than in

kernel space like all other protocol layers below RTP. (Example:

JMF -Java

Media Framework

).

RTP doesn’t ensure real-time delivery itself, but it provides means for:

!

Jitter

eliminat

ion/reduc

tion (wi

th applica

tion’s p

layout

buffers)

!

Synchronization

of

several audio

and/or

video

streams that

belong

to

the same multimedia session.

!

Multiplexing

of audio/

video

streams that

can

belong to

different

?????

!

Translation

of video a/v streams from one encoding type to another

With the help of RTCP, RTP also provides “hooks” for adding reliability and

flow/congestion control which is implemented within the multimedia application

(this property is sometimes called:

(18)

RTP RTP header header UDP UDP header

header RTP PayloadRTP Payload PaddingPadding

Pad Pad count count

RTP (cont.)

RTP packets are e

RTP packets are encapsulated in UDPncapsulated in UDP

dat

datagramagramss (TCP ca(TCP can not bn not be used due used duee

to overhead and to the fact that most

of real time traffic is multicast)

Some encoding algorithms

require fixed block sizes,

therefore a padding is needed

Variable size,

Variable size, dependindepending ong on

the application (i.e. when

needed). Designed to be small

to decrease the overhead.

One byte

The size of the payload is

The size of the payload is aa

compromise between overhead and

pack

packetizaetizationtion deladelay. y. In In additadditionion, , it it isis

desirable that the

desirable that the multimediamultimedia

packets are as small as possible so

that the packet loss doesn’t decrease

the quality of reception (for

the quality of reception (for exampleexample

20 ms of uncompressed voice has 160

samples (bytes). Loss of 20 ms of

voice

voice is is tolerable. tolerable. Loss Loss of of largerlarger

packets can be

packets can be annoying.annoying.

Port number is an even number

randomly assigned at the session

initiation time

RTP Packets

(19)

Version Version number number (current (current version = 2) version = 2) Padding bit Padding bit ( (11 – – packetpacket contains contains padding) padding) Extension bit Extension bit (

(11 – – fixed header isfixed header is

followed by an e

followed by an extensionxtension

header which is required

by some

by some applicationapplications)s)

F F i i x x e e d d p p a a r r t t ( ( 1 1 2 2 b b y y t t e e s s ) ) Marker bit Marker bit 1

1 – – marks the framarks the frame boundme boundaryary

(e.g. beginning of a talkspurt

in audio, the end of frame in

video) video) C C S S R R C C i i d d e e n n t t i i f f i

i e e r r

s

SSRC –

SSRC – Synchronization source Synchronization source (source that (source that is generating the is generating the RTP packets fRTP packets for this session)or this session)

CSRC –

CSRC – Contributing source Contributing source (used by a (used by a mixer to identify thmixer to identify the contributing sourcee contributing sources)s)

Number of Number of CSRC CSRC identifiers identifiers Incremented for Incremented for each RTP packet each RTP packet (used to indicate (used to indicate the packet loss the packet loss and to restore the and to restore the packet sequence packet sequence (see next slide)

(20)

RFC 3389 RFC 3389 8000 8000 Audio Audio CN CN 13 13 8000 8000 Audio Audio QCELP QCELP 12 12 RFC 3551 RFC 3551 44100 44100 Audio Audio L16 L16 11 11 RFC 3551 RFC 3551 44100 44100 Audio Audio L16 L16 10 10 RFC 3551 RFC 3551 8000 8000 Audio Audio G722 G722 9 9 RFC 3551 RFC 3551 8000 8000 Audio Audio PCMA PCMA 8 8 RFC 3551 RFC 3551 8000 8000 Audio Audio LPC LPC 7 7 RFC 3551 RFC 3551 16000 16000 Audio Audio DVI4 DVI4 6 6 RFC 3551 RFC 3551 8000 8000 Audio Audio DVI4 DVI4 5 5 8000 8000 Audio Audio G723 G723 4 4 RFC 3551 RFC 3551 8000 8000 Audio Audio GSM GSM 3 3 RFC 3551 RFC 3551 8000 8000 Audio Audio G721 G721 2 2 RFC 3551 RFC 3551 8000 8000 Audio Audio 1016 1016 1 1 RFC 3551 RFC 3551 8000 8000 Audio Audio PCMU PCMU 0 0 References References RTP timestamp RTP timestamp Clock rate (Hz) Clock rate (Hz) Type Type Name Name PT PT

RTP (cont.)

Payload Type

(21)

90000 90000 Video Video H263 H263 34 34

Dynamic Payload Types

96-127 96-127 RFC 2250 RFC 2250 90000 90000 A/V A/V MP2T MP2T 33 33 RFC 2250 RFC 2250 90000 90000 Video Video MPV MPV 32 32 RFC 2032 RFC 2032 90000 90000 Video Video H261 H261 31 31 RFC 3551 RFC 3551 90000 90000 Video Video nv nv 28 28 RFC 2435 RFC 2435 90000 90000 Video Video JPEG JPEG 26 26 RFC 2029 RFC 2029 90000 90000 Video Video CellB CellB 25 25 8000 8000 Audio Audio G729 G729 18 18 22050 22050 Audio Audio DVI4 DVI4 17 17 11025 11025 Audio Audio DVI4 DVI4 16 16 RFC 3551 RFC 3551 8000 8000 Audio Audio G728 G728 15 15 RFC 3551 RFC 3551 90000 90000 Audio Audio MPA MPA 14 14 References References RTP timestamp RTP timestamp Clock rate (Hz) Clock rate (Hz) Type Type Name Name PT PT

MPEG1

and

MPEG2

(22)

Timestamp and Sequence Number

Audio:

t

s

=

x

s

n

=

y

Audio stream

t

s

=

x+

1

60

60 s

s

n

=

y

+

1

1 Timestamp clock rate for audio

Timestamp clock rate for audio

8000 Hz

(= 125

µµ

s)

One RTP packet caries

20 ms

of audio samples, each RTP packet sent by

sepa

rate

UDP

data

gram

to

avoi

d p

acke

tiza

tion

del

ay

Timestamp increments by 20,000

µµ

s/125

µµ

s =

160

160 (even if the packet is not

(even if the packet is not

sent after silence suppression)

Packet rate = 1 sec/20 ms =

50 Hz

Number of b

its per RT

P payloa

d for uncom

pressed au

dio = 16

0*8 = 128

0,

for compressed audio typically 8 times less.

Sequence number increments by one for each RTP packet

Audio stream

RTP packet

k-1

RTP packet

k

RTP (cont.)

(23)

Video:

t

s

=

x

s

n

=

y

Video stream

Timestamp clock rate for video is

90,000 Hz

One RTP packet caries

one video frame

(it is possible that one frame is spread

between sev

eral RTP pac

kets)

Each RTP packet sent by separate UDP datagram.

RTP packet rate = 30 Hz (sometimes 25 Hz)

Timestamp increments by (1/30)/(1/90000) = 9000/30 =

3000

, or 9000/25 =

3600

Number of b

its per R

TP payload

for uncom

pressed vid

eo confere

ncing =

352x240x12 = 1,000,000, for compressed audio typically 30 times less.

Required bit rate must cover the overhead (protocol headers and bit stream

headers)

Sequence number increments by one for each RTP packet

Video stream

RTP packet

k-1

RTP packet

k

ts

=

x+

30

00

00 s

(24)

Source Identifiers

Synchron

Synchronization

ization source

source identi

identifier

fier

(SSRC)

Uniquely defines the multimedia source.

The initial value is determined randomly.

Contributing source identifier

Contributing source identifier (

(

CSRC

CSRC)

)

Used by mixers to identify sources in a mix.

RTP (cont.)

(25)

RTP is a protocol that provides basic transport layer for real time applications

but doe

s not pro

vide any m

echanism for e

rror and flow

control, c

ongesti

on

control, quality feedback and synchronization. For that purpose the RTCP is

added as a

companion to RTP

to provide end-to-end monitoring and data

delivery, and QoS

RTCP is responsible for three main functions:

!

Feedback on performance of the application and the

network

!

Correlation and synchronization of different media streams

generated by the same sender (e.g. combined audio and video)

!

The way to convey the identity of sender for display on a

user interface

(26)

RTCP (cont.)

UDP

header

RTCP

header

RTCP Data

8 bytes

8 bytes min 12 bytesmin 12 bytes

RTCP Packets

Variable, 32-bit boundary

RTCP UDP port number = RTP UDP port number + 1

(Usually one port number pair per

(Usually one port number pair per multimedia session)multimedia session)

RTCP packet RTCP packet

UDP

header

RTCP

packet

RTCP

packet

RTCP

packet

Usually several R

Usually several RTCP packetTCP packets are combins are combined in a bed in a bundle of undle of severalseveral packetspackets

which are encap

which are encapsulated in thsulated in the same UDP datagrame same UDP datagram (to reduce the ov(to reduce the overhead due toerhead due to

headers).

(27)

Same as

RTP version

(v = 2)

(v = 2) Padding bitPadding bit

(same as RTP)

The number of reception

report blocks contained

in this packet

RTCP packet type

(see next slide)

2 bits

₁

_b

_i

_t

_s

₅

_b

_i

_t

_s

₈

_b

_i

_t

_s

₁

₆

_b

_i

_t

_s

Message length

Packet type

RR count

P

Version

(28)

XR XR APP APP BYE BYE SDES SDES RR RR SR SR NACK NACK FIR FIR Acronym Acronym RTCP extension RTCP extension 207 207

Application specific functions Application specific functions 204

204

Good

Goodbye –bye – IndiIndicates end ocates end of partif participatcipationion 203

203

Source d

Source description escription items (items (including including CNAME CNAME –– canonicalcanonical name)

name) 202

202

Receiver report for

Receiver report for reception statistics from participants thatreception statistics from participants that are not active

are not active senders (periodically transmitted)senders (periodically transmitted) 201

201

Sender report for transmission and reception statistics from Sender report for transmission and reception statistics from active senders

active senders (periodically transmitted)(periodically transmitted) 200

200

Negative

Negative acknowledgemacknowledgementent 193

193

Full INTRA-frame request Full INTRA-frame request 192 192 Description Description Packet Packet Type Type References: RFC 2032, 3550, 3611 References: RFC 2032, 3550, 3611

RTCP (cont.)

Packet Type

(29)

Message length Message length PT = 200 PT = 200 RR count RR count P P Version Version SSRC of sender SSRC of sender

NTP timestamp (the absolute wall clock time when report was sent) NTP timestamp (the absolute wall clock time when report was sent)

RTP timestamp (relative timestamp used in RTP packets) RTP timestamp (relative timestamp used in RTP packets) Sender’

Sender’s packet couns packet count (RTP packets trat (RTP packets transmitnsmitted by this sendted by this sender soer so farfar in this session)

in this session)

Sender’s octet count (same but total number of RTP

Sender’s octet count (same but total number of RTP payload octets)payload octets) sender) sender)

Sender

information

block

SSRC SSRC Fractio

Fraction lost (fractin lost (fraction of RTP pktson of RTP pkts from this soufrom this source lost since lasrce lost since last rep.)t rep.) Commutative number of lost packets

Commutative number of lost packets

Extended highest seq. number received (the last significant 16 bits) Extended highest seq. number received (the last significant 16 bits) Interarri

Interarrivalval jitter (esjitter (estimattimate of jitter of RTP date of jitter of RTP data from SSRCa from SSRC Last SR timestamp received from this source

Last SR timestamp received from this source

Delay since receiving the last SR report from this source Delay since receiving the last SR report from this source

Reception

report

block

(for each

source)

NTP

NTP – – Network Time ProtocolNetwork Time Protocol used to synchronize clocks of all coused to synchronize clocks of all computers in Internet. Uses UDP/IP,mputers in Internet. Uses UDP/IP,

UTC (Universal Time Coordinated, former Greenwich Mean Time)

Necessary fo

r

synchronizing

A/V streams

(30)

RTCP (cont.)

Receiver Report

Message length Message length PT = 201 PT = 201 RR count RR count P P Version Version SSRC of sender SSRC of sender

Reception statistics from receivers that are not senders. Reception statistics from receivers that are not senders. Same format

Same format as as SR, buSR, but t without without the sender’s information the sender’s information block.block.

SSRC (of the sender source) SSRC (of the sender source) Fractio

Fraction lost (fractin lost (fraction of RTP pktson of RTP pkts from this soufrom this source lost since lasrce lost since last rep.)t rep.) Commutative number of lost packets

Commutative number of lost packets

Extended highest seq. number received (the last significant 16 bits) Extended highest seq. number received (the last significant 16 bits) Interarri

Interarrivalval jitter (esjitter (estimattimate of jitter of RTP date of jitter of RTP data from SSRCa from SSRC Last SR timestamp received from this source

Last SR timestamp received from this source

Delay since receiving the last SR report from this source Delay since receiving the last SR report from this source

Report

block

(for each

source)

(31)

Used by source to

Used by source to provide more information about itself. Useful for user provide more information about itself. Useful for user interfacesinterfaces (GUI displays who is the source in a “human understandable” format)

(GUI displays who is the source in a “human understandable” format) Message length Message length PT = 202 PT = 202 So Soururcece cocoununtt P P Version Version SSRC/CSRC of sender SSRC/CSRC of sender

List of SDES items

for each

_source

SDES item types:

CNAME

CNAME – – Canonical end-point identifier unique amonCanonical end-point identifier unique among all participants likeg all participants like user@

user@hosthost (CNAME doe(CNAME doesn’t chasn’t change evenge even if SSRC chann if SSRC changes)ges) NAME

NAME – – Real user name of sourceReal user name of source EMAIL

EMAIL – – E-mail address ofE-mail address of PHONE

PHONE – – Telephone number Telephone number LOC

LOC – – Geographical locationGeographical location TOOL

TOOL – – Name of application generating the streamName of application generating the stream NOTE

NOTE – – Transient message describing the current stat of the sourceTransient message describing the current stat of the source PRIV

(32)

RTCP (cont.)

RTCP Bye Message

Message length Message length PT = 203 PT = 203 So Soururcece cocoununtt P P Version Version

Used to confirm to receivers that a prolonged silence is due to

Used to confirm to receivers that a prolonged silence is due to departure of one ordeparture of one or more sources rather than network failure.

more sources rather than network failure.

SSRC/CSRC of sender SSRC/CSRC of sender Reason for leaving (optional) Reason for leaving (optional)

for each

departing

source

NOTE: In audio with silence suppression, no packets are

NOTE: In audio with silence suppression, no packets are sent between two talksent between two talk spurts. The receiv

spurts. The receiver might ther might think that ink that the sender hthe sender has departed as departed oror the there is the there is aa

network failure. The BYE message at least removes the suspicion of network failure if network failure. The BYE message at least removes the suspicion of network failure if a source

(33)

Message length Message length PT = 204 PT = 204 So Soururcece cocoununtt P P Version Version

Used to confirm to receivers that a prolonged silence is due to

Used to confirm to receivers that a prolonged silence is due to departure of one ordeparture of one or more sources rather than network failure.

more sources rather than network failure.

SSRC/CSRC of sender SSRC/CSRC of sender Name (ASCII) Name (ASCII) Application-dependent data Application-dependent data

(34)

RTCP (cont.)

Scalability of RTCP

The volume of RTCP traffic

The volume of RTCP traffic may exceed the RTP traffic during a conference may exceed the RTP traffic during a conference sessionsession involving larg

involving larger number er number of participants of participants (normally only on(normally only one participant talks at e participant talks at the timethe time while other participants are listening. In the meanwhile RTCP

while other participants are listening. In the meanwhile RTCP messages are sentmessages are sent periodically regardless if participant is talking or not.) Therefore the RTCP packet periodically regardless if participant is talking or not.) Therefore the RTCP packet

transmission rate

transmission rate is dynamically is dynamically changed dependchanged depending on ing on the numberthe number of participants.of participants. Standard dictates that

Standard dictates that 20%20% of the session bandwidth is allocated to RTCP. In other wordsof the session bandwidth is allocated to RTCP. In other words RTCP RR and SDES packets are

RTCP RR and SDES packets are sent for every 5 RTP packets transmitted.sent for every 5 RTP packets transmitted. In addition,

In addition, 5%5% of the RTCP bandwidth is allocated to particular participant (CNAME).of the RTCP bandwidth is allocated to particular participant (CNAME).

The RTCP

The RTCP transmission intervaltransmission interval of a participant is a function of of a participant is a function of the total number ofthe total number of participants in the RTP session that ensures required 5% of bandwidth alloc

participants in the RTP session that ensures required 5% of bandwidth allocation. For thatation. For that purpose each participant has to co

purpose each participant has to continuously estimate the session size. The transmissionntinuously estimate the session size. The transmission interval is

interval is randomizedrandomized to avoid synchronization effect.to avoid synchronization effect.

RTCP messages

RTCP messages are stackable –are stackable – to amortize to amortize header overhead header overhead multiple RTCP multiple RTCP messages canmessages can be combined and sent in a

(35)

A packet is lost if:

packet neve

r arrived

packet arriv

ed but c

orrupted (R

TC test, IP and

UDP checksu

m tests fai

led)

packet arriv

ed after it

s schedu

led playo

ut

time

RTP and RTCP do not provide error control (as TCP does via ARQ).

How can lost packets be recovered?

Three approaches:

!

Forward Error Correction

(FEC)

!

Interleaving

!

(36)

Generally FEC is

Generally FEC is based on adding redundancy to transmitted data.based on adding redundancy to transmitted data. There are

There are two mechanismstwo mechanisms:: Using

Using parity parity (FEC) (FEC) packetspackets Using redundant A/V streams Using redundant A/V streams

Using parity packets

d

₂₂

d

₁₁

d

₃₃

d

₄₄

. . . . .

d

_n-1_n-1

p

₁₁

FEC RTP packet

Data RTP packets

FEC block

1 1

(( ,,

11 22

,,...,,

n n 11

))

p

p X

=

X O

OR

R d

d d

d

₋₋

If any of k packet gets lost, it can be recovered. For example if

d

₃₃

is lost, then:

3

(( ,,

11 22

,,

44

,,...,,

n n 11

,,

11

))

d

d X

=

X O

OR

R d

d d

d

₋₋

p

Recovering from Packet Loss (cont.)

Forward Error Correction (FEC)

(37)

FEC packets are marked as

dynamic packet type

in RTP header.

A popular algorithm to generate redundant data is

Reed Solomon Erasure

Correcting code

(RSE code).

In determining the FEC parameters

n

and

k

, the following must be taken into

account:

!

Additional FEC packets increase the overhead, i.e. the required network

bandwid

th: sm

all FEC block ca

use large

overhead.

!

Large FEC blocks cause delay in receiver (receiver needs to wait for all

packets i

n FEC bloc

k in ord

er to be abl

e to do t

he packet rec

overy)

This can be extended to several parity packets. The parity packets here can help to

recover the loss of any

n-k

out of

n

packets

d

₂₂

d

₁₁

_d

₃₃

. . . . .

_d

_k_k

_p

₁₁

FEC RTP packets

Data RTP packets

p

₂₂

. . .

p

_n-k_n-k

(38)

Using Redundant A/V Streams

d

₁₁

Scalable

A/V

encoder

r

₁₁

d

₂₂

d

₃₃

d

₄₄

d

₅₅

r

₂₂

r

₃₃

r

₄₄

r

₅₅ A scalable encoder generates two redundant streams: the

A scalable encoder generates two redundant streams: the enhanced-levelenhanced-level and theand the base-level base-level

compressed bit streams. Enhanced-level stream has smaller compression ra

compressed bit streams. Enhanced-level stream has smaller compression rate, higher qualityte, higher quality and longer streams, while base-level streams

and longer streams, while base-level streams have higher compression rate, lover quality andhave higher compression rate, lover quality and smaller streams (example: PCM coded audio at 64 kbps and GSM encoded audio at 13

smaller streams (example: PCM coded audio at 64 kbps and GSM encoded audio at 13 kbps). kbps).

d

₁₁

r

₁₁

d

₂₂

r

₂₂

d

₃₃

r

₃₃

d

₄₄

r

₄₄

d

₅₅

Lost packet

d

₁₁

d

₂₂

r

₃₃

d

₄₄

d

₅₅

_{recovered packet}

Stream with

The streams are then combined for transmission in a single bit stream.

Recovering from Packet Loss (cont.)

FEC (cont.)

(39)

The FEC-based methods for data recovery increase the required bandwidth and

latency. An alternative method which doesn’t increase the bandwidth is

interleaving

interleaving. The method is shown on Internet phone application.

. The method is shown on Internet phone application.

1

5

9

1

3

2

6

1

0

1

4

3

7

1

5

4

8

1

2

1

6

6 Original RTP packets (each containing 20 ms of voice samples):

Original RTP packets (each containing 20 ms of voice samples):

1

2

3

4

5

6

7

8

9

1

0

1

2

1

3

1

4

1

5

1

6

6 20 ms

20 ms

5 ms voice units

The transmission stream is generated by rearranging the voice units within

RTP packets:

Block of interleaved packets

(40)

Interleaving (cont.)

The FEC-based methods for data recovery increase the required bandwidth and

latency. An alternative method which doesn’t increase the bandwidth is

interleaving

interleaving.

.

The method is shown on Internet phone application.

1

5

9

1

3

2

6

1

0

1

4

3

7

1

5

4

8

1

1 2 1

2 1

6

6 Packets received by the receiver:

Packets received by the receiver:

Lost packet

1

2

3

4

5

6

7

8

9

1

0

1

2

1

3

1

4

1

5

1

6

6 Reconstructed RTP payloads after resequencing:

Reconstructed RTP payloads after resequencing:

Lost packet causes small (5 ms) gaps in the restored audio stream. These gaps

can’t be not

iced. A loss of se

veral packe

ts will cause larg

er or

more frequen

t

gaps, which decreases the reception quality gracefully.

This method doesn’t increase bandwidth, but increases delays, which are

proport

ional to t

he length

of interle

aving bl

ocks. This l

imits the u

se in int

eractive

applications like Internet phone. Good for

streaming stored audio

Missing units

Recovering from Packet Loss (cont.)

(41)

This method doesn’t increase bandwidth requirements nor delays. Works with

smaller packets. Based under the assumption that there is a small difference between

two neighboring packets, which is specially true in case of voice.

1

2

3

4

5

1

2

3

4

5 Lost packet

Lost packet

1

2

4

5

1

2

4

5

1

2

3

3 ’

’

4

5

5 Interpolation

Interpolation

Packet recovered by repetition

Packet recovered by interpolation

(more computationally expensive,

4-40 ms

(42)

Session Initiation Protocol (SIP)

RFC 3261, June 2002 RFC 3261, June 2002

SIP is used to set up, modify and ter

SIP is used to set up, modify and terminate an RTP-based multimedia session (like IPminate an RTP-based multimedia session (like IP telephone conversation or videoconferencing).

telephone conversation or videoconferencing). There are several issues related to this:

There are several issues related to this:

!

! How to determine the current IP How to determine the current IP address of the callee? He/she may have address of the callee? He/she may have severalseveral

devices like Internet phone on PC (one at

devices like Internet phone on PC (one at home, one at office), PDA, realhome, one at office), PDA, real phone/cell phone.

phone/cell phone.

!

! The person can have dynamic IP address (DHCP), or The person can have dynamic IP address (DHCP), or the person is not currentlythe person is not currently

available (not signed in), or doesn’t want

available (not signed in), or doesn’t want to answer at this moment.to answer at this moment.

!

! How can we route the cHow can we route the call if we even do not know where the person is all if we even do not know where the person is at theat the

moment. How can we be notified once the person is

moment. How can we be notified once the person is located, or the person islocated, or the person is available.

available.

!

! Once the connection is established, how can we define the Once the connection is established, how can we define the encoding type for theencoding type for the

audio/video stream, or change the

audio/video stream, or change the encoding type during the session,encoding type during the session,

!

! How can we invite new particHow can we invite new participanipants during the calts during the call, perform calll, perform call transfetransfer and callr and call

holding. holding. All these isues

All these isues are addresare addressed in SIP. An alternatsed in SIP. An alternative protoive protocol to SIP iscol to SIP is H.323H.323, which is more, which is more complex and more difficult to implement/configure. SIP is a simple protocol well suited f complex and more difficult to implement/configure. SIP is a simple protocol well suited foror Internet.

(43)

A A p p p p_l_l i i c c a a t t _i_i o o_n_n d d a a e e _m_m o o_n_n k k e e _r_r n n e e _l_l MGCP MGCP /Megaco /Megaco

T

C

P

U

D

P

IPv4, IPv6

H H..332233 SSDDPP SIP SIP R RTTSSPP RRSSVVPP RRTTCCPP RTP RTP H.261, H.261, MPEG MPEG P PPPPP AAAALL33//44 AAAALL55 PPPPPP S Soonneett AATTMM EEtthheerrnneett VV..3344

Signaling

Quality of Service

Reservation

Reservation MeasurementMeasurement

transport

SDP

(44)

SIP is used to transfer calls, terminate calls, and change call SIP is used to transfer calls, terminate calls, and change call parameters in mid-session (such as adding a 3

parameters in mid-session (such as adding a 3-way-way conference). conference). Session Session management management

SIP tells the end point that its phone should be “ringing;” SIP tells the end point that its phone should be “ringing;” SIP is used to agree on session attributes used

SIP is used to agree on session attributes used by theby the calling and called party.

calling and called party. Session setup

Session setup

SIP is used by end points to negotiate media capabilities, SIP is used by end points to negotiate media capabilities, such as agreeing on a mutually supported voice codec. such as agreeing on a mutually supported voice codec. User capabilities

User capabilities

SIP is used by end points to determine whether they will SIP is used by end points to determine whether they will “an

“answswer”er” a ca callall.. User availability

User availability

End points (telephones) notify SIP proxies of their location; End points (telephones) notify SIP proxies of their location; SIP determines which end points will participate

SIP determines which end points will participate in a call.in a call. User location and

User location and registration registration Description Description Function Function

SIP (cont.)

Basic functions supported by SIP

:

(45)

Call transfer request Call transfer request

REFER

Event notification after an explicit/implicit subscription Event notification after an explicit/implicit subscription

NOTIFY

Request notification of call events Request notification of call events

SUBSCRIBE

Used for mid-session signaling Used for mid-session signaling

INFO

Registers a user's current location Registers a user's current location

REGISTER

Solicits information about a server's capabilities Solicits information about a server's capabilities

OPTIONS

Terminates a request, or ongoing transmission Terminates a request, or ongoing transmission

CANCEL

Terminates a connection between users or

Terminates a connection between users or declines a calldeclines a call

BYE

Used to facilitate reliable message exchange for INVITEs Used to facilitate reliable message exchange for INVITEs

ACK

Invites a user to a call Invites a user to a call

INVITE INVITE Description Description SIP Method SIP Method

SIP Commands (Methods)

methods) and responses. They are summarized as follows: methods) and responses. They are summarized as follows:

(46)

Global

Global Failure Failure (e.g. (e.g. 603 -603 - DeclinDecline)e)

6xx

Server Fa

Server Failure (ilure (e.g. 5e.g. 501 -01 - Not ImpNot Implemenlemented)ted)

5xx

Req

Requesuest Failt Failure (eure (e.g. 40.g. 404 -4 - Not foNot foundund, 480 -, 480 - TemTemporporariarilyly unavai

unavailable, lable, 486 486 –– Busy Busy there)there)

4xx

Redire

Redirection (ction (e.g. 30e.g. 302 -2 - MoveMoved tempod temporarily)rarily)

3xx

Suc

Successcessful ful (e.(e.g. g. 200 200 -- OK, OK, 202 202 -- AcceAcceptepted)d)

2xx

Info

Informarmatiotional (nal (e.ge.g. 100 . 100 -- TryTryinging, 180 -, 180 - RinRinginging)g)

1xx 1xx Description Description Code Code

SIP Responses

SIP (cont.)

(47)

BYE

5

Media session over RTP

4 4 200 OK 200 OK 6 6 200 OK 200 OK C=IN IP4 1.2.3.4 C=IN IP4 1.2.3.4 M=audio 48753 RTP/AVP 3 M=audio 48753 RTP/AVP 3 2 2 ACK ACK 3 3 INVITE sip:[email protected] INVITE sip:[email protected] C=IN IP4 12.26.17.91 C=IN IP4 12.26.17.91 m= audio 38060 RTP/AVP 0 m= audio 38060 RTP/AVP 0 1 1

RTP

(12.26.17.91) (12.26.17.91) User User client client Alice Alice User User server server Bob Bob (1.2.3.4) (1.2.3.4) 1 1 2 2 3 3 4 4 SIP response/request SIP response/request

(over well known TCP or

UDP port 5060)

and UDP, port 48753

Encoding: GSM (AVP 3)

and UDP, port 38060

Encoding:

Encoding: µµ Law audio (AVP 0)Law audio (AVP 0)

12.26.17.91 12.26.17.91 1.2.3.41.2.3.4 I IN N V V IITTEE Port 5060 Port 5060 Terminal rings Terminal rings Port Port 5060 5060 OO K K A ACCKK Port Port 48753 48753 Port Port 38060 38060 5 5 6 6 B

B Y Y E E

A