Multimedia Communications

Lesson

Lesson 1

1

M

M

_u

_u

_lt

_lt

ii

_me

_me

_d

_d

ii

_a

_{a  Concept &}

  _{Concept &}

_To

_To

 _pics

 _pics

Multimedia

Multimedia Concep

Concep

tt

Multimedia

Multimedia Computing

Computing

Multimedia

Multimedia Classi

Classif 

f 

c

ca

ation

tion

Multimedia

Multimedia Topics

Topics

Multimedia Driving

Multimedia Driving F

Forc

orces

es

Multimedia Applica

Multimedia Applica

tt

ons

ons

Course Ou

Course Ou

tt

ine

ine

What is

What is Mult

Multimed

imedia

ia

Multi

Multi::more thanmore than oneone Medium

Medium singusingull_aar!:r!: middle" middle" iinntteermediar rmediar ##"" mmeaeann

Media

Media _{pluraplura}ll_{!:!: meansmeans for conve#ingfor conve#ing infor infor matiomationn}

$$MM_{ediaedia in in thethe p preress"ss"nene%sp%spapeaper r "" raradd}ii_{oo aandnd}T&T&_{cconte'onte'tt $$}

$$MM_{ediaedia in in comcommumuniniccatiations:ons: cca(les"a(les" satsatellell}ii_{te"te" net%net%or or ) )} 

$ Media in

$ Media in computer storage:computer storage: flopp#" CD"flopp#" CD" DD&D&D_{"" *D" +,- *D"}

+,-$$MM_{ediaedia in in **C.C. coconte'nte'tt:: ttee't"'t" imageimage"" aaudio"udio" video"video" C/C/}

Multimedia

Multimedia:: refers to variousrefers to various ii_{nformationformationn forms forms te't" image"te't" image" audaud}ii_{o"o" vvideo"ideo"}

graphics"

graphics" and animation and animation in in a variet# of a variet# of applicationapplication environmeenvironmentsnts

 product"

 product" application"application" technolog#"technolog#"  p pll_{atfor atfor m"m"  (oard" device"  (oard" device" net%net%or) or)} 

What is

What is Multimedia

Multimedia

in

in terms

terms

of

of

Com

Com putin

 puting

g

C o

C o m pm puut i t i nng g : Comput: Computer$(er$(asedased tetechnochnoll_{ogiogieses and and appliapplicacatitionsons}

What

What computers0computers0 &&_{ariariousous  forms of  forms of comcom puter  puter s1des1devicesvices22}

.n terms of

.n terms of computing"computing" fourfour fundamentalfundamental multimultimemeddii_{aa aattri(ttri(utes:utes:}

$$ AllAll_{mediamedia incinc}ll_ududii_{ngng audaud}ii_ovovii_{dedeoo are are representedrepresented inin digidigitata}ll_{for for mamatt}

$$ ::The information conve#edThe information conve#ed ii_{ss remote"remote" ee}ii_{ther ther  pre$produced pre$produced aandnd}

stored or

stored or  produced produced inin realtime"realtime"didistr str  (ut ₍ii uteedd over over nenett%%or or )s)s $

$ .t .t is is possi(le possi(le to to affect affect the the information information received" received" and and sendsend o%no%n information"

information" in in aa non$non$trivtrivii_{alal%a# %a# (e#ond s(e#ond s tart" tart" stop" faststop" fast for%ar for%ar dd}

$$ :: The media areThe media are treatedtreatedin a uniform %a#"in a uniform %a#" presented presented ii_nnanan

orchestrated

orchestrated %a#" (ut are%a#" (ut are  p poossss _(le (le toii  _{to manipulatemanipulate}ii_{ndependentndependent}ll_##

Definition of

Definition of

_Multimedia:

_Multimedia:

Comom puter  puter $$ (ase (asedd techni3uestechni3ues of te't"of te't" ii_{mages"mages"  audio" video"  audio" video" graphics"graphics"anan}ii_{matiomationn""}

and an# other medium %here ever# t#pe of information can (e

and an# other medium %here ever# t#pe of information can (e r r ee presented presented""  processed"

 processed" stored" stored" transmitted" transmitted" producedproduced andand presented digi presenteddigitatall#ll#44

This

This

focus

focus

Audio

Audio and

and &ideo

&ideo

-enefits of Multimedia

-enefits of Multimedia

,ome

,ome authorsauthors claim thatclaim that humanshumans get their information in the follo%ingget their information in the follo%ing %%aa#:#: more

more than than 56 7 (# sight $ of 56 7 (# sight $ of %hich 86 %hich 86 7 is7 isreremem(emem(ereredd 11 7 (# hearing

11 7 (# hearing $ of $ of %hich 96 7 is%hich 96 7 isreremem(emem(ereredd 94 7 (#

94 7 (#smesmellll

14 7 (# touch and 14 7 (# touch and tastetaste44 444 %here

444 %here 6 6 7 of 7 of %hat is %hat is (oth seen (oth seen and heardand heard iiss reremmem(em(eredered 444 further

444 further 56 7 of %56 7 of %hathat ii_{ss  seen" heard and done" is  seen" heard and done" isreremem(emem(ereredd}

That is"

 A Classification of Multimedia

Te't $

A,C.

_+n

_c

_ode"

 _{*TML" ;ostscript" ;DF}

Audio

,ound" music" speech" structured audio e4g4 M.D.!

,till .mage $ Facsim

_{le" photo" scanned image}

ideo Moving .mages!

Movie" a se3uence of pictures

/raphics Computer produced image

Animation

A se3uence of graphics images

Disc r ete Media  DM" ,tatic!: te't" image" gr 

a

 phi

c

Cont i n u o us Media  CM" D#namic!: audio" video" anima

tion

Captured vs ,#nthesi<ed med

_a

,tandalone vs  =et%or)ed media

,#stem .mplications of Multimedia

Multimedia i_{m poses ne% re3u}i_r ements on all parts of the s#stem architectur e

:

>epr esentation

$ digiti<ation and coding compr essing! ,torage

$ data(ase"larger   volumes and ne% access  patter ns ;rocessing

$ O," scheduling" inde'ing" searchi_ng

+nderstanding

$ speecho(?ect r ecogni_{tion" content ana}l_#sis

;roduction

$ more comple' authoring and user interface soft%ar e ;resentation

$ user percepti_{on"  user friendl#} i_{n  *C. *uman Computer .nterface!}

;rotection

media encr#ption" cop#right"  privac# Distr i _(ution

$ media del_{iver# and  (roadcast}

Communication

$ media transmission over net%or)internet" session control

Why is Multimedia Important ?

Forces Driving the Multimedia Revolution

Digi_{tal audiovideo}is revolutioni<ing music" film" game" and video & audio i_ndustr i_es

Convergence of computers" tel_ecommunicati_{on"  radio" and T}

Caused (# technolog# and com petition

Dramatic changes i_{n products and infrastructur e}

 =e% application  potential *uge potential mar )ets

.mproving our l_{ives learning" enterta}inment" and %or)! .nteresting technical issues

M u lti m e d ia h as (ecome hot and ( ee n e mer g e d in C,.T s i nce 1 @ 5

vol_{ution of communication and data net%or)s: .ncreasing}

avai_{la(ilit# of  (and%idth on demand in the off} i_{ce" home"}

road4444 Than)s to high$speed data modems" ca(le modems" h#(rid fi(er$coa' s#stems" 'D,L" %ireless4

+(i3uitous access to net%or)4 &i_{a local$area net%or)s L}A =!" %i_{r eline and %ire}less net%or)s" .nternet" %or l_{d %ide %e("}

Fast  processor and large capacit# storage devices" incl_uding

9$D hard%are4 : co m putation and  memor#

Technological Aspects

techni3ues" graphics" computer vision" speech understanding444

,mart termi_{nals  such as digital phones" screen  phones"}

$T&" personaldigitalassistants" etc4" accessing and i_{nteracting the net%or) %ith %ired and}

%ir eless connections4

And of foremosti_{m portance"  the digiti<ation of virtuall# an#}

device : cameras" videocapture and  pl_{a#(ac)  devices"}

hand%riti_{ng terminals" sound capture" etc4" together %ith}

 plug$and$pla# standardsB and the digiti_{<ation of}

te't1_audi_ovi_{deo documents and li(raries that a}l_lo%s (etter 

communications" storage" and fast access and  (r o%sing4

Techni3uesfor com pressi_{ng and cod}i_{ng the various media: models"}

algor i_{thms" forms" standards" etc4}

Communications aspects: do%nloading and streaming techni3ues" s#nchroni<ation" la#ering of signals" issues i_{nvolved in the definit}i_{on of}

o, 3ualit# of service4!

Techni3ues for accessing multimedia signals  (# providing tools that

conversion tools and multimodal user interface speech recogni_tion" lip

reading" face trac)ing" OC>"44!" agents that moni_{tor  the multimedia}

sessions and provide assistance in all phases of access and utili<ati_on4

Techni3uesfor organi<ing" stor i_{ng and retriev}i_{ng multimedia" for}

searching and (ro%sing i_ndividual_mul_timedi_{a documents and} l_i(raries4

 Are Multimedia Applications Hard?

Large si<e of

multimedia

o(

 ?

ect

s

,peech: 5666 samples1s

8

Kbyte

CD audio: "166

samplessec" 8

 (#

te

s

sample"

stereo

aud

_o

₁₇₆

_Kby

_t

 =T,C video: 96

frames1s"

E6'56 pi'els" 9

 (#tes1pi'el

30

Mbytes/

too (ig" 8$5 M(itss if

compre

ssed!

More storage re3uired

More main memor#

16$96 /-

secondar#

s

torage

*igher data r 

ate

s

Fast .O

su(s#stem ,C,." fi(er

channel" *.;;.!

4g4" +ltra ,C,.8 56 M(#tess

*igh

speed ( ac)plane ;C. or fa

s

ter!

Faster net%or) 1$8M(s per video

stream!

1$ /(itss

net%or) 

*ard%are CODC" modified C;+ 0!" and mod

ified

frame  (uffergraphics su(

s#s

tem

!amples of Multimedia Applications

"eneral #vervie$ of a Multimedia %ystem

Digi_talli _(rar i_es

Distance lear ning ,ci_{ence and technolog#} irtual environment ,ci_entifi_cvi_suali<ation"  protot# ping ntertainment /ames .nteractive T

;ost  production of movi_e

and music

Medicine" We( applications" etc4

Timeline Audio&'ideo diting Interface

Audioideo -roadcast over the

.nt

ernet

Viewers ,treaming Media ,er ver   List en ncoder   n ernet

,hared Applications and C,CW

Des)top

Audiov

isua

l

_Confe

_rencing

Digita

l

 _Li(rar#

Teaching ;lan

;a rt .: Mu

ltimedia

Fundamenta

ls

and

Coding

Techni3ues

'ideo  Audio Te!t Lesson 14 Multimedia Concept and Topics

L_{esson 84 Audio Fundamentals} L_{esson 94 Audio Coding and ,tandard} Lesson 4 .mage&ideo Fundamentals

Lesson 4 .mage&ideo Coding: ;/ and *48E'

Lesson E4 M;/ Coding ,tandar ds

Inde!edData(ase

Inde!edData(ase

L_{esson  G4 >evie% of Advanced} MM_Coding

ui< Test .4 uestions relate d to ;art .

>eport .4 ,ummar# of Audio and ideo Coding" or 

Libr

  Creati o

r a

Explorat i

xtracio Natural Languag n rp tation

or Spoke Natural Langua Query emantic xp ns Reueste! Segment io _{Choice me} "igital Compression

Inde!ed

Transcript _Compressed%egmented

 Audio&'id

Teaching

;la

n

;a rt ..: Mu

ltime

dia Techn

ologies

and

Applications

L_{esson 54 Media O(?ect ;r oduction}

L_{esson  @4 Media .ntegration and ;resentation} L_{esson 164 Med}i_{a ;rotection}

Lesson  114 Media >etr i_eval

Lesson  184 Media Distri(uti_on Across .nternet

Lesson 194 Media Communication $ .; Tele phon# & Teleconference

Lesson 14 Mo(ile Multimedia ,ervice over Wireless  =et%or)s >eport ..4 ,ummar# of Multimedia Technol_{ogies" or} 

A_{,tud# on a ,pecific Mu}l_timedi_{a Technolog#}