Tvpical ly the order in \\'hich coded pictu res arc pre­ sented to the decoder docs not correspond to the

order in which they are to bt d isplayed. Consider the

t()llowing example:

Dispbv Order I

l

Decoder

[ nput

I 1

B2 R3

_P4

BS B6 _P7 B 8

P 4

B 2 B 3 _{P 7} B 5 B 6

IlO

The order mismatch is an ::�rtibct o f the com

p

r

e

ssion algorithm-::\ B

_-p

_i

_c

_t

ur_e

_{C:Jilnot be decoded until both} its past

and

finure reference ti·amcs h:J\'C been de.coded ·

Similarly a

P-picture

cannot be

d

eco

d

e

d

until J ts past rdcrcncc ti·amc has been ckcodcd . To get around th1s problem, S U B

defi nes

an output multibufkr. The si ze of this mu ltibufrer is approxim:nclv equal

to

three times the size of

a

single Ll llco mpn.:sscd frame. For example,

t(>r

a 4:2:0 subsa mplcd

C!F

image, the si

z

e of

t

he

multihufti:r

\\'Ould be 352 lw 2 8 8

_by

_{1 .5}

_{b\' 3}

_bvtcs ( the exact size is retu rned by the libr:�rv du ring initi:�l c()(kc setu p ) . After steady st:ltc hJs been rea

c

h

e

d,

each

invocation

to

the decompress c ::� l l yields the correct next ri·a mc to

be

displayed as shown in

F

ig

u

re ll . To avoid expensive copy opcr<nions, the

m u l ti b u ffer is

a

l lo

c

ated :�nd owned by the software above S U B .

ITU-T's Recommendation

H . 261 (a.k.a. p

X

64)

At

the li br:�n· level , decompressing an H . 2 6 1

_{stre:un is ver�·}

simiL

;

r

to

M P EG- 1 _de

_c

_o

_d

_{ing \\'ith one}

_ex_cep

_tion: instead of three types of pictu res, the H . 2 6 1 _{rccom ­} membtion defines onl�· t\\'O , key ti·a mcs a n d no

n

-key h·anlCs ( no

bid irectional

prcdinion ) . The implication

t(>r

implementation is that the size of the

multibuffer

is approximately twice the size of ::1 singlc decompressed

h·;unc . Furthermore,

the order in which compressed

h·amcs :11-c p

re

sented to the de

co

mp

res

s

or is

the same

as the order in \\'hich they arc to be displaved. To satist\• the H . 2 6 1 _{1-ccomnH:nd::�tion, S U B im}

_p

_le­

ments a

;

trc::�ming

i ntertace

f(>r com

pr

es

si

o

n

and

decompression. In this

m

o

d

e l , the

app

l i

c

ation teeds input bu fti:rs to the codec, \\'hich processes the data 111 the bu fti:rs and re turns the processed datJ to

the app li­

ution through a call back rou tine. During dcco

�

l1-

pression, the

application

l ayer passes input bufters

cont::�ining sections of ::�n H . 2 6 1 _{bit stream . Th}

�

_b1t

stream un be divided arbitrarily, or, in the

case

of live tcleconkrencing, each bufkr can contai n data from a transmission packet. Empty output bu ffe rs are also passed to the

codec

to fil l \\'ith reconstructed imag_es. PictuiT ti·;uncs

do not

have

to

be a l igned on

buffer

boundaries. The codec parses the

bit

stream and, "'hen enough data is a\·ai lablc, reconstructs ::�n image. Input b

ufkr

s arc

ti

·e

e

d

by

cal

l

ing the c11lback rou tine . When an image is

reconstructed,

it is pl aced Ill an

out­

put butter and the

buffer

is re

tur

ned

to the applica­

tion th

mu

gh the

cal lback

routi ne. The comprcss1on process is

s

imil

ar

,

but

input bufkrs cont::�in images and output bufkrs contai n bit-stream

data.

One advantage to this

stre

aming

interface

is that the ::�pplication layer docs not need to know the svnrax of the H . 2 6 _I

_b

_it

stream. The cod

e

c is

r

esponsible

t(>r

:1!1 bit-stream parsing. Another advantage is th:Jt the c::�ll b<lck mecha­ nism f(>r retu rning c

o

mplete

d

images or bit-stream bu

f

krs a l l o\\'S th

e

app

l

i

ca

tion

to

do other tasks \\'ith­ out implementing multithreading.

S U B's

architecture

and A P I _{can easi ly accommo­} date !SO's

M P EG-2

and ITU -T's H . 2 6 3 _video

_com­

pression

a

lgo

r

ithms because of their s

i

mi

br

i

t

v to tbe

M PEG- 1

:1nd H . 2 6 1 _algor

_i

_th m_s.

Implementation of Video Rendering

O u r

s

oft

wa

r

e implementation of ,·ideo

renderi ng

cssc ntiallv

parallels

t he hard\\·are realization detailed c lsc\\'hcn: in this

issue -" As \\'ith

the hard\\':Jre Imple­ mentation, the solt\\'are renderer is bst and si mple because the co

m

plica

t

e

d

compu tations

arc p

e

r

fo

r

med off line in building

the

various look- up t:1blcs. I n both hardware and so

ft

war

e cases, a

shortcu t is ach ieved by dithering in YUV s

p

a

c

e and then

c

o

n

verti

n

g to some snull number of RGB i ndex values in :1 look-u p table. "' Although

in

most

cases

the m�1pping ,·a lu cs

i n

the

look-up t::�blcs remain

tixed for the duration of

the

r u n , the video li brary provides rou ti nes to dvnamicallv adjust im:1gc brightness, contrast, satu ration,

am�_

the nu mber of

c

olo

r

s. Image scaling is possible

but

affects pert(mnance .

When qual itv is

i mport�lllt,

the

so

ft

w

a

re perf(mns scaling before dit

h

ering :1 1Hi when speed 1s the prim::�rv concern, it is done a

ncr

d i theri ng.

Optimizations

We app

r

oach

ed

the p

r

oblem of optimization fi·om two

directions: Plattorm -indcpendcnt opti mi zations, or

algorithmic

enhancements,

\\'ere done

b,·

e

x

p l

oi

t

i n

g

kno\\'ledgc of the com

pr

essio

n

algo

r

i thm and the

TIME Figure

1 1

M u lributkring i n S U B

66

i n

pu

t data stream.

Plattorm-dcpcndcnr

optim izations were done b

y

examining the sen·iccs <Wailablc ti·on1 the und

e

r

l

yin

g

operating system and by c\·a l u <lting

the

Jttri bu res of the system's

pr

o

ce

ssor.

As can be seen t

i

·

o

m

Ta ble 2 ,

th

e

DCT' is

o

ne of the

most comput<ltional lv

intcnsi,·c components in t

h

e

compression

pipeli

n

e.

It is a lso com mon to Jll five

i

nt

ern

ationa

l stand ards. Therd(xe, a

sp

e

cial

e ftort

was made in c hoosing and

optimizing the

DCT. Since

all

ri

ve

stand ards

c1l l tc>r t

he

i nverse

DC

T ( I DCT) to be

postproccs

sc

d

with inverse q uantization, sign ificant

alg

o

ri

th

mi

c s

a

v

in

g

s were

o

b

ta

ined by

compu

ting

a

scalar mu

l

tiple of

the

DCT and

m

ergin

g

the appropri­

ate scJiing into the q uan

t

i

z

e

r

. The

DCT implemented

in the

l

ibra

ry

is <1 moditied version of

the

one­ d i mensional su

led DCT prop

os

e

d

by Ar<�ri cr a l . " The

two-d i mcnsion<ll DCT

is obtained

by

p

ertorming

a

one-dimensional ncr on

the columns

t

(

)ll owed by a OIH:-dimcnsional ncr on

the rows.

A totJI

of

80

mu ltiplies and

464 adds arc

nee

de

d

for

a

fu l l y popu ­

In document dtj v07 04 1995 pdf (Page 66-68)