Copyright 2006 TeleMídia

(1)

(2)

Agenda

• ISDB-T Reference Model and ITU-T

Reference Model overview

• Middleware requirements

• ISDB-T Reference Model and ITU-T

Reference Model overview

• Middleware requirements

• Ginga

• Final Remarks

• Ginga

(3)

DTV

Reference Model

DTV

Reference Model

(4)

Other Data

MUX

Source Signal

Coding

DataStream

Audio Video

DEMUX

Source Signal

Decoding

Middleware

Interactive

Applications

Audio Video

Interactive Channel

Channel coding / Modulation

MUX

TS

Channel decoding /

Demodulation

DEMUX

TS

(5)

Other Data

MUX

Source Signal

Coding

DataStream

Application

Specification

Audio Video

DEMUX

Source Signal

Decoding

Middleware

Interactive

Applications

Audio Video

Interactive Channel

Application’s

5 Broadcasting

Transmission

Channel coding / Modulation

MUX

TS

Receiver

Reception

Channel decoding /

Demodulation

DEMUX

TS

(6)

Transport System

Video Coding

Audio Coding

Reference Model

Transport System

Physical Layer

(7)

MPEG2 - SDTV

MPEG2 - HDTV

MPEG2 BC

MPEG2 AAC

DOLBY AC3

Reference Model

Video

Audio

8-VSB

COFDM

MPEG-2 System

(8)

MPEG - 4 HE-AAC@L4

Vídeo

Áudio

_{MPEG - 4 HE-AAC@L3}

Reference Model

8-VSB

COFDM

(9)

Transport System

Video Coding

Audio Coding

Reference Model

Transport System

Physical Layer

(10)

MPEG2 - SDTV

MPEG2 - HDTV

MPEG2 BC

MPEG2 AAC

DOLBY AC3

Reference Model

Video

Audio

8-VSB

COFDM

MPEG-2 System

(11)

MPEG - 4 HE-AAC@L4

Vídeo

Áudio

_{MPEG - 4 HE-AAC@L3}

Reference Model

8-VSB

COFDM

(12)

Audio

MPEG

Transport Stream

GINGA

Real-Time Operating System Device Drivers Hardware Conditional Access

specification language

Video Data

MPEG

SBTVD

specification interpreter

(13)

MPEG-2 System

Program_number

Component_tag

(Program map table)

1…n

PAT

(Program association table) PCR- Program

Clock Reference

program stream

(14)

Transport System

Video Coding

Audio Coding

Reference Model

Transport System

Physical Layer

(15)

MPEG2 - SDTV

MPEG2 - HDTV

MPEG2 BC

MPEG2 AAC

DOLBY AC3

Reference Model

Video

Audio

8-VSB

COFDM

MPEG-2 System

(16)

MPEG - 4 HE-AAC@L4

Vídeo

Áudio

_{MPEG - 4 HE-AAC@L3}

Reference Model

BST-OFDM

(17)

ISDB

CODEC

S-Video

Modulation

Editing

MPEG-2

(18)

ISDB

Net

CODEC

Internet

S-Video

Modulation

MPEG-2

(19)

Rede Externa API SO 19 Copyright © 2006 TeleMídia Demodulador Decod. do Canal Dec. Áudio Dec. Vídeo CPU Memória VC RF IR CR Video out Audio Surround Audio out RF out Rede Externa RF in API SO

Transport System

Physical Layer

(20)

Rede Externa API SO

APPL1

APPL2

...

APPLn

Demodulador Decod. do Canal Dec. Áudio Dec. Vídeo CPU Memória VC RF IR CR Video out Audio Surround Audio out RF out Rede Externa RF in API SO

Transport System

Physical Layer

(21)

Middleware

APPL1

APPL2

...

APPLn

21 Copyright © 2006 TeleMídia Demodulador Decod. do Canal Dec. Áudio Dec. Vídeo CPU Memória VC RF IR CR Video out Audio Surround Audio out RF out Rede Externa RF in API SO

Transport System

Physical Layer

(22)

APPL1

APPL2

...

APPLn

Ginga

Transport System

Physical Layer

(23)

Runtime Environments

Common Core

APPL1

APPL2

...

APPLn

Ginga

Transport System

Physical Layer

(24)

APPL1

APPL2

...

APPLn

Ginga

Ginga-NCL

Ginga-J

Ginga-CC

Transport System

Physical Layer

(25)

Ginga-CC

APPL1

APPL2

...

APPLn

Ginga

Ginga-NCL

Ginga-J

Transport System

Physical Layer

(26)

Ginga-NCL

Ginga-CC

APPL1

APPL2

...

APPLn

Ginga

Transport System

Physical Layer

(27)

Middleware Requirements

(28)

Rural Area Urbain Area TOTAL Have never used the Internet Has never used a Computer Computer + Internet Mob. + Inter net TV (cable) TV Tel.

Fixed Mobile Computer

82%

75%

4%

9%

57%

49%

20%

23%

61%

53%

18%

21%

91%

98%

97%

1%

7%

6%

15%

40%

36%

72%

76%

72%

8%

28%

25%

(29)

Rural Area Urbain Area TOTAL Have never used the Internet Has never used a Computer Computer + Internet Mob. + Inter net TV (cable) TV Tel.

Fixed Mobile Computer

82%

75%

4%

9%

57%

49%

20%

23%

61%

53%

18%

21%

91%

98%

97%

1%

7%

6%

15%

40%

36%

72%

76%

72%

8%

28%

25%

10%

7%

91%

58%

Class A > R$ 4.151,00

53%

90%

97%

95%

55%

47%

16%

23%

25%

20%

58%

41%

99%

100%

5%

19%

40%

75%

81%

94%

25%

70%

Class DE < R$ 1.245,00

92%

1%

13%

51%

8%

3%

1%

77%

84%

100%

(30)

Specification Language

• Simple to be understood and learned

• Lightweight

• Powerful

• Simple to be understood and learned

• Lightweight

• Powerful

• Declarative DSL language

(31)

Programming Paradigms

• Imperative (procedural)

– algorithm specification: “how to do”

– more expressiveness

• Imperative (procedural)

– algorithm specification: “how to do”

– more expressiveness

• Declarative

– specification: “final intention”

– highest level specification

• Declarative

– specification: “final intention”

– highest level specification

(32)

NCL

Nested Context Language

NCL

Nested Context Language

• The Brazilian innovation in the ISDB System:

– NCL (Nested Context Language) declarative language

– Its script NCLua language

– Its engine: Ginga-NCL middleware.

• The Brazilian innovation in the ISDB System:

– NCL (Nested Context Language) declarative language

– Its script NCLua language

– Its engine: Ginga-NCL middleware.

• ITU-T H.761 Recommendation for IPTV services

• ITU-R BT 1691-1 Recommendation for Terrestrial DTV

– Its engine: Ginga-NCL middleware.

• ITU-T H.761 Recommendation for IPTV services

(33)

Synchronization

(34)

Synchronization

(35)

Interactivity

(36)

Interactivity

(37)

TV

is not

a Computer

TV

is not

a Computer

• Broadcast transmission

• Viewers are usually far from the screen

and interact via remote control devices

• Usually more than one viewer

• Broadcast transmission

• Viewers are usually far from the screen

and interact via remote control devices

• Usually more than one viewer

Viewer interaction must be

treated as just an example of

(38)

TV

is not

a Computer

TV

is not

a Computer

• Broadcast transmission

• Viewers are usually far from the screen

and interact via remote control devices

• Usually more than one viewer

• Broadcast transmission

• Viewers are usually far from the screen

and interact via remote control devices

• Usually more than one viewer

• Video based applications

• Usually more than one viewer

• Video based applications

(39)

(40)

(41)

(42)

(43)

(44)

(45)

(46)

(47)

(48)

(49)

(50)

(51)

(52)

(53)

Middleware

Requirements

(54)

TV

is not

a Computer

TV

is not

a Computer

• Broadcast transmission

• Viewers are usually far from the screen

and interact via remote control devices

• Usually more than one viewer

• Broadcast transmission

• Viewers are usually far from the screen

and interact via remote control devices

• Usually more than one viewer

• Video based applications

• Usually more than one viewer

(55)

Single Exhibition Device

(56)

Single Exhibition Device

(57)

Single Exhibition Device

(58)

Multiple Exhibition Devices

(59)

Multiple Exhibition Devices

(60)

(61)

(62)

(63)

(64)

(65)

(66)

Adaptation

(67)

Adaptation

(68)

Adaptation

(69)

Adaptation

Beba Guaraná

(70)

Content and Presentation Adaptation

• Presentation device

• User profile

• Presentation device

• User profile

• User location

(71)

Programming Paradigms

• Imperative (procedural)

– algorithm specification: “how to do”

– more expressiviness

• Declarative

• Imperative (procedural)

– algorithm specification: “how to do”

– more expressiviness

• Declarative

– specification: “final intention”

– highest level specification

• Declarative

– specification: “final intention”

– highest level specification

Media synchronization

Adaptability

(72)

Declarative:

Imperative:

Lua

ECMAScript

Declarative:

NCL

HTML

MHEG

ECMAScript

Java

(73)

Imperative

Declarative

Imperative

+ efficient

+ easy to use

+ general

(74)

Declarative X Imperative

(75)

Declarative X Imperative

Declarative

Imperative

(76)

Declarative X Imperative

(77)

State of the art – Declarative

Middleware

State of the art – Declarative

Middleware

• Focus on interactivity

– Synchronization and adaptability by using

scripts (procedural)

• Focus on interactivity

– Synchronization and adaptability by using

scripts (procedural)

(78)

DTV Middlewares

• J.200: Worldwide common

core –

Application

environment

for digital

interactive television services

• J.201: Harmonization of

declarative content format

• J.200: Worldwide common

core –

Application

environment

for digital

interactive television services

• J.201: Harmonization of

declarative content format

MHP 1.0 DVB - HTML

EE

PE

MHP 1.1 ACAP-J ACAP-X ARIB-AE (STD-B23) BML (STD-B24) Europe (DVB) USA (ATSC) Japan (ARIB)

declarative content format

for interactive TV

applications

• J.202: Harmonization of

declarative content format

for interactive TV

applications

• J.202: Harmonization of

(STD-B23) (STD-B24) Ginga-J Ginga-NCL Brasil (SBTVD)

(79)

Declarative X Imperative

Declarative

Imperative

(80)

Why NCL?

(81)

NCL – Nested Context Language

• Synchronization support

– Structure-based synchronization

– Interactive channel support

• Multiple device facilities

• Synchronization support

– Structure-based synchronization

– Interactive channel support

• Multiple device facilities

• Support for content and presentation adaptation

• Live editing support

• NCL is free software

• Support for content and presentation adaptation

• Live editing support

(82)

Declarative X Imperative

Declarative

Imperative

(83)

Declarative X Imperative

Declarative

Imperative

Scripting languages

ECMAScript

Lua

System languages

Java

C

(84)

Script

System

Footprint?

Efficiency?

Script

+ easy to use

(85)

Script

System

Script X System

(86)

Script

System

Script X System

(87)

Script

System

Script X System

(88)

Lua

Java

Ginga Options

Small to medium complex tasks: Lua

High complex tasks: Java

(89)

Why Lua?

(90)

Why Lua?

• Lua is Simple and Powerful

• Lua is Portable

• Lua is Embeddable

• Lua is Simple and Powerful

• Lua is Portable

• Lua is Embeddable

• Lua is Fast

• Lua is Robust

• Lua is Fast

• Lua is Robust

(91)

http://shootout.alioth.debian.org

/

http://shootout.alioth.debian.org

/

• Lua vs JS

• LuaJIT vs JS

JavaScript SpiderMonkey = 936 Kbytes

Lua = 120 Kbytes

(92)

onBegin

Start

Stop

onBegin

Start

onEnd

Stop

java

NCL

Set

size

Start

onSelection

Set

size

onBegin

Start

Stop

onEnd

Start

(93)

onBegin

Start

Stop

onBegin

Start

onEnd

Stop

java

NCL

Set

size

Start

onSelection

Set

size

onBegin

Start

Stop

onEnd

Start

Smil

(94)

javajava

onBegin

Start

Stop

onBegin

Start

onEnd

Stop

Set

size

onSelection

Set

size

onBegin

Start

Stop

Start

onEnd

Smil

(95)

(96)

Final Remarks

(97)

Composer 1.0

(98)

NCL Eclipse

(99)

R&D

Ginga at the Content Producer

R&D

Ginga at the Content Producer

• Composer 2.0

– Composer 1.0 + NCL Eclipse facilities

– Keeps the functional requirements of Composer

1.0 and adds non-functional requirements

• Composer 2.0

– Composer 1.0 + NCL Eclipse facilities

– Keeps the functional requirements of Composer

1.0 and adds non-functional requirements

– Integrated with the transmission system

– Optimized data carousel generation

– Support to live content production

1.0 and adds non-functional requirements

– Integrated with the transmission system

– Optimized data carousel generation

(100)

R&D

Ginga at the Content Producer

R&D

Ginga at the Content Producer

• Composer 2.0

– Based on a micro-kernel that may be extended

with plug-ins

– Each authoring views acts as a plug-in

• Composer 2.0

– Based on a micro-kernel that may be extended

with plug-ins

– Each authoring views acts as a plug-in

– Open-source development, from the first step

thought to be extended

– Open-source development, from the first step

thought to be extended

(101)

CPU _Memória IR CR Rede Externa API SO

Rede

Internet

Audio SBTVD

PUSH

PULL

101 Copyright © 2006 TeleMídia Demodulador Decod. do Canal Dem ux. Dec. Áudio Dec. Vídeo Memória VC RF Video out Audio Surround Audio out RF out RF in Audio Video Data

MPEG TS

(102)

R&D

Ginga Authoring in the Client Side

R&D

Ginga Authoring in the Client Side

• Composer 2.0

– Composer 1 + NCL Eclipse facilities

– Context aware

• Composer 2.0

– Composer 1 + NCL Eclipse facilities

– Context aware

– Visions for cooperative authoring

(103)

Ginga-NCL Reference Implementation

• C++ Language

• Linux platform

• High performance

• Hard to embed

• C++ Language

• Linux platform

• High performance

• Hard to embed

(104)

Ginga-NCL Virtual Set-top Box

(105)

Ginga -NCL Live CD

(106)

(107)

SAGGA Project

• SAGGA: Suport for Automatic Generation

of Ginga-NCL Applications

• Definition of several templates for

application authoring

• SAGGA: Suport for Automatic Generation

of Ginga-NCL Applications

• Definition of several templates for

application authoring

107

application authoring

– Applications with dynamic content

– Widgets

– NCLua lib

application authoring

– Applications with dynamic content

– Widgets

– NCLua lib

(108)

• IPTV: Recommendation H.761

– Symbian

– Android

• IPTV: Recommendation H.761

– Symbian

– Android

Ginga –NCL for IPTV

(109)

• 1-seg/ Full-seg USB-SBTVD: PlayTv PixelView,

Intera

– Linux

– Windows

• 1-seg/ Full-seg USB-SBTVD: PlayTv PixelView,

Intera

– Linux

– Windows

Ginga –NCL for USB ISDB

– Windows

(110)

• 1-seg/ Full-seg USB-SBTVD

• IPTV

• 1-seg/ Full-seg USB-SBTVD

• IPTV

Ginga –NCL for Windows

• Broadband TV: plug-in Firefox

(111)

Multiple devices

– iPhone (passive)

– Android (passive e active)

– iPhone (passive)

– Android (passive e active)

(112)

NCL Evolution

(113)

NCL 3.1 Raw Profile

• A new profile closer to the Ginga-NCL internal data

structure

• Completely compatible with NCL 3.1 EDTV profile

, but

without any “syntactic sugar”

• Player much more simple, efficient and less error-prone

• A new profile closer to the Ginga-NCL internal data

structure

• Completely compatible with NCL 3.1 EDTV profile

, but

without any “syntactic sugar”

• Player much more simple, efficient and less error-prone

• Player more simple, converter more fancy

• Application much more difficult to be understood and

cloned

• Player much more simple, efficient and less error-prone

• Player more simple, converter more fancy

• Application much more difficult to be understood and

cloned

(114)

NCL Evolution

• TAL 1.0: Template authoring language

• NCL 4.0

– Better context aware support

– 3D objects

• TAL 1.0: Template authoring language

• NCL 4.0

– Better context aware support

– 3D objects

– Multiple devices

– Social networks

– 3D objects

– Multiple devices

– Social networks

(115)

NCL Evolution

• TAL 1.0: Template authoring language

• NCL 4.0

– Better context aware support

– 3D objects

• TAL 1.0: Template authoring language

• NCL 4.0

– Better context aware support

– 3D objects

– Multiple devices

– Social networks

– 3D objects

– Multiple devices

– Social networks

(116)

http://www.ncl.org.br

Copyright 2006 TeleMídia

Agenda

Agenda

• ISDB-T Reference Model and ITU-T

Reference Model overview

• Middleware requirements

• ISDB-T Reference Model and ITU-T

Reference Model overview

• Middleware requirements

• Ginga

• Final Remarks

• Ginga

DTV

Reference Model

DTV

Reference Model

Reference Model

Reference Model

Other Data

Other Data

MUX

Source Signal

Coding

DataStream

DEMUX

Source Signal

Decoding

Middleware

Interactive

Applications

Interactive Channel

Channel coding / Modulation

MUX

TS

Channel decoding /

Demodulation

DEMUX

TS

Other Data

Other Data

MUX

Source Signal

Coding

DataStream

Application

Specification

DEMUX

Source Signal

Decoding

Middleware

Interactive

Applications

Interactive Channel

Application’s

Contents

Application’s

Contents

5

Broadcasting

Transmission

Channel coding / Modulation

MUX

TS

Receiver

Reception

Channel decoding /

Demodulation

DEMUX

TS

Transport System

Video Coding

Audio Coding

Reference Model

Transport System

Physical Layer

MPEG2 - SDTV

MPEG2 - HDTV

MPEG2 BC

MPEG2 AAC

DOLBY AC3

_{MPEG - 4 HE-AAC@L3}

_{MPEG - 4 HE-AAC@L3}