s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Universität Klagenfurt
Institut für Informationstechnologie
Vorlesung 623.715
"Multimedia Kommunikation"
Kapitel 1: Multimedia Trends und Anwendungen
Überblick
Dr. Eckart Hundt Siemens AG CT IC 2 München Tel.: ++49 / 89 636 53388 eckart.hundt@mchp.siemens.des
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Multimedia Communication
Living Spaces of the Future are
Dominated by Multimedia Communication
Home Leisure Travelling Work Driving Forces: • fast broadband
networks mobile and fixed
• powerful processors & highly integrated microelectronics • increasing part of system functionality in software • increasing use of multimedia • standards and algorithms for
transmission and signal processing
Driving Forces:
• fast broadband
networks mobile and fixed
• powerful processors & highly integrated microelectronics • increasing part of system functionality in software • increasing use of multimedia • standards and algorithms for
transmission and signal processing
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Definition: Multimedia
• means get together of text, graphics, speech, audio, image, video,... • Multimedia systems can in principle address every human sense and
perception. In the moment it is dominated by acoustic and visual media.
Systematics: Multimedia Technologies are subdivided in 5 categories and sorted along the Content Value Chain
• Metadata & Rights & Security • Interactivity & content-based Data Manipulation • Data Access & Search • Data Bases • Storage Media Content Creation • Imaging & Recording and Data Fusion • Data Synthesis &
Graphics Generation • Data Modeling • Data Formats • Authoring & Composition Content Processing & Analysis • Source Coding &
Compression • Processing & Enhancements • A/V, Image Analysis • HW & SW Architectures Content Presentation • Devices & Screens • Adaptation & Printing • Rendering • Synchronization
• Synthetic & Natural Worlds Combination • 3D Visualization • User Interfaces Content Trans-mission • Channel Coding • Distribution & Communication Mechanisms & Protocols & Transcoding • Middleware / Platforms Content Archiving & Retrieval Content Presentation
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
ä Cheap cameras as well as easy re-use & download of content allow everybody to create, process and distribute content.
ä 3D imaging and modeling provide improved lifelike visualization and scene interpretation. Virtual and augmented reality gain increasing importance.
ä Dramatically increasing processing power pushes wide spread use of
multimedia and allows more elaborate analysis and presentation algorithms.
ä Despite higher bandwidth in mobile and access networks, transmission techniques need higher compression and improved QoS techniques.
ä Multimedia is more and more used for providing intuitive and appealing user interfaces and portals.
ä Metadata allow interactive and content based data access, including DRM and security issues.
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Computer
Industry
Transportation
Information &
Communication
Manufacturing
& Production
Consumer
Industry
Entertain-ment Appl.Services
Communi-cation Information ServicesEnergy
OfficeAutomation Intelligent.Home Industrial Production Robotics and Agents Business Services Telemetry/ Telematic Service & Mainten-ance
Health Care
HealthCare / Medical ImagingMilitary
Military Automotive Learning & TeachingMultimedia Application Segments cover
the Industry-Landscape
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
ä
Mobile use of multimedia for entertainment, games, business
ä
Increasingly interactive use of multimedia in entertainment (TV,..),
learning & teaching and intelligent home
ä
Cheaper devices enforce surveillance, telemetry, entertainment,
business, communications, automotive, etc
ä
Communications go more multimedial and with improved quality
ä
Reduced cost of imaging makes robots and industrial us more economic
ä
Multimedia gets an important issue in automotive for navigation, safety &
driver assistance.
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Future Scenarios have high demand on Multimedia
PoF Information and Communications
änew networks and devices will change landscape
äconverged and integrated applications and solutions
äExcitement and fun for Entertainment and gaming
PoF Services
äJust in Time and Just in Place servicing on site and remote
äProactive services and intelligent sensors
äEfficiency and cost reduction for Business- Enterprise and Industrial applications and Energy issues
PoF Transportation
äDigital car technologies to come
äIntelligent solutions for improved mobility
PoF Health
änew ways of medical imaging provide new ways of surgery
äEnhanced health services such as telemedicine and home health solutions
PoF Energy
äbetter ways of monitoring and diagnosis
äsimulations for a wide range of processes
PoF Automation and Control
äVirtual development, test and training
äautomated and networked buildings and processes
Future-Scenario was compiled in PoF(Picture of the Future) for 6 Living Spaces. Core business and trends were identified and described. The main trends and are:
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Future Scenario
"Information and Communications“ (1/2)
Key Segment 1: Networks • Convergence will take place on
application level (e.g. TV-PC) and starts on network level with seamless
interoperability
• Thus, media conversion mechanism and gateway technologies become essential • Content understanding and adaptation
across all network levels is required
Key Segment 2: Commercial Solution • The vision of integrated enterprise business
solutions with high level of automation drives development
• This is supported by sophisticated application development with open standards (e.g. ERM) • Collaboration solutions enforce efficiency • Investments will be driven by cost cutting
ambitions
PoF Information and Communications is structured in 4 Key segments and each segment analyzes technologies, high growth areas, business impact and Siemens impact. The key findings are shown in turn:
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Future Scenario
"Information and Communications“ (2/2)
Key Segment 3:
Consumer Content and Applications • Applications migrate over all digital
networks
• Technology improvements are pushed by Electronic entertainment, especially
mobile gaming
• Content facilitation (distribution and rights management) are issues for realization
Key Segment 4:
Information and Communication Devices • Intelligent user interfaces are context aware,
act intuitively and react to emotions • The always ON scenario is enabled by
embedded devices and smart labels - allows users to compute and communicate
everywhere and all the time
• Home networks and entertainment platforms will prevail
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Available Network Technology
DECT 32kbit/s/ channel Cordless telephony Available GSM 9,6kbit/s (22,8) Mobile telephony Operational
GPRS <115kbit/s Mobile data Introduction
Mobi le Net w orks
UMTS <2Mbit/s Mobile multimedia Start in 2002
IEEE 802.11B 11Mbit/s Wireless data Available
Bluetooth 768kbit/s Wireless multimedia Introduction
Wi reless Net w orks HomeRF a) 2Mbit/s
b) 10Mbit/s Voice and Data Introduction ISDN Px64kbit/s Integrated services
(Voice,data,conference) Operational
XDSL <20Mbit/s
(ADSL:768kbit/s) Internet Introduction
Ethernet 100Mbit/s Internet Available
Wired Net w orks
Fiber >1Git/s Various Available
A broad range of networks and protocols exists BUT which is the best choice?
A broad range of networks and protocols exists BUT which is the best choice?
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Mobile access will dominate
0 200 400 600 800 1000 1200 1400 1600 1800 1995 2000 2005 2010
Subscriptions worldwide (millions)
Mobile internet
subscribers
Mobile
subscribers
Mobile Fixed Mobile Internet Fixed Internets
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
UMTS Status 2004
UMTS is happening now:
• UMTS customers already over 3 million worldwide.. First networks deploywd in Japan and Europe, soon to be joined by South Korea, ,Hong Kong and others
• Licensing largely complete in Europe. >98% of 3G licenses awarded so far (approaching 120 operators) specify WCDMA
• 3G licensing continues in other world regions. E.g. China anticipated 2004
• European operator deployment plans characterized by soft launches, followed by commercial introductions during 2004/2005
• Currently seven EU Member States are reviewing license conditions in a process of flexible dialogue with operators
UMTS is happening now:
• UMTS customers already over 3 million worldwide.. First networks deploywd in Japan and Europe, soon to be joined by South Korea, ,Hong Kong and others
• Licensing largely complete in Europe. >98% of 3G licenses awarded so far (approaching 120 operators) specify WCDMA
• 3G licensing continues in other world regions. E.g. China anticipated 2004
• European operator deployment plans characterized by soft launches, followed by commercial introductions during 2004/2005
• Currently seven EU Member States are reviewing license conditions in a process of flexible dialogue with operators
UMTS trials and launches are proceding:
• Austria 4 operators commercial launch in April/ Ma y2003 • Italy 3 operators commercial launch in 2003
• UK 3 operators commercial launch in 2003 UMTS trials and launches are proceding:
• Austria 4 operators commercial launch in April/ Ma y2003 • Italy 3 operators commercial launch in 2003
• UK 3 operators commercial launch in 2003
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004 Access Networks: •Satellite •Terrestrial •Cable •DSL Inhouse Networks:
Wired and Wireless (Ethernet, HyperLAN, Home RF
The big five applications:
• Games
• TV/Video/Video on Demand • Music
• Internet
• e-commerce (t-commerce)
Today's Video/TV will change to two way information /
entertainment channel
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Scenario Homenetwork
Telco Cable xDSL Broadcast Broadcast Netwoks Netwoks Internet Internet Cellular Cellular Networks Networks TV-Set /STB DVD-Player PSTN/ PSTN/ ISDN ISDN Wireless Home Network Wired Home Network:Ethernet,Hyper Lan, Home RF;... TV-Set PC Multimedia PC Control Heating Surveillance Camera Hub
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Evolution of Multimedia Terminals
PhonePlus PC
TV/ STB STBs will evolve to an open platform system (Java, HTML, ..)with a return channel for Internet access and interactive TV.
mobile Terminal
Mobile terminals will support multi modes (UMTS, GPRS, GSM), as well as graphics/ video, Internet applications.
Phones will be extended to screen/ video phones. IP phones will dominate.
PCs will be equipped with multimedia/ digital TV boards,
broadband interfaces xDSL/ cable modem, wireless interfaces.
n All terminals will be enhanced by multimedia functions and infotainment services n Wireless interfaces ( WLAN, UMTS, ...) will become dominant
Different terminal types will coexist.
Terminals will get more intelligence and multi-functionality. With IP they will enable network independent communication.
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Interactive TV: SAMBITS
Broadcast
Service
Broadcast
Service
Internet
Service
Internet
Service
Broadcast Channel
DVB-S, DVB-C, DVB-TBroadcast Channel
DVB-S, DVB-C, DVB-TInternet Channel
PSTN/ISDN, Cable (Coax & Fiber)Satellite
Internet Channel
PSTN/ISDN, Cable (Coax & Fiber)
Satellite
SAMBITS
Integrated
Receiver
Decoder
SAMBITS
Integrated
Receiver
Decoder
Bro ad cast Netw o rk Ada p te r In ternet Netw o rk Ada p te r In te rne t Inte rf ac e M o du le Bro ad cast I n te rf ace M o du le Forward Interaction path Return Interaction path SAMBITS User Terminal (STB)s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
SAMBITS: Systems for
Advanced Multimedia Broadcast and IT Services
SAMBITS
SAMBITS
Objectives
Objectives
and Features
and Features
ä
MPEG-4 and MPEG-7 for broadcast technology together with
related Internet services
ä
provide multimedia services to a terminal that can display any
type of general interest integrated broadcast/internet services
with local and remote interactivity
(combination of DVB and Internet infrastructure)
ä
Improving quality of Web and broadcast experience
ä
Services consist of high quality video enhanced by multimedia
elements and interactive personalised information retrieval
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Tools for advanced presentation / visualization systems (based on MPEG-4 Standard):
• 3D virtual worlds, navigation and visualization
• Avatar as talking head (Face Animation)
• Interaction with objects
Example development of a 3D Interactive e-Shop together with SBS eBT eBA for use in a commercial environment
• Integration into e-business environment of SBS:
• Integration into commercial e-shop (IBM Websphere Commerce Suite)
• Integration into CRM (Siebel 7) (e.g. Call Center)
• Communication with ERP-System (SAP R/3)
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Avatars and Talking Heads:
An Appealing MPEG-4 Front End for Web Applications
Face model for convenient rendering and animation
• Realistic representation of a “human” face
• Animated by speech, facial expressions, or others • Easy adaptation/personalization
May I help you ?
Benefits
•Attractive eye catcher
•Natural way to communicate
•Creating own animations in entertainment and games •High-level animation with natural facial expressions •Easy speech- & text-based authoring of animations •MPEG-4 conform (ISO standard)
• Adaptation to natural persons
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Avatar - Technology
3-D Geometric model/scene graph with
• Feature points (84) • Facial texture,
• Face animation rules
Animation controlled by a complete set of standardized facial actions
§ allows a compact representation of natural facial expressions • 2 high-level FAPs:
- 14 Viseme (visual correlate of phoneme) - 6 basic Expression (joy, anger, fear, ...)
- Textual description of expression parameters
- Aggregation of low-level FAPs to achieve any expression • 66 low level FAPs
- Associated with the displacement or rotation of the facial feature points -Imitate the facial muscles
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
2D Avatars – Giving a Face to Communication
Features:
• Simple creation of 2D models from photos • Automatic animation generation (text or voice) • User friendly generation of animations
• Very low file size / data rate
• Adaptation to low performance mobile devices
Author input Phoneme to FAP Speech synthesis Speech recognition Facial Action Editor Viewer pho-neme list MPEG-4 conform animation stream audio audio Text Voice Custo-mizer 2D Face model 2D Image
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Applications on Mobile Terminals
Animation of voice messages
Text-based visual speech synthesis
• New message on voice box
• According to sender_id a face model is selected • User lets the “Talking Head” speak
- real-time phoneme recognition(VSR) & viseme translation
- delaying the audio a little bit for synchronisation)
• User annotates a text message with short facial actions (similar to smiley’s -blinking with an eye-lid) and dispatches it
• If receiver has booked “Talking Heads Service”, an audio stream and its corresponding facial animation stream is created by the service provider • Animation is delivered to receiver (including reply smiley’s)
Applications in Messaging, Web Applications, e-business, games and entertainment
Applications in Messaging, Web Applications, e-business, games and entertainment
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Personalized Traffic Information Visualization
Example: ISO MPEG-4 based video traffic control system
WestendEXIT START
Kaiserdamm
Multimedia Communication will bring major enhancement to future traffic systems: •Intelligent Navigation
•Inter Car Communication, •Intelligent Traffic control
Multimedia Communication will bring major enhancement to future traffic systems: •Intelligent Navigation
•Inter Car Communication, •Intelligent Traffic control
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Surveillance Archive using MPEG-7
The Surveillance Provider • Standard Conform Interface • Employees got used to their GUI, and
• keep the GUI even if changing the customer
• Query Formulation based on Visual Features Surveillance DB ... ... Server Query Client Demultiplexer MPEG4/7-Codec Query Results Object Texture and Shape Scene and Object Trajectory Object
Representation Video Sequence Security Facilities in the Company
• MPEG-4/7 Cameras feed • a Multimedia DB, providing • a Standard Conform Interface.
Critical Data is kept in the Company
MPEG-4 Video+ MPEG-7 Descriptions
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Bandwidth Requirements for Multimedia
Multimedia PC PSTN/ISDN Cable UDSL/ADSL GSM/DECT/UMTS Broadcast TV: 1,5 - 5 Mbps MPEG-2 HDTV: 6 - 20 Mbps MPEG-2 Videophone: 64 - 384 kbps H.261/H.263 Conferencing: 384 - 2 Mbps H.261/H.263
Mobile video: 16 - 64 kbps H.263/MPEG-4
Broadcast TV: 1,5 - 5 Mbps MPEG-2 HDTV: 6 - 20 Mbps MPEG-2 Videophone: 64 - 384 kbps H.261/H.263 Conferencing: 384 - 2 Mbps H.261/H.263 Mobile video: 16 - 64 kbps H.263/MPEG-4
LQ audio: 2x64 - 96 kbps MPEG-audio
HQ audio 5.1 channel: 384 kbps MPEG-audio
ISDN-Telephony: 64 kbps G.711
DECT-Telephony: 32 kbps DECT
Voice o. IP: 5.3 - 8 kbps G.723/G.729
LQ audio: 2x64 - 96 kbps MPEG-audio HQ audio 5.1 channel: 384 kbps MPEG-audio ISDN-Telephony: 64 kbps G.711 DECT-Telephony: 32 kbps DECT Voice o. IP: 5.3 - 8 kbps G.723/G.729 mail Mobile Multimedia Terminal TV/HDTV-Set + STB
The requested quality of audio and video presentation determines the needed bandwidth for multimedia services.
The requested quality of audio and video presentation determines the needed bandwidth for multimedia services.
n Broadcast Services more than 500 programs available, 2...8 Mbit/s per video including PayTV, NVoD, local channels >2..3 Gbit/s in total
n Internet Services World wide web 1..10 Mbit/s per each active user Information on Demand (VoD, streaming)
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Multimedia Technologies
Medium/ Network Feature Technology/ Standard Property Benefit/ Application Still Image High Resolutione.g. 4000x4000 pixel JPEGJPEG 2000 · High Resolution· Region of Interest · Progressive Digital Camera Deatil Information Video · Motion · Reduced Resolution · Life Scenes H.263
MPEG-2/4 · Good Quality forData Rate · Cost effective Codecs (HW/SW) available · Surveillance · Enhanced Communication · Videophone/ -confernece Multimedia · Combination of
· Life Video & Audio · Still Images · 3D-Objects · CAD Models · Text,... MPEG-4 · Composition of MM Information · Hyperlinking · Interactive Manipulation · 3-D Navigation · Interactive Multimedia · Information Retrieval · Teleserviece/ Telemaintenance · Virtual Reality · E-Commerce Multimedia Database Standardized description of
Multimedia data for fast & efficient search
MPEG-7 Search &
transformation of media according to description of information Personalized media selection Digital libraries Universal access
s
Universtät Klagenfurt - IWAS Multimedia Kommunikation 623.715 Juni 2004
Multimedia is Possible Today
n IT industry enables affordable terminal systems - high processing power, large memory, display
n Application areas for Phone, TV, and PC merge together
n Convergence of networks supports multimedia communication - gateways, IP everywhere
n Standards (MPEG, DVB, IETF, ITU-T, ...) create planing security - digitization of images, audio/video compression
- multiplexing schemes - network interfaces
- protocols and programming languages (HTML)
n Online service and Internet access has been established
n Transport of broadband realtime information is possible - ATM, IP, SDH, WDM, switches, router
- high bandwidth access networks
Elements of a multimedia terminal (example)
Audio (G.7xx) Processor Network interfaces (DECT)
Video (H.264) Busses Device interfaces (Bluetooth)
Data Operating system (CE) User interfaces (JAVA, HTML)
Memory Protocol (IP) Applications (videophone, WWW)
Elements of a multimedia terminal (example)
Audio (G.7xx) Processor Network interfaces (DECT) Video (H.264) Busses Device interfaces (Bluetooth) Data Operating system (CE) User interfaces (JAVA, HTML) Memory Protocol (IP) Applications (videophone, WWW)
PC
TV
Telephone
Fast technical development enables multimedia services today. Digitization and compression of images gave the start off.