Workshop on Multimedia Semantics. Semantic processing of multimedia data by MPEG-7 for capacious knowledge management

(1)

Lehrstuhl Informatik V (Informationssysteme) Prof. Dr. M. Jarke

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Workshop

on

Multimedia Semantics

Semantic processing of multimedia data by

MPEG-7 for capacious knowledge management

Marc Spaniol,

Ralf Klamma, Matthias Jarke

RWTH Aachen

(2)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Agenda

• Multimedia knowledge creation

• Framework of community-centred knowledge management

– Transcription

– Addressing

– Localization

• Semantic knowledge management by XML and MPEG-7

• The Virtual Entrepreneurship Lab (VEL)

(3)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Knowledge creation process

• Making implicit knowledge

explicit (Externalization)

• Management of

explicit data

(Combination)

• Retrieval of

explicit knowledge

(Internalization)

• Sharing of

implicit knowledge

(Socialization)

⇒

There is no media free knowledge

⇒

(Multi) media semantics?

⇒

Knowledge Management = Multimedia Management + Ontologies

Externalization

Internalization

implicit

explicit

implicit

explicit

Combination

Socialization

(4)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

How to transfer

media dependent knowledge?

Ontologies

• Abstraction of information

• Strict classification and

categorization of data

• Modeling of data sources and

information types

• Creation of domain specific

structures

• Predefined data structures

Multimedia Concepts

• Explanation of activities

• Media versatility

• Unspecified information

• Sample creation

• Learning by doing

(5)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Analysis of

knowledge management concepts

⇒

Goal: Combining their benefits by simultaneously

reducing their deficits

Multimedia Concepts

Applications

Ontology based

Information Systems

Content adaptability

o

Media versatility

+

o

Community suitability

+

-Interactivity

o

Personalization

-

+

User interface flexibility

o

(6)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Knowledge creation framework:

Transcription - Addressing - Localization

•

New computer science approach:

Combination of

ontologies

and

multimedia concepts

→

Supporting Transcription

by (re-) combination of

transcription tools

→

Stabilization Addressing

→

Optimizing Localization

•

Testing in an interdisciplinary

collaborative research center

Externalization

Internalization

implicit

explicit

implicit

explicit

Combination

Socialization

The Knowledge-Creating Company [Nonaka, Takeuchi 95]

Transcribing – Media/Reading [Jäger, Stanitzek 02]

Signals of disturbance [Kümmel, Schüttpelz 02]

Multimedia

(7)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Semantics in

community multimedia

(8)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Structural and content based KM

with XML and MPEG-7

Basis

Schema-

Links&Media Basic Basic

Tools Localization Tools datatypes

XML

MPEG-7

[IEEE 02]

User

interaction

User

preferences

Usage

History

Navigation

& Access

Index

Views

Variations

Content organization

Collections

Models

Creation &

Production

Content Management

Media

Usage

Content description

(9)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Structuring of

heterogenous Data data XML

<

Mpeg7

>

...

<

ContentDescription

xsi:type

="

ContentEntityType

">

<

MultimediaContent

xsi:type

="

VideoType

">

...

<

VideoSegment

id

="

seg1

">

<

TextAnnotation

type

="

Question

">

question

</

TextAnnotation

>

<

TextAnnotation

type

="

TextAnnotation

type

="

Companies

">

company name

</

TextAnnotation

>

<

MediaTime

>

starting and ending point of the medium

</

MediaTime

>

</

VideoSegment

>

...

</

MultimediaContent

>

</

ContentDescription

>

...

3-dimensional Categorization

Multimedia Information

Additional Information

(10)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Content and context

description by MPEG-7

<

Mpeg7

>

<

DescriptionMetadata

id

="

Kais Rolle

">

general information about the document

</

DescriptionMetadata

>

<

ContentDescription

xsi:type

="

ContentEntityType

">

<

MultimediaContent

xsi:type

="

MultimediaCollectionType

">

<

ContentCollection

name

="

Kais Rolle

”>

<

TextAnnotation

>

Collection type (public/private)

</

TextAnnotation

>

<

Content

xsi:type

="

ImageType

">

Medium, technical Data, Annotations

</

Content

>

<

Content

xsi:type

="

VideoType

">

---</

Content

>

</

ContentCollection

>

</

MultimediaContent

>

</

ContentDescription

>

<

ContentManagement

xsi:type

="

UserDescriptionType

">

user management

</

ContentManagement

>

Multimedia Information

Transcription

(11)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Case study:

Didactic concept of VEL

• Hypermedia learning environment

• Extensive, close to reality media material

• Explorative, non-linear learning

• Context dependent presentation

• Repeated considerations due to context changes

• Learning structure:

– Student = explorer

– Teacher = consultant, mentor

• Public / private collections

• Users‘ annotations

(12)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

(13)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Conclusion & Outlook

• MPEG-7 helps to overcome gap between

Multimedia Concepts

and

Ontologies

• Combining

loose characterization

and

generic classification

with

domain modeling

and

categorization

• Research on personalization and adaptive user interfaces

• Current projects at :

i5

SOCRATES:

Chat/Talk-tool for aphasics self help groups

MARS:

Electronic music analysis tool

MECCA:

Virtual movie archive for cinematic scientists

⇒

Covering multimedia semantics by MPEG-7

(14)

M. Spaniol,

R.Klamma,

M. Jarke

Workshop on

MM Semantics

Workshop on

MM Semantics

Contact

www.graeculus.de

[email protected]