Architecture for using Java bytecode for processing Digital Items

(1)

GOPINADHAN, GAUTAM V.S. Architecture for using Java bytecode for processing Digital Items. (Under the direction of Assistant Professor Gregory T. Byrd).

The MPEG-21 Multimedia Framework aims to provide an end-to-end framework for transparent and universal access to media content. It defines mechanisms to represent, process and exchange aggregate media objects, called Digital Items.

One of the main requirements in MPEG-21 is to devise a mechanism for making Digital Items dynamic, by allowing program code to be packaged along with them. This executable code consists of operations that are used toprocessthe Digital Item. The design of this mechanism is categorized into three parts: a specification to express and package these processing operations, an engine on the platform to execute the processing operations, and base APIs on the platform that will assist in implementing these processing operations. This thesis proposes a solution that uses Java for expressing Digital Item processing operations. Also as a part of this solution, it proposes adapting the MPEG-J specification to define an engine for the execution of the the Java based processing operations. The MPEG-J specification is an application engine specification for the execution of mobile MPEG-Java code, defined in the MPEG-4 systems standard.

The MPEG-21 standards committee later realized that two kinds of processing operations should be supported: simple processing operations, andcomplex processing op-erations. It was decided that a scripting language should be used to support simple pro-cessing operations, while complex propro-cessing operations should be supported by a language like Java. The thesis adapts the first solution and proposes a second one to use Java to specifycomplex processing operations under the new architecture.

(2)

by

Gautam V.S. Gopinadhan

A thesis submitted to the Graduate Faculty of North Carolina State University

in partial satisfaction of the requirements for the Degree of Master of Science in Computer Science

Department of Computer Science

Raleigh

2003

Approved By:

Dr. Frank Mueller Dr. Douglas S. Reeves

(3)

(4)

Biography

Gautam Varissery Subramanian Gopinadhan was born 26 years back in a small town called Palakkad in Kerala, India. He completed his 10th standard from Campion School, Mumbai and 12th standard from Jai Hind College, Mumbai.

(5)

Acknowledgements

I would like to thank Dr. Greg Byrd, for his support, encouragement and his patience, while directing me through this project. I also greatly appreciate the independence he has allowed me while working on this project.

I am greatly indebted to Dr. Vishy Swaminathan from Sun Microsystems Inc., for giving me the opportunity to work on this project. I would specifically want to thank him for the technical discussions, criticisms and most of all, his personal support, all of which have made this thesis possible.

I would also like to thank, Dr. Douglas Reeves and Dr. Frank Mueller for their encouragement and for being on my advisory committee.

Last but not the least, I would like to acknowledge my friends Amit, Jaydeep, Preety, Palash and Kashi for their understanding.

(6)

List of Figures

1.1 The User, Digital Item(DI), and the MPEG-21 Framework. . . 6

1.2 Digital Item Declaration model concepts [22]. . . 10

1.3 Digital Item Declaration model concepts [22]. . . 11

1.4 Example relationships between elements in DID model. . . 12

1.5 Sample syntax diagram [22]. . . 13

1.6 XML syntax for DIDL elements: Part 1[22]. . . 13

1.7 XML syntax for DIDL elements: Part 2[22]. . . 14

1.8 A DIM-centric view of the MPEG-21 architecture. . . 18

1.9 The MPEG-J Application Engine. . . 22

3.1 Writing a Java DIM. . . 28

3.2 Object and Object Type. . . 29

3.3 The Digital Item Manipulation Engine. . . 35

3.4 Example DIBO APIs. . . 38

4.1 Relationship between DIM, DIBO and DIxO. . . 40

5.1 A J-DIxO class. . . 50

5.2 Mapping of DIML Data Types to Java Data Types. . . 51

(9)

Chapter 1

Introduction and Background

Executable code communicated between entities can be used to achieve different objectives. To list a few, they can be used

• to access resources on a remote terminal;

• to establish communication semantics between two entities;

• to encode a data document in the form of interpretable directives;

• to improve the capabilities of a static document;

• to improve the capabilities of the destination entity;

• as a tool to interact with other entities.

We turn our attention to the specific objective ofimproving capabilities of a static document. An environment where we can easily observe some existing solutions for achieving this objective is the World Wide Web.

(10)

mechanisms are supported. For example, scripting languages such as Javascript [34] (or ECMAScript [4]) are used to implement interactive and dynamic web pages without much client support and programming know-how. Other mechanisms, such as Java Applets [11], are used to achieve more complex interactions and presentations at the terminal. Java Ap-plets are able do this because they have access to a much broader presentation and utility support at the client. Also, they consist of compiled binary code thus having better per-formance for larger implementations than interpreted plain-text Javascript code. Thus, we have observed that packaging executable content is indeed an effective way to increase the capabilities of a static document. There are many ways to do this, each providing different capabilities.

We now shift focus to technologies in a peer-to-peer paradigm of networking and try to draw parallels. The MPEG-21 Multimedia framework [24] is designed to be used in a peer-to-peer setting. It defines a static declaration language-based on Extensible Markup Language (XML) [9] to create documents. XML, a more powerful language than HTML, is used to represent the data in a structured manner and convey information regarding, not just presentation, but also structure and relationships between data resources. As part of the requirements that are addressed by this thesis, we are trying to design a mechanism to make XML documents in the MPEG-21 framework more dynamic. We would like these XML documents to be able to interact with the user at different levels.

We note the similarities between the MPEG-21 scenario and the web, and can thus use our observations from the web domain to form a strategy for designing such a mechanism: We can package executable code along with MPEG-21 XML documents to make them more dynamic. Depending on the capabilities that we want to provide, we can use either a text-based interpreted language, or we can use a binary format like Java.

In this thesis we address the specific case of supporting the more complex opera-tions embedded in the XML document. Java is our choice for designing and implementing such a functionality and we justify this choice later in this thesis.

(11)

The ideas behind the work in this thesis were developed in response to a call for proposals [19] for the design of the Digital Item Processing part of the MPEG-21 framework.

1.1

Outline

The remaining sections in this chapter cover background material. We present an overview of XML, the MPEG-21 framework and MPEG-J. The material in this thesis assumes some knowledge on XML, Java and ECMAScript. However many examples are covered in this thesis that have sufficient documentation so that the reader can gain a good level of understanding of the work presented.

We state the problem in Chapter 2 and list out the various requirements for the design of the solution. Chapter 3 discusses the first solution. A second solution is proposed in response to a change in requirements and architecture. The changes in the architecture is presented in Chapter 4, while the solution specification is presented in Chapter 5. We carry out a brief comparison and analysis of the solutions presented and outline directions for future work in Chapter 6 and Chapter 7 respectively.

1.2

XML and XML Schema

XML (EXtensibleMarkupLanguage) [9] is a text-based language that is used to express data along with its logical structure and layout. Two meta languages have been developed in order to describe the characteristics and structure of data that can occur in an XML document - the Document Type Definition (DTD) [8] and the XML Schema Definition Language [38]. An XML document is said to be valid if the contents of that document conform to rules laid out in its DTD file or its Schema file.

(12)

1.3

The MPEG-21 Multimedia Framework

This section presents background information about the MPEG-21 framework, and the technologies and standards that are a part of this framework. We start off by identifying the need for an MPEG-21 framework. A use-case is presented that will serve as the basis for developing examples in the chapters to come. An overview of the different parts of the framework is presented and subsequently we look into the details of two specific parts that are important for the understanding of the material in this thesis.

1.3.1 The Need for MPEG-21

Currently, delivery and consumption of multimedia content occurs mostly through ad hoc solutions. There exists no end-to-end framework that takes into consideration as-pects such as media format, transport methods, user environments, device characteristics, advanced media interactions and user’s rights over the media. As a result, users trying to access a media item have to ensure that the media type, transport mechanism, device, etc. are correct and compatible for accessing the media.

To better facilitate transparent media consumption and delivery, there is a need for a standardized end-to-end framework that would shield the basic user from the complexities that arise from such variations and one that would support complex types of interactions between users transacting with media documents.

(13)

want to create separate versions of the library for different devices, formats, and users. A person on the computer should be able to play the library over the cable line, and so should a person with a handheld PDA via a wireless link.

While many of the actions mentioned in the use case can be achieved individu-ally using various technologies, there is no single technology that can support all of the actions above. The MPEG-21 standard aims to define such a framework, by integrating current technology solutions where available and creating new ones to support the identified requirements.

MPEG-21 provides description tools to ensure that all different technology so-lutions related to assembly, transport, exchange and rendering of multimedia files coexist together. This is different from the description support that technologies such as Session Ini-tiation Protocol (SIP)[36], Session Description Protocol (SDP)[12] or Real Time Streaming Protocol (RTSP)[37] provide. All these protocols deal with describing individual instances of a media resource or a media related transaction. The MPEG-21 description tools al-lows multiple such media resources coexist together. For instance, a media resource in an MPEG-21 object can be a reference to an internet telephony session information (described using SIP) that will allow the user to connect to another user. Within the same MPEG-21 object, there could exist a link to a media streaming source that is described using SDP.

MPEG-21 also provides mechanisms to express user rights and mechanisms to aggregate many media files in a structured manner.

1.3.2 Introduction to MPEG-21 concepts

MPEG-21 defines two concepts that are central to the design and understanding of the framework: TheDigital Item and theUser.

Digital Item

A Digital Item is a structured digital object with a standard representation, iden-tification and meta-data within the MPEG-21 framework. This entity is the fundamental unit of distribution and transaction within this framework.[17]

(14)

Figure 1.1: The User, Digital Item(DI), and the MPEG-21 Framework.

also any kind of digital content that accompany a media object. For example, a DI for the use case in the previous section will consist of resources such as album cover art, all audio and video recordings of songs, lyrics for individual songs, as well as guitar strumming instructions. All these resources will be organized using some description tools into albums or in some other manner as desired. Metadata in the DI can provide information about the QoS requirements for a particular media resource that may be utilized to stream the media across the wireless link.

User

In MPEG-21 a User is any entity that interacts in the MPEG-21 environment or makes use of Digital Items.[17]

(15)

1.3.3 Parts of MPEG-21

MPEG-21 has defined twelve parts of standardization. An overview of the various parts is given below. Parts 2 and 10 are explained in greater detail in the sections that follow, since they are relevant for the understanding of the contributions presented in this thesis.

Part 1: Vision, Technologies and Strategy [17] This part is concerned with defining the vision to support “transparent and augmented” [17] use of multimedia content. This document also serves to integrate other parts of the MPEG-21 standard and other relevant technologies, to outline how the MPEG-21 vision is achieved. Finally, it outlines astrategyto develop the required standards through the collaborative effort of the various bodies.

Part 2: Digital Item Declaration [22] Digital Item Declaration (DID) defines an ab-stract model and an XML Schema-based declaration language, called Digital Item Declaration Language (DIDL), that is used for the representation of Digital Items. The abstract model is extensible and can be used to create more complex models, as is done within other parts of the MPEG-21 standard. See Section 1.3.4 for further details.

Part 3: Digital Item Identification [21] Since Digital Items are intended to be used over a wide variety of applications and domains, and have a potentially infinite lifetime, it is essential to have an identification mechanism to refer to a DI or addressable sub-parts of a DI. Digital Item Identification (DII) defines XML Schema elements that are used for realizing this identification mechanism. However DII doesnot define an identificationsystem; it defines amechanismthat can use such identification systems. Examples of such identification systems are, an ISBN [2] number for books in print, a URN [6], etc.

Part 4: Intellectual Property And Management [20] This part of the MPEG-21 stan-dard is devoted to developing a framework for Intellectual Property Management and Protection (IPMP). The project defines standardized techniques for accessing IPMP tools and frameworks for the exchange and authentication of such tools.

(16)

a language to express rights using terms defined in the Rights Data Dictionary (see next item). These rights can be used to control access to digital resources.

Part 6: Rights Data Dictionary [16] Rights Data Dictionary defines a set of terms for use with the Rights Expression Language. It also specifies how further terms can be created and registered for use with the REL.

Part 7: Digital Item Adaptation [23] In order to achieve transparent and universal access to multimedia, Digital Items need to be adapted to meet the needs of various networks, devices or user preferences. Digital Item Adaptation is concerned with providing tools that can aid an adaptation engine to adapt Digital Items and enable it to meet constraints. The adaptation engines themselves are non-normative and are not a part of the standard.

Part 8: Reference Software [29] All previous parts deal with providing descriptors or tools but do not consider the platform related specifications. TheReference Software part is concerned with creating system related specifications. This part of the standard is relatively new and requirements have not been fully identified.

Part 9: File Format [27] DIs can consist of binary media data and textual information such as metadata, XML constructs, and DIDL documents. Therefore theFile Format part of the standard defines how DIs having such varied content can be aggregated and stored as files. It derives a lot of concepts from MPEG-4 file formats, which were designed to allow for binary and textual data.

Part 10: Digital Item Processing [30] DIs, as defined initially, were static as they were built using DIDL declarations. Digital Item Processing (DIP) is focussed on making DIs dynamic, by allowing executable code to be packaged with DIs. Thus, users can add dynamic functionality to a DI. Further, these operations can shield the MPEG-21 capable terminal from the internal representation of the DI promoting interoperability of DIs at the processing level. DIP is presented in detail in Section 1.3.6.

(17)

include watermarking and fingerprinting of audio and video content.

Part 12: Test Bed for MPEG-21 Resource Delivery [26] This part provides for a test bed for testing scalable media delivery, and evaluation of these delivery pro-cedures in the face of varying network characteristics.

1.3.4 Digital Item Declaration

DID is the second part of the MPEG-21 standard. It defines various abstract concepts and develops a model using these concepts. This model forms the basis for under-standing and expressing the structure of a Digital Item. This flexible abstraction or model is referred to as theDID model. The MPEG-21 DID standard also specifies theDigital Item Declaration Language (DIDL), which is an XML Schema representation of theDID model. DIs are, in essence, instance documents of XML Schema classes defined by the various parts of the MPEG-21 standard. The most basic of these schema classes is developed in the MPEG-21 DID part and may be extended by the other parts of the standard. These basic schema classes form the DIDL.

DID model

Figures 1.2 and 1.3 define the abstract terms and state their relationships with other terms within the DID model as an EBNF [15] representation. The EBNF forms presented in the diagram are defined in the MPEG-21 DID [22] document.

Figure 1.4 illustrates example relationships between some of the important ele-ments of the DID model. It shows a container consisting of three item children. Each item either contains sub-items and/or components. Descriptors contain information describing its parent. This figure has been derived from an example figure also presented in the DID [22] document.

As mentioned, every element in the abstract model has a DIDL representation such that the relationships expressed in the model are preserved in the representation. For example, the element Container in the model has an equivalent DIDL element named as CONTAINER 1.

1

(18)

DID term with Explanation EBNF representation

Container A structure for grouping items and containers. It can be used to form log-ical packages or loglog-ical shelves.

container ::= container* item* descriptor*

Item A grouping of sub-items and compo-nents bound to relevant descriptors. Items may allow choices and be conditional on pred-icates. An Item is considered as the represen-tation of some “work”.

item ::= (item | component)* choice* descriptor* condition* annotation*

Component Component binds resources to descriptors.

component ::= resource descriptor* anchor* condition*

Anchor An anchor binds descriptors to a fragment. A fragment is specific location or a part of a resource.

anchor ::= fragment descriptor* condition*

DescriptorA descriptor associates informa-tion with the enclosing element.

descriptor ::= descriptor* (component | statement) condition*

Figure 1.2: Digital Item Declaration model concepts [22].

Apart from the elements corresponding to the terms in the DID model, DIDL defines three more elements; theDIDLelement, theDECLARATIONelement and theREFERENCE element. Their exact semantics are defined in the next subsection.

(19)

DID term with Explanation EBNF representation

Condition It defines the enclosing element to be optional and links it to selections that affect its inclusion.

condition ::= predicate+

ChoiceDescribes a set of selections that can affect the configuration of an item.

choice ::= selection+ descriptor* condition*

Selection Describes a specific decision that

will affect one or more conditions in an item. selection ::= predicate descriptor*_condition*

AnnotationDescribes information about an-other identified element of the model without adding to that element.

annotation ::= assertion* descriptor* anchor*

AssertionIt defines the value of a choice by asserting true , false or undecided values for associated predicates.

assertion ::= predicate*

Resource A resource is a uniquely identifi-able asset.

Fragment A fragment denotes a specific point or range within a resource. (e.g. sec-tion of media clip)

StatementIt is a literal textual value that is not an asset.

Predicate A declaration that can be true, false or undefined.

Figure 1.3: Digital Item Declaration model concepts [22].

DIDL Element syntax

We present the syntax of some of the important DIDL elements defined in the previous subsection. Syntax diagrams presented in this section use the format specified in the DID standard [22]. The reference diagram in Figure 1.5 taken from the MPEG-21 DID standard, gives information about how to interpret the syntax diagrams given in Figure 1.6 and Figure 1.7.

(20)

Descriptor Item

Item Descrptor

Descriptor Resource Component

Component Descriptor

Resource

Descriptor Component Descriptor

Item Component Descriptor

Resource

Resource Container

Item

Figure 1.4: Example relationships between elements in DID model.

representation of the various DIDL elements. Additional information about their syntax and validation rules is given below.

DIDL The DIDL element must be the root element of any DIDL instance document. It should include a namespace declaration with the value of the namespace attribute as “urn:mpeg:mpeg21:2002:01-DIDL-NS”. This will announce that the instance doc-ument is meant to conform to the DIDL specification.

DECLARATIONS As mentioned earlier the DECLARATIONS element is used to define a set of DIDL elements without actually instantiating them. These elements will be instanti-ated only when the elements are referenced by theREFERENCE element.

CONTAINER There are no special validation rules regarding CONTAINER.

(21)

Figure 1.5: Sample syntax diagram [22].

(22)

Figure 1.7: XML syntax for DIDL elements: Part 2[22].

within descendant items.

COMPONENT The COMPONENT is a building block within the DID language, since it links resources to descriptors. If there are multiple resources defined for aCOMPONENT, they are to considered equivalent and any one may be selected.

RESOURCE A RESOURCE has three attributes - mimeType, ref and localPath. mimeType defines the “Mime Type” [7] of the resource. refrefers to the URI value that identifies the resource. If therefattribute is omitted then the resource must be included inline. If the inlined resource is binary then it should be encoded using base64 [7] encoding.

DESCRIPTOR There are no specific validation rules for descriptors.

(23)

REFERENCE A REFERENCE element is a mechanism to reference any addressable DIDL ele-ment. A DIDL element is addressable only if it has anidattribute or the choice id attribute. CONTAINER,ITEM,COMPONENT,DESCRIPTOR,ANCHORandANNOTATIONare el-ements with anidattribute. AREFERENCEelement has no children, only one attribute target, which is a URI expression pointing to the element being referred to.

Other DIDL elements, specifically ANCHOR, CHOICE, SELECTION, CONDITION and ANNOTATION are not presented since future examples and work presented in this thesis do not use them.

1.3.5 Illustrative Example

Let us consider the use case presented in Section 1.3.1. Given below is a possible DID, represented using DIDL, for the use case. Let us also name this DI as “The Boo Far Band Media Library” for further reference. Note that this DI only illustrates how the content of the DI is expressed using DIDL and does not present the XML for expressing rights or any other metadata.

<?xml version="1.0" encoding="UTF-8"?>

<DIDL xmlns="urn:mpeg:mpeg21:2002:01-DIDL-NS"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="urn:mpeg:mpeg21:2002:01-DIDL-NS DIDL.xsd">

<Declaration>

<Statement mimeType="text/plain"> Jane’s Song

</Statement> </Descriptor> </Declaration>

<Item>

<Descriptor>

<Statement mimeType="text/plain"> Boo-Fars first album

</Statement> </Descriptor>

(24)

<Component>

<Reference target="#JANE_2" />

</Component> <Component>

</Item>

<Item>

<Reference target="#JANES_SONG"/> </Descriptor>

<Descriptor>

<Statement mimeType="text/plain"> Hello darkness, my old friend.

</Statement>

</Descriptor>

(25)

</Component> </Item>

</DIDL>

1.3.6 Digital Item Processing

This section describes the Digital Item Processing (DIP) part of the MPEG-21 standard in greater detail.

Motivation

As we have seen earlier, Digital Items are essentially static declarations that are expressed using the DIDL. The MPEG-21 standard does not impose rules on how these Digital Items should be manipulated or interpreted. Barring REL and IPMP constraints, it is up to the discretion of the application program on how it should deal with the descriptors within a DI.

However, we can envision scenarios where the author of the Digital Item would like to impose some rules on how the Digital Item is viewed or experienced by other entities. For instance, consider the “The Boo Far Media Library” example (Section 1.3.5) that we have seen before. It is possible that Jane, the author of the DI, would like all songs in media library to be listed in alphabetical order. However, how does she ensure that other band members adding songs to the DI respect this representational detail about the DI? A simple idea would be to package a operation called “AddTrack” with the DI so that any user wanting to add tracks into the DI uses this operation to do so. Internally, Jane ensures that the AddTrack implementation adds tracks to the library in alphabetical order. Using this simple mechanism, Jane ensures that other usersprocessher DI in an intended manner. This is the basic idea behind Digital Item Processing.

(26)

DII Engine Engine REL Engine RDD Engine IPMP Engine DIA Engine DID DIME User Application DIM DIM DIM Repository

Multiple DIMs running in DIME

Network File Sys. Device UI DIBOs Other (n/w i/o device etc.) DIBOS & APIS

...

conn. DIBOs to other engines resource codec Media

Figure 1.8: A DIM-centric view of the MPEG-21 architecture.

Digital Item Methods, Digital Item Method Language and

Digital Item Base Operations

The operations that are packaged along with a DI are termed as Digital Item Methods or DIMs. They are analogous to the member functions in aggregate data types or objects. DIMs can be used for

configuring a DI - e.g. Assert the correct predicates so that appropriate choice elements are selected in the DIDL description.

processing a DI - e.g. Produce an output by processing the internal content of the DI in a particular manner.

(27)

expressing the presentation of the DI - e.g. When playing song X, the user should be asked to decide if she would like to play the long version of the song or the short version.

DIMs are intended to be executed by a normatively defined engine in the MPEG-21 platform. This engine is called the Digital Item Method Engine or the DIME. The responsibility of the DIME is to ensure that all DIMs that are packaged with the DI are managed properly and executed on request.

A DIM must have access to some libraries or base operations that it can use to implement its functionality. In the DIP scenario, these base operations would provide func-tionality such as DIDL parsing support, calls to find out information about the platform or the environment, hooks on the platform that the DIM can call to implement presen-tation, etc. These operations are provided by the platform and are called Digital Item Base Operations or DIBOs. They are analogous to the standard library for a programming language.

The MPEG-21 DIP standard requires that DIBO operations should be provided for performing DIDL manipulation operations (which may be provided as a service from the DID engine), operations for accessing services of other engines on the MPEG-21 platform, operations for accessing system and network resources, operations for controlling media codec resources, and operations for displaying basic interfaces to the user. A complete list of the requirements is presented in the requirements document[25].

Digital Item Methods should not be used to implement full fledged applications, since MPEG-21 provides different mechanisms to package applications. DIMs should be viewed as a set of requests to manipulate the Digital Item at the DIDL level or requests to the Digital Item Processing Engine to provide the output of some DIBO call. DIM implementations are constrained to invoking other DIMs or DIBOs. DIMs should not be used to implement things like media manipulation functionality. DIMs can make requests to the DIP Engine to perform such media manipulations only if the engine implements such a functionality (either implicitly or as a DIBO). Otherwise the DI author can package separate tools for this purpose.

(28)

DIMs can also invoke other DIMs.

Digital Item Method Engine and Digital Item Processing Engine

It is important to draw a distinction between the Digital Item Method Engine and the Digital Item Processing Engine at this point. DIMEs, as mentioned earlier, are used to executeDIMs. A DIME, however, is only one part of the DIP Engine. The DIP Engine itself may provide linkages between the DIME and the other engines of the MPEG-21 platform such as the engines responsible for DID, DII, IPMP, etc.

Consider that the PlayAllTracks DIM calls a base operation called dibo PlayMedia, which takes an argument specifying the location of the media. The user selects the PlayAllTracks DIM to be invoked. The DIME needs to check if the user is indeed authorized to invoke thePlayAllTracksDIM. Therefore it needs to contact the RDD and the REL engines to ensure that the user indeed does have sufficient rights to invoke the DIM. Assume further that the user has a license to play the track. The track must first be decrypted using the key in the license before the device can actually play the song. This framework support for providing connectivity with the other parts of the MPEG-21 plat-form and services to fetch and decrypt media, etc., is a part of the implementation of the DIP Engine. The sole responsibility of the DIM Engine is to manage the DIMs that it has received and to execute these DIMs on request. All other aspects relating to the processing of the Digital Item is the responsibility of the Digital Item Processing Engine.

Use cases

We list some use cases for DIMs. These use cases will be used to validate some of the choices made in the designs presented ahead and will also clarify some of the concepts presented in the discussions above.

Incremental Search DIM Consider a DIM in the media library DI that provides an incremental search facility. The interface accepts characters from the user while si-multaneously updating a list of the songs in the DI that contain the string or are closest matches.

(29)

database system. The DIM queries a database system to get a media items stored in the database and it then imports these media items into the DI.

1.4

MPEG-J

1.4.1 Introduction

The MPEG-4 standard is a Multimedia standard that was created for provid-ing three basic functionality [10, 5]: provide improved content-based interaction, improve compression efficiency and provide universal access to media content. MPEG-4 was also designed to work on a number of devices ranging from high definition televisions to hand-held devices. As a result there is a great variation in the resources available for rendering the media. MPEG-J, based on the Java technology, was therefore defined for adapting and modifying MPEG-4 streams in a resource-aware manner. MPEG-J is a part of the MPEG-4 systems specification.

A typical use case for MPEG-J technology is to package MPEG-J programs with the MPEG-4 streams and transport them to client machines. The program senses the resources available at the client terminal, and adapts the media so that the decoding engine does not consume more resources than is available at the terminal. This adaptation can be done dynamically as the resources on the terminal vary.

MPEG-J can also adapt the media in response to user interactions. This introduces the possibility of truly interactive multimedia applications.

1.4.2 The MPEG-J Application Engine

The MPEG-4 standards committee has defined the MPEG-J application engine for executing MPEG-J programs. It has been designed to execute MPEG-J applications as well as mobile MPEG-J code that is transferred along with the MPEG-4 streams. It exploits the mobility support provided by the Java programming language and the JVM.

(30)

1. Net APIs - for providing programmatic access to the network resources.

2. Decoder APIs - for programmatic access to decoder resources.

3. Scene APIs - for programmatic access to the MPEG-4 audio/visual object manipula-tion.

4. Service Information and Section Filtering APIs - for programmatic access to MPEG-2 related data structures.

These APIs are available as libraries for the MPEG-J programs executing in the application engine.

MPEG−J Application

java.lang java.io java.util MPEG−J APIs

JVM

Presentation Engine

Figure 1.9: The MPEG-J Application Engine.

1.4.3 The MPEG-J Architecture

MPEG-J programs come in two flavors: The MPEG-J application, which is an application running on the MPEG-4 terminal, and the MPEGlet, which is essentially an MPEG-J application that is streamed with the multimedia content to be demultiplexed by the MPEG-4 terminal and executed. MPEGlet classes are MPEG-J applications that implement the MPEGlet Java interface. We focus on the MPEGlet in further discussions since it is more relevant in this thesis.

(31)

related classes and objects are carried in a separate MPEG-J elementary stream2. A demul-tiplexer differentiates between the different elementary streams and directs the MPEGlet related data to the MPEG-J Engine. MPEGlets are automatically loaded on receipt by the terminal in a separate thread.

The MPEG-4 systems standard specifies specific rules about how MPEGlets are executed at the MPEG-4 terminal. The following sections describe these rules.

MPEGlet lifecycle

The MPEGletinterface defines four main functions. init,run,stopanddestroy. initis invoked to setup the MPEGlet for execution, anddestroyis to deallocate resources before the MPEGlet is destroyed. run is called when the MPEGlet is run in a separate thread and finallystop is called to stop the thread of execution.

Name Scope

Since MPEGlets are created by different users, they can potentially give the same name for their MPEGlet classes. Therefore a name scope must be defined to ensure that there are no naming conflicts.

TheObject Descriptor3 defines the name scope of MPEG-J classes and objects. In other words name clashes will not occur if the two classes having the same name arrived in different elementary streams.

Delivery

Since MPEGlets are time aware, time stamps are included to ensure that MPE-Glets are loaded by a particular deadline. In cases where an MPEGlet uses multiple helper classes to define its functionality, MPEG-J requires a dependency graph to be included so that the Java Virtual Machine loads all dependent helper classes before it loads the MPE-Glet. The need for such a dependency graph is obviated if the classes are bundled together using the Java archival format [11].

2

An elementary stream is a stream that carries only one kind of data: either audio data, video data or in this case MPEG-J data.

3

(32)

Security

MPEGlets and MPEG-J applications come from potentially untrustworthy sources, therefore it is necessary to ensure that such foreign code cannot cause havoc on the client terminal. The standard defines the MPEG-J Security Modelfor these MPEGlets [5]. Rules of the security model are to be enforced by an MPEG-J Security manager. The rules are as follows:

• MPEGlets are not allowed to load libraries or implement native methods.

• MPEGlets cannot read or write files on the MPEG-J terminal.

• MPEGlets are limited to the kind of system properties that they can access.

(33)

Chapter 2

Problem statement

To define a mechanism that will enable packaging of processing operations along with DIs such that all relevant requirements as identified by the MPEG-21 DIP requirements are met.

Requirements for Digital Item Processing are divided into requirements for Digital Item Methods, Digital Item Method Language (DIML), Digital Item Base Operations and Digital Item Method Engine.

DIM requirements state that DIMs should be a list of operations that are packaged with a DI. They should be platform independent, be individually identifiable, be editable and embeddable inline in a Digital Item Document.

The DIML is the language that DIMs are implemented in. They should have support for basic and user defined data types, concurrency support, and have support for standard language features such as functions, loops, conditional statements, etc.

The DIME should be able to execute DIM code, have ways of contacting the other engines in the MPEG-21 platform and be able to manage the acquisition of resources that are operated upon by the DIMs.

DIBOs are base APIs that are supported by the platform and these APIs should encompass functions relating to display and presentation of Digital Items, playing of re-sources, user interaction, functions for storage and network access, etc.

(34)

Chapter 3

Solution 1: Java for DIMs

This chapter looks at details of the first solution for packaging executable content with Digital Items. We first introduce the proposed solution followed by a detailed design discussion. We perform an analysis of the design and finally conclude with details of an implementation. Note that this solution is fairly high level, since, at the time this solution was prepared the exact specifics of various parts of the MPEG-21 framework were not formulated.

3.1

Introduction and Motivation

This solution proposes the use of Java as a Digital Item Method Language and defines an adaptation of the MPEG-J Application engine as the Digital Item Method Engine. Previously, with the design of the MPEG-J application engine, MPEG standards have defined the specifics of an engine that is capable of handling packaged programs. If this application engine meets all requirements that the Digital Item Processing standard has laid out, it would be a safer alternative to use a fully developed application engine with slight modifications than redesign one from scratch.

(35)

delivering platform independent, mobile code over networks. It has powerful programming constructs and an extensive library to aid programmers to write complex programs. It is supported on a variety of platforms ranging from large server systems to mobile phones and smart cards.

However, we realize that Java cannot be used for all purposes. For operations that do not utilize the capabilities of Java and for operations that are relatively simple and uncomplicated, using Java may be overkill. It is easy to envision use-cases, where complex DIMs will have to be written (e.g. see Section 1.3.6 - “Use Cases”). We would therefore like to propose that Java should be used for such purposes.

3.2

Design

There are different aspects to realizing a Java-based solution for packaging exe-cutable code with DIs. DIMs are meant to be single operations, whereas Java does not have the concept of an operation. We first define what the Java equivalent of a DIM is and state rules on how to implement a Java DIM. Further, we lay down rules on how Java DIMs can refer to XML arguments. Embedding a Java DIM in a DIDL document in a com-pliant manner is addressed subsequently. We then address how DIMs can support specific requirements and we identify problem areas. DIME design issues are tackled next, where the focus is on understanding the big picture and addressing sensitive issues such as security and namespaces.

3.2.1 Defining a Java DIM

Public member functions defined in Java classes are proposed to be viewed as DIMs. Specifically, there are three steps to defining a Java DIM.

1. Define a class with a default constructor (i.e. constructor without any arguments).

2. Within the class, define a public non-static member function that implements the intended DIM. This member function can call other member functions or instantiate other classes to implement the functionality.

(36)

methods, which is specified in the DI. Details of this declaration list are presented in the next subsection. The code in Figure 3.1 illustrates how to define a Java DIM.

package com.myplayer.dim;

public class PlayAllTracks {

/* Default constructor */ public PlayAllTracks() {

// do some initialization.

}

/* Intended DIM */

public void playTrack(org.iso.mpeg.mpeg21.objectTypes.Track tr) {

// code to play track.

} }

Figure 3.1: Writing a Java DIM.

These Java sources are then compiled into Java bytecode[11] which is then pack-aged in the DI as the executable content.

Passing Arguments to Java DIMs

(37)

(i.e. CONTAINER, ITEM, COMPONENT, DESCRIPTOR, ANCHOR, ANNOTATION and CHOICE) and considers such elements to be potential arguments to DIMs. It recommends that these argument Objects be classified into types termed asObject types. The author of the the DI provides this explicit categorization of Objects into Object Types. Such a categorization or mapping is termed as an Object Map. A standard set of Object Types are expected to be supported by the MPEG-21 platform.

<DIDL> <ITEM id="ALBUM−BOO−FAR−DEBUT"> <COMPONENT id="TRACK−BOO−ROCK"> ... </COMPONENT> <COMPONENT id="TRACK−FAR−POP"> ... </COMPONENT> </ITEM> </DIDL>

Object of ObjectType "Track"

Object of Object Type "Track" Object of Object Type "Album" {

// do something }

public void DIM−PlayTrack(Track a)

Figure 3.2: Object and Object Type.

Since Java methods in general can only take Java objects or basic datatypes as ar-guments, a mechanism is defined to convert Objects into Java arguments1. We propose that in Java DIMs, Object Types be represented by a corresponding set of Java classes defined on the platform and that Objects be represented as instantiations of these Java classes. These instantiations are initialized with a reference to the relevant Object at runtime. This conversion and initialization happens transparently within the DIME. Java object types are defined under the following package:

org.iso.mpeg.mpeg21.objectTypes.*.

There should therefore be a class under the above package for every Object Type that MPEG-21 defines.

Apart from standard object types, Java DIMs can also take basic datatypes that Java defines, namely, int, float, long, bool and double, as well as the type java.lang.String.

1_{The term “Java object” will be used to refer to an instantiation of a Java class. The term “Object”}

(38)

Java DIMs can also return values. However, when invoked by the DIME, these return values are ignored. Return values are informative when invoked by other methods or by an application.

3.2.2 Embedding DIMs in a DIDL document

This subsection deals with how Java-based DIMs are to be embedded in a com-pliant manner into a DIDL document. Since the addition of processing code constitutes a “work”, it should be placed within an item. This item, known as the Processing Item [18], will contain Java DIM declarations, Java DIM definitions, and the Object Map. We walk through the steps of creating a Processing Item below.

1. We first announce the item type by specifying a DII type to recognize that the ITEM contains Digital Item Method information. The string “DIM” [30] (line 51) is the identifier used to identify this as a DIM Processing Item.

...

45  46 <Item>

47

48  49 <Descriptor>

50 <Statement mimeType="text/xml">

51 <dii:Type>DIM</dii:type>

52 </Statement>

53 </Descriptor>

54

55 </Item>

2. We now define a sub item (lines 55-77) that contains the Java DIM declaration lists and resources that are required for the DIM. This sub-item contains multiple components. Each component should list in its descriptor a declaration list of Java DIMs defined in its child resource item. While it is possible to define multiple resources for a component, for a COMPONENT “if multiple RESOURCE children are present, they are considered equivalent and any one may be used” [22]. Thus to include a helper library a new component is introduced (line 73).

...

(39)

46 <Item> 47

52 </Statement>

53 </Descriptor>

54

55  56 <Item>

57 <Component>

58

59 <Descriptor>

60 <!-- Java DIM declaration list for

61 associated resource consisting of

62 fully qualified Java DIM names -->

63 <Statement mimeType="text/plain">

64 com.boo-far.mpeg21.TrackMethods.playTrack,

65 com.boo-far.mpeg21.TrackMethods.addTrackToPlaylist,

66 com.boo-far.mpeg21.TrackMethods.RemoveTrackFromPlaylist

67 </Statement>

68 </Descriptor>

69 <Resource mimeType="application/java"

70 ref="http://java.boo-far.com/boo-dims.class">

71 </Component>

72

73 <Component>

74 <Resource mimeType="application/java-archive"

75 ref="http://java.boo-far.com/boo-helper.jar"

76 </Component>

77 </Item>

78

79 </Item>

3. Once the Java classes and helper libraries have been inserted, we include the Object Map (lines 79-98) item in the Processing Item. For the sake of clarity the DIM Item is collapsed (lines 56-77), and an item listing media items (101-121) referred to in the Object Map is revealed.

...

47

(40)

52 </Statement>

53 </Descriptor>

54

55  56 <Item>

-- ...

77 </Item>

78

79

80  81 <Item>

82 <Component>

83 <Descriptor>

84

85 <Statement mimeType="text/plain">

86 AudioTrack

87 </Statement>

88 </Descriptor>

89

90  91 <Resource mimeType="text/plain">

92 Track-Boo-Rock

93 </Resource>

94 <Resource mimeType="text/plain">

95 Track-Boo-Pop 96 </Resource> 97 </Component> 98 </Item> 99 100 </Item>

101  102 <Item>

103 <Item id="Track-Boo-Rock">

104 <Component>

105 <Resource ref="ftp://mp3.boo-far.com/boo-rock.mp3"

106 mimeType="audio/mp3">

107 </Component>

108 </Item>

109 <Item id="Track-Boo-Pop">

110 <Component>

111 <Resource ref="ftp://mp3.boo-far.com/boo-pop.mp3"

112 mimeType="audio/mp3">

113 </Component>

114 </Item>

115 <Item id="Video-Boo-Rap">

116 <Component>

117 <Resource ref="ftp://vid.boo-far.com/boo-rap.mp4"

(41)

119 </Component> 120 </Item>

121 </Item>

3.2.3 Java DIMs: Addressing special requirements

We have specified sufficient rules for realizing Java DIMs. However, supporting some specific requirements deserves detailed explanation and analysis.

Editing DIMs

Consider the following DIM requirement - “Supporting the capability for editing (by textual or other means) and modifying of Methods by any User at any point in the value chain.” (see the DIP requirements document[25])

Since Java bytecode is not a textual format, it is tedious to support editing of DIMs. We propose some ideas for supporting code editing.

Due to the object-oriented nature of Java, the simplest way to change DIM func-tionality would be for the author to design DIM classes in such a way that users can inherit functionality from base classes and override them in their implementations. The newly created classes can then be compiled, and they can replace previous DIM implementations. DIMs or DIBOs may be provided that can automate the act of replacing DIM definitions.

It is also possible to support direct editing of code as well. Java source code can be packaged into descriptor elements, which can then be downloaded by users. This code can then be changed and compiled and the new classes can replace the existing ones. A more secure way of doing this would be for the author to provide tools that will compile the newly created source code and perform sanity checks to ensure that the new code does not implement illegal operations or that it does not destroy the functionality of other dependent DIMs in the DI. Details of this mechanism have not been provided. Security aspects will have to carefully reviewed before such a solution can be suggested.

(42)

Including DIMs inline

Another requirement of DIM states that Users should be able to include DIMs inline in a DIDL document. Since XML does not allow binary content to be included in a DI, bytecode will have to be encoded using “base64” encoding before inlining it into a DI. Base64 encoding essentially converts a three byte binary data into a four byte ascii text data. The downside of base64 encoding is that it incurs a space overhead of about 33%. Java supported compression techniques (jar and zip) can be used as a mechanism to compensate for this overhead to some extent.

Our solution proposes that inlining, though supported, should not be the preferred way to include Java bytecode in a DIM.

3.2.4 Design of the Digital Item Method Engine

Figure 3.3 gives an overview of the DIME proposed by this solution. It is similar to the MPEG-J Application engine. The main difference is that all MPEG-J related APIs are removed, and instead interfaces to communicate with the other engines in the MPEG-21 terminal are now part of the DIME. DIBO APIs are also a part of the DIME since they are required for the execution of the DIMs. Though not shown in the figure, an integral part of Java platform is the classloader, responsible for loading classes into the JVM. A Java platform is always packaged with a default classloader; however, custom classloaders can be used to enforce certain rules or maintain protection domains. For the rest of the discussion a classloader is assumed to be a part of the DIME.

The following are proposed to be a part of the DIME.

1. The Java Virtual Machine

2. supported platform packages i.e. java.io,java.lang and java.util

3. The DIBO API.

(43)

JVM

java.lang java.io java.util

DIP Engine

DII Engine

REL Engine

RDD Engine

IPMP Engine

DIA Engine

...

Frame work Conn.

API DIBO

Figure 3.3: The Digital Item Manipulation Engine.

Control Flow

To explain the working of the proposed DIME engine, the control flow within the DIME is given. In the example considered, the user receives a DI and waits for a list of DIMs in the DI to be displayed. The user selects a DIM, sets appropriate arguments and chooses to execute the DIM.

The DI on receipt is sent to the DID engine. While parsing, if the DID engine encounters a Processing Item (identified by the DII identifier “DIM”), it passes control to the DIME. On receipt of the Processing Item, the DIM engine parses the descriptors for each resource component to get a list of the DIMs defined for the resource. It then stores the resource into a buffer or a temporary repository. On receipt of the Item containing the Object Map, the DIME simply stores it for future reference.

(44)

classloader loads and instantiates the class that contains the DIM. It examines the argu-ments that the DIM takes (Java supports introspection of objects and member functions), refers to the object map to find out which elements in the DI are of the same Object Type, and displays them to the user. The user selects an appropriate resource or element for each such argument. The DIME again contacts the RDD and REL engines to ensure that the user has the correct rights to access these elements. Finally Java Object Type definitions corresponding to the arguments expected are instantiated and initialized with the elements (Objects) that correspond to the arguments selected. While executing Java DIMs there may be instances where actions such as verifying licenses or decoding media using a license need to be implicitly performed. Doing this transparently is the responsibility of the DIBO implementation. For example, assume that a “renderMedia” DIBO tries to render a media item that is passed to it. It is the responsibility of the DIBO implementation to check if the media item is encrypted and perform decryption using services provided by the DIME or otherwise.

Also note that the DIME is capable of handling multiple DIMs from multiple DIs at the same time while ensuring that they execute independently. The JVM has good multithreading support and is implemented to ensure that these multiple execution contexts do not interfere with each other.

Namespace issues

The DIME takes its cue from the MPEG-J engine (see Section 1.4.3) to set rules for handling name collisions. All classes packaged under one DI are under a single namespace. Therefore DIM classes with the same name occurring in two different DIs do not have name collisions. Name collisions within the same DI may be resolved by simple rules such as load time precedence. Under this rule, the classloader can refuse to load a class if a class with the same name has been previously loaded in the current namespace.

Delivery

(45)

required to run any of the DIMs that it declares. Note that individual components in the Processing Item need not be self contained and can depend on helper libraries contained in other components (see Section 3.2.2). There can even be components that serve only to package helper libraries and not package any DIMs. The order that these components are placed does not usually matter (unless there are name clashes), since the DIMs are listed for invocation only after all the components are parsed.

Security

Security is an important requirement that is conspicuously missing from the list of requirements for DIME. It is important to realize that MPEG-21 can run in any kind of a networking environment, ranging from a peer-to-peer network to a traditional client-server network. As a result the degree of trust that can be associated to the source of the DI is variable. It is therefore imperative that the engine executing these operations has sufficient security features to allow running of third-party executable content. The amount of damage that malicious DIM code can do on an MPEG-21 platform depends on the resources that DIMs can access.

The latest version of Java, has several security mechanisms [33] to ensure that untrusted code does not gain illegal access to system resources. The security mechanism is highly flexible and the user can express different levels of permissions afforded to the executing code. The user may set up protection domains so that code that is digitally signed and whose source is known and trusted, can get access to various DIBO functions and the local file system, while code that is untrusted does not have these kind of permissions.

In the least, this thesis suggests that the security model for the DIME should closely follow that of the MPEG-J Application engine. Especially, the security model must ensure that Java DIMs should have access only to other DIM code packaged within the DI, DIBOs and other Java APIs defined in the environment.

3.2.5 DIBO APIs

(46)

difference is that DIBOs are trusted platform APIs, while DIMs are third-party executable code. Designing DIBOs syntactically similar to DIMs enables a User to declareDIBOs as DIMs within a DI. Therefore, if a particular DIBO provides relevant functionality, and the user would like to provide this functionality within the DI, she can do so freely.

This thesis does not intend to propose DIBO APIs, but we list broad categories of Java DIBOs that maybe defined. This is listed here only to illustrate the approximate model for DIBOs assumed in the design of this solution. It is suggested that these DIBOs be built above the communication framework APIs provided by the Java DIME architecture, as shown in Figure 3.3. This ensures that there is maximum reuse of components.

Figure 3.4 gives an overview of possible DIBO APIs.

org.iso.mpeg.mpeg21.dibo

rdd−rel

UI did system dia . . . Frame Listener Query Choice Selection Message Prompt Content Condition Choice Element Component Resource Item Statement . . Textbox . . File−IO Device Codec . . . (all rel verbs) Grant Permission Condition Principle Resource . Adapt Transcode Terminal QoS .

Figure 3.4: Example DIBO APIs.

3.3

Prototype implementation

In this section we give an account of the prototype implementation to test and validate the design presented in this chapter.

3.3.1 Implementation details

(47)

• An adaptation of the J engine which includes an implementation of the MPEG-J classloader and a subset of the MPEG-Java platform API. This constitutes the DIME.

• An XML parsing utility as the core implementation of an aggregate DID-DII engine.

• A set of Java DIM functions that allow the manipulation of Digital Items contain-ing a library of media items. The DIMs implemented are AddTracks, DeleteTracks, ListTracks and GetTracks. These DIM functions are embedded in a Media library DI.

• A subset of DIBO functions required to implement the DIM functions.

• An application that provides the driver code for invoking the DIM functions. This application was capable of invoking the GetTracks DIM to get the location of a track and play the track.

3.3.2 Working and Validation

The prototype allows a user to specify a location from where to download a DI. The DID-DII engine parses and validates the DI. DIM item parsing is also implemented as a part of the DID-DII engine. DIMs packaged in the DI are collected and cached within the classloader. The application frontend takes user input to invoke various DIMs. Each of these DIMs is then loaded by the classloader and executed. These DIMs invoke DIBO functions internally.

(48)

Chapter 4

MPEG-21 Architectural Changes

After the first solution was proposed, a number of design changes were made in the MPEG-21 architecture and the MPEG-21 DIP architecture. This led to the formulation of new requirements. These architecture and requirement changes are documented in this chapter. A solution based on the new architecture and requirements is presented in the next chapter.

DIxO DIxO

DIM

DIBO DIBO DIBO DIBO

Multimedia Middleware API (M3W API)

Part of Platform

Packaged in DI.

Direct call from DIM Indirect call from DIM

(49)

4.1

Overview of Architectural Changes

A new addition to the MPEG-21 architecture is the introduction of an API layer called the Multimedia Middleware API (M3W API). This is a low level API that will presumably have detailed control over various parts of the system, device and the media encoding/decoding engines. The M3W API is not a part of any one subsystem. It has been introduced to provide a standardized view of the low level controls available on the platform. It will be used to implement some of the higher level APIs on the MPEG-21 platform. For instance, in the MPEG-21 DIP architecture, the DIBO APIs will be implemented using the M3W APIs.

New to the MPEG-21 DIP architecture is a mechanism called a Digital Item Ex-tension Operation or DIxO. DIxOs are executable programs, packaged along with the DI, capable of implementing complex functionality using the DIBO and the M3W APIs.

These changes have caused a paradigm shift in the DIP architecture. DIMs are to be viewed as simple scripts that implement operations using DIBOs and DIxOs as building blocks. Complex operations in the DIP architecture should be implemented by DIBOs and DIxOs. As a result,ECMAScript [4] - a scripting language - has been adopted as the DIML, while a more featured programming language like Java is suggested as a language to implement DIxOs.

We give some details on the specification of ECMAScript as the DIML and take a closer look at DIxOs and their expectations. We do not consider the M3W API in any of the discussions or designs below, since even basic requirements for the M3W API have not been established at this point.

4.1.1 DIM, DIML, DIME, DIBO

The entire ECMAScript standard, detailed in ECMA-62(edition 3) [4], is norma-tively specified to be the DIML. The ECMAScript specification provides for the following features:

• Lexical conventions and syntactic structure.

• Flow control structures (if/then, while, etc.)

(50)

• Composite data types (Objects and arrays)

• Standard operators (arithmetic, logical and bitwise)

• Exception handling

• Error definitions

• Support for regular expressions

Apart from these, the DIML is also specified to include a set of DIML specific object types. These object types are listed below with a brief explanation:

MpegDIPException This exception object is thrown whenever a DIME runtime error occurs.

MpegDIPObjectMap This represents the Object Map that was defined in Section 3.2.1.

MpegDIDDocument This object represents a DID instance document.

MpegDIDNode This represents a DIDL element node within the Digital Item document.

The exact syntax and semantics of ECMAScript-based DIMs have been estab-lished [30]. With respect to packaging DIMs, mechanisms have been specified to include ECMAScript DIM operations in the DI. Specifications of the DIME have been laid out so that all the requirements identified for DIME are supported. Finally, DIBO API semantics have been defined in terms of the DIML datatypes (primitive datatypes and DIML-specific object types) so that they can be invoked readily from DIMs.

4.1.2 Digital Item Extension Operations

(51)

The Figure 4.1 shows the relationship between DIMs, DIBOs and DIxOs. DIxOs cannot be invoked directly by the user. The user must invoke a DIM, which in turn may implement its complex functionality by invoking DIxOs. To implement its functionality, DIxOs are limited to invoking other DIBOs, DIxOs, and any standard APIs that may be a part of the DIxO execution environment. The MPEG-21 standard is open about which languages and how many languages are used to support DIxOs. However, proposals introducing new languages and mechanisms to support DIxOs should be presented to the MPEG-21 standard and deemed normative, before they can be used.

The Figure 4.1 shows DIxOs and DIBOs to be at the same level, since they are both considered to be operations used to implement DIMs. However, DIxOs are packaged and transferred along with the DI.

4.2

New Requirements

New requirements were born due to the changes in the DIP architecture. These requirements are informally listed here:

• DIxOs are meant to implement complex operations that are not supported by DIBOs. They may be used in situations when speed and response time are of importance. They should be treated as extensions to the base operations provided by a platform.

• The language used to implement DIxOs must support most features that are ex-pected from a high level programming language. It should have support for methods, primitive and user-defined data-types, flow control, exception handling, expressing concurrency and safety, to name a few.

• Mechanisms to package and distribute DIxOs in DIs should be compliant to MPEG-21 Part 2 (The DID specification).

• There should be adequate mechanisms to invoke DIxOs from DIMs and DIBOs from DIxOs.

(52)

Chapter 5

Solution 2: Java for DIxOs

This solution is proposed in response to the requirements for Digital Item Exten-sion Operations (DIxOs). The solution is an adaptation of the first solution and is also aimed at packaging Java code with Digital Items. The only difference between this solu-tion and the first is that the role of Java is slightly different here, since Java is proposed for DIxOs, which coexists with an ECMAScript-based DIM specification. The design presented in this chapter is also more detailed than the first. Rather than repeating arguments, ref-erences will be made, as far as possible, to aspects of the first solution that are also valid in this solution specification.

5.1

Overview

This solution proposes that Java should be used as a Digital Item Extension Lan-guage and the MPEG-J-based platform defined in the first solution (refer to Section 3.2.4) should be the engine used to execute Java DIxOs (or J-DIxOs). The motivation for using Java and the MPEG-J engine is unchanged since DIxOs are viewed as an additional way to package program code along with DIs.

(53)

specified manner, compiled, and packaged in a DIDL document. DIM code, written in ECMAScript, invokes the J-DIxO as required. J-DIxOs are viewed at the same level as DIBOs, i.e., as a building block to implement DIM functionality. The DIMs are processed by the DIME. When the DIME encounters a call to a J-DIxO, it passes control to the J-DIxO engine. The J-DIxO engine fetches the J-DIxO definition from its repository, executes the J-DIxO and returns control back to the DIME. J-DIxOs can in turn call DIBOs within their implementations. Exceptions thrown from the J-DIxO space are converted into exceptions within the ECMAScript space and delivered back to the caller DIM.

5.2

Design

There are two main objectives addressed by the design of this solution:

1. Designing a mechanism to package Java code with DIs and to execute this Java code as per requirements of the DIxOs and the DIxO engine.

2. Designing Java-based DIxOs in such a way that they interface with the ECMAScript DIMs and DIBOs.

In this section, we specify the process and rules to include J-DIxOs in a Processing Item, and we specify how J-DIxOs should be defined. Standard data structures are defined so that there is interoperability between J-DIxOs and ECMAScript DIMs. We propose the need for Java DIBOs (J-DIBOs), which are Java bindings for the DIBO library, and lay down rules to be followed to define new J-DIBOs. Finally we specify the internals of the J-DIxO execution engine.

5.2.1 Embedding a J-DIxO in a DIDL document

As before, we present a worked example on how J-DIxOs should be embedded within a DI. J-DIxOs should be embedded in a Processing Item along with other DIMs and possibly DIxOs in other languages.

(54)

...

47

52 </Statement>

53 </Descriptor>

Architecture for using Java bytecode for processing Digital Items

Contents

1.4 MPEG-J . . . 21

4.1.1 DIM, DIML, DIME, DIBO . . . 41

List of Figures

Chapter 1

Outline

The MPEG-21 Multimedia Framework

MPEG-J

Chapter 2

Chapter 3

Design

<DIDL> <ITEM id="ALBUM−BOO−FAR−DEBUT"> <COMPONENT id="TRACK−BOO−ROCK"> ... </COMPONENT> <COMPONENT id="TRACK−FAR−POP"> ... </COMPONENT> </ITEM> </DIDL>

API DIBO

org.iso.mpeg.mpeg21.dibo

Prototype implementation

Chapter 4

DIBO DIBO DIBO DIBO

Overview of Architectural Changes

4.1.1 DIM, DIML, DIME, DIBO

New Requirements

Chapter 5

Design