A Modular Data Pipelining Architecture (MDPA) for enabling Universal Accessibility in P2P Grids

A Modular Data Pipelining Architecture

(MDPA) for enabling Universal

Accessibility in P2P Grids

Sangmi Lee

Department of Computer Science

Florida State University

July. 03. 2003

11

Outline

Part I. Introductory Concepts and Motivation

Part II. Related Works

Part III. Enabling universal accessibility in Services

Part IV. Prototype of MDPA

Part I. Introductory Concepts and

Motivation

33

Some Observations for Distributed Services

• Grids manage and share asynchronous resources in a rather centralized fashion

• Peer-to-peer networks are “just like” Grids with different implementations of services like registration and look-up

• Computers are fast and getting faster. One can afford many strategies that used to be unrealistic

– All messages can be publish/subscribe – Software message routing

• Need Synchronous and Asynchronous Resource Sharing

– Can provide universal access using synchronous collaboration technology

• Web Services interact with messages

– Everything (including applications like PowerPoint will be a WS?) • XML will be used for most interesting data and meta-data

Classic Grid



P2P Grid

n Content_Access

P2P

P2P P2P

P2P P2P P2P P2P

Universal Accessibility for P2P Grid

•

Emerging Miniaturized Devices

•

Wireless Network Communication

(Wireless LAN(802.11b), 2.5,

3G wireless technology, PAN (Bluetooth), Satellite

Communication, etc.)

•

Widespread use of Mobile Devices – Affordable mobile devices

and services

•

Tendency of extending service for mobile devices.

– As a user: Mobile devices can be users of traditional distributed services – As a service provider: A mobile device can itself be a service provider

with its portability (e.g. weather or traffic sensor and send information with wireless network)

•

However problems pertaining to limited computing and

display capabilities in addition to unreliable network

Our Experiences I

•We’ve investigated on the Distributed Collaborative Services: Tango Interactive (NPAC, Syracuse Univ.) Garnet Collaborative Service (FSU)

•Garnet Collaborative Service

•Purpose : Support distance Education, Training and if possible Computing as Grid(Web) Services

•Integrate Synchronous and Asynchronous collaboration

•Support universal access including PDA’s collaboration with desktops •Uniform XML event (message) based architecture

•All data structures defined in an XML Schema called GXOS

•XML for all metadata (Users, documents, computers) and object changes -- from text chats to display changes etc.

•MyXoS manipulates GXOS objects

•We build on GMS/JMS (Java Message Service) as industry standard to implement publish/subscribe model

•Support collaborative features : basic interactive features (textchat, whiteboard, etc.), shared resources (shared display, shared export), AV conferences.

77

Our Experiences III: PDA Adaptor (Personal

Server)

99

Lessons from Earlier works

•

Adaptor architecture provided us an easy way to integrate

PDAs to existing distributed systems.

•

However, we faced the limitation of proxy architecture.

– Content adaptation concentrated on adaptor could cause inefficient network bandwidth usage.

– Proxy-based architecture for integrating network communication protocols could not ensure end-to-end service features of intelligent messaging frameworks.

– Optimization of individual messages could be an overhead on message delivery

Motivation

• A P2P Grid provides a natural environment for heterogeneous resource sharing

• A traditional 3-tier distributed service expects unified user devices with the infrastructure focused on back-end resources or the middleware messaging service.

• Early efforts

– Software architectures for user interface (Seeheim model, MVC, PAC, etc) which have been developed to support various advanced approaches in user interface design.

– Scientific Visualization, which maps raw data sets to a graphical form for

effective human visual senses. These efforts (IRIS, AVS…) were developed for a specific purpose and are not adaptable into a general purpose distributed service as it is.

 The idea of separating user presentation from raw data is still a major approach in distributed services such as the Web service infrastructure. • Therefore, the P2P Grid needs a software architecture, for supporting

heterogeneous devices, which clearly defines data processing within the services that it provides.

1111

Research Objectives

•

Investigating

a software architecture

which provides,

– Modular data processing

– Flexible Interoperability between object processing – User Interactivity

– Universal Accessibility

•

Developing a

prototype

based on this software architecture

•

Ability to apply this software architecture design to other

areas

Part II. Related Work

1313

Classical Approaches for Presentation Generation

• PAC-Amodeos model : Presentation-Abstraction-Control (PAC) model + Arch model( refined Seehiem model )

• MVC (Model-View-Controller) Architecture

• Advantage : Separating User Presentation from Raw data object 

Provides flexible data representation (multiple synchronized views on the same information)

• Open problems : Data processing is defined coarsely. As a consequence, the design of the adapting process in multiple service instances cannot be unified  cannot ensure interoperability between data processing

Proxy-based architectural approaches

•

Provides an interface to the service, which adapts mobile

devices to the conventional network communication

environments. (iMobile from AT&T, WBI project from IBM,

ICEBERG from Berkeley)

•

Advantages

– More manageable for a specific group of users – Easy to integrate new devices to legacy service.

•

Disadvantages

– Network communication overhead if the content adapting scheme is performed only in proxy.

– Hard to ensure end-to-end service for existing communication framework (security, fault-tolerance, etc)

1515

Standards for Heterogeneous Environments

•

W3C(XML based markup languages, CC/PP..) OASIS (Web

service description language), TEI, UnicodeConsotium, ANSI,

Dublin Core Metadata Initiate etc.

•

Advantage

– Interoperability between heterogeneous devices, applications and network environments.

•

Open Problems

– In some specific area, “Data Standard” does not have meaning of

standard because of too many standards—limited interoperability within the group using same metadata sets.

Content Adapting Technologies

•

Transcoding : Data process which mutates and

customizes an object based on the requesting device and

the user’s preferences

•

Web Clipping (Palm), Quality Aware Transcoding

(Duke Univ.) TranSqui (Univ. of Massachusetts), Web

Sphere Everyplace Access (IBM)

•

Transcoding technology can be a remote resource in

distribute services.

1717

Modular Data Pipelining Architecture (MDPA)

•

Basic Constraints for the software architecture for

enabling Universal Accessibility

– Modularity – Adaptability

– User Interactivity – Flexibility

– Reusability – Scalability

– Interoperability

1919

Modular design of Data Processing

• DAT (Data) stage: Data source (facing remote resources)

• CTL (Control Logic) stage: Semantic control of the data

• DTX (Data Transformer) stage: Process of filtering the object for heterogeneous devices

• PCL (Presentation client) stage: Generating abstract presentation for each client

Event Model in MDPA (I)

• MDPA is an event based software architecture

• Two directions of dataflow – Event transmission

– Presentation Generation

2121

EXT event

Event Model in MDPA (II)

• Different stages have different data processing

• User Event is a combination of multiple event types or single event type : e.g. loading new URL includes moving mouse to text window (MCL) + Typing text (MCL) + Loading new data (EXT)

• We define minimal unit of user event based on the pattern of event processing.

• We should show MDPA supports these event types successfully.

MCL event

PCL event

DTX event

CTL event

Architecture of Each Stage

• Interface: A set of ports facing other stages, users, or resources.

• Presentation Filters: Processes for generating user presentations.

• Event Filters : Processes for

transforming user event to deliver to next stage based on the original information

• Stage Manager : Basic concurrency management.

• Finally, Event type is decided by which stage executes its Event Processor.

2323

Modeling of MDPA

•

Using software architecture pattern of “

Pipes and

Filters

” : suitable for presenting data flow

– Each processing step is encapsulated in a filter component – Data is passed through pipes between adjacent filters.

– Recombining these filters allows us to build families of related filters.

•

Representation of data process in MDPA : Modular

PT-nets (Place and Transitions petri-net)

– The main uses of Petri nets are for discrete event simulation and modeling complex system such as communication protocols or distributed databases.

Modular PT-net

• Original PT-net includes four components: a set of place P, a set of transitions T, a weight function W, and a mapping function for the initial marking Mo.

• Modular PT-net : sharing Place or Transition, so, easy to

represent Modular approach for big systems [Christensen+00]. • Each module should be able to

generate output independently. • Each stage of MDPA is

represented as a Module and sharing transitions (interface between stages)

Transition T

Place A Place B

Place A Fusing Transition T Place B

Two modules sharing one transition

Module A Module B

A Service with MDPA

• A Service : A set of processes from capturing user event to delivering user presentation

• Defining of a Service within MDPA following the Definition of Modular PT-net

– Define Stage as a Module of Modular PT-net – There is no fusing Place

– Define Interface as a fusing Transitions

Definition of a Service

Define a Service architecture following Definition of Modular PT-net

Def. 4.7 ~ 4.11

2727

Representation of Stage with Modular PT-net

Presentation

Filters

Event

_Filters

Event

Stage

Fusing

transitions

: Interface

Tools for Modeling and Analysis

•Demonstration : Visual Object net++

Technical University of Ilmenau, Germany

http://www.systemtechnik.tu-ilmenau.de/~drath/visual_E.htm

•Analysis : Tina Ver. 2.5.1

LAAS ( Laboratory for Analysis and Architecture of Systems), France

http://www.laas.fr/tina/announce-2.5.1.html

2929

Analysis of Modeling MDPA

• Specification : Specify every path for every event types (Table 4.2) 

specify every paths inside of stage for each event

• Implementation : Defining a Modular PT-net (Def.4.7 ~ 4.12) • How to verify : Specification .iff. Implementation ?

– We analyze our implementation with PT-net tools.

– Show the specification is satisfied with our PT-net implementation (Is each path strongly connected?)

• SCC (Strongly Connected Components) : Two nodes are strongly connected, iff, there exists a finite directed path, which starts at a source node and ends in a destination node.

• Finally, each path specified in the stage (module) is strongly connected.

• Obviously, since we used fusing transitions for representing interfaces, each interface is strongly connected.

Aggregation of Services

•

Integrating services and enabling multiple service

instances for user

– Interface facing each data pipeline

– Shared process of presentation generation

– Independent event transmission to each data pipeline – Unified MCL stage for a single device

Service Instance A

Service Instance B

Aggregated Service

3131

Collaborative service with MDPA

•

MDPA considers the interoperability between users in

P2P Grid environments.

•

Asynchronous collaboration

•

Synchronous collaboration (real-time collaboration, 10 ~

1000 msec.)

– Shared Display : Sharing image data in the master’s framebuffer – Shared Input Port : Sharing user input event and generating

individual display with the full set of data processing.

– Shared Output Port : Sharing user output, more flexible than shared display because it is customizable.

•

The collaboration model within MDPA aims to support a

flexible collaborative environment for heterogeneous

Shared Display model

•Immediate sharing of the bitmap image data from the master user’s framebuffer memory

•Basic filtering (compression data, etc) is performed in the Messaging middleware

•Master/Participants share all stages except for the MCL stage •Therefore easy to implement. However, this is less flexible.

3333

Shared Output Port Model

•Sharing Output port to PCL stage

•Basic content adaptation is possible

Shared Input Port Model (CTL Event)

•Every user has full set of MDPA

•Only sharing user input event

•For each shared input event each service generates user

presentation

3535

Shared Input Port Model (DAT/EXT Event)

•DAT

event

PDAs(iPaq3900) Resolution:240 x 320 Wireless LAN 802.11b J2ME(Personal Java) Image size:553 x 465 50.9 KB Bitmap Image

Application –Aware Transcoding

for Shared Display

• Goal : Fast and Efficient

Propagation upon change of the master’s display.

• Approach: Reduces size of data over the wireless network

– Graphic Format Transformation for specific devices (Based on Java MDPA)

– Data Compression

– Color Depth Adjustment

– Image Resolution Adjustment – Aggregative Update

Desktop PC LAN/J2SE 1106 x 930 4017.9 KB Bitmap Image

SmartPhone(treo300) Resolution:160x160 3G wireless comm. J2ME(MIDP)

Image size:120x100 1.7 KB/PNG image

Filtering shared display message for wireless devices

3737

Application –Aware Transcoding

for Shared Export

• Goal : Support Flexible view points.

• Approach: Manipulate abstracted data and Reduce size of data

– Graphic Format Transformation for specific devices (Based on Java MDPA)

– Stylesheets (CSS) – Data Compression

– Image Resolution Adjustment Styling with CSS for Black/White PDAs

Workflow of Generating image for Shared Export (Collaborative SVG browser)

SVG

Document Rendering FormattingImage CompressionData

Part IV. Prototype of MDPA

3939

Challenge of the Web Service Architecture

• Provide clear framework with the representation of resources and well defined interfaces (ports) specified in WSDL

• Enable the development of collaborative services that can interoperate with existing services implemented in different ways (Perl, Phython, Java, etc.) • Support for unified user setup of collaborative environment for multiple

services.

• Easy to reuse/discover for integrated collaborative features.

Developing MDPA Service as a Web Service

• Each service is encapsulated as WS semantics

• Interfaces between services, users and resources are

defined clearly in WSDL

• Each WS is accessible to other WS via a resource facing port. • Multiple services are

integrated into a Portlet aggregation environment (Jetspeed, WebSpheres)

• Accessible to other resources (including DB, file systems, URL, 3rd _{party services, A/V}

conference services, etc.) via resource faced “ports”.

• Flexible packaging method is possible WSDL Web Service Collaborative Application W RS P

Collaborative Applications as Web Services: defined with WSDL

User Face Web Service WSRP Ports : define WS as a Portlet

Packaging and Structuring the service

AT TLC TXD

Remote resource s

Example 1. Put it all together in one WS WSDL WS W RS P WSDL WS W RS

P _TLC

D AT WSDL WS W RS P D TX Remote resource

s Example 2. One Stage in One_WS

DTX WSDL WS W RS P WSDL WS W RS

P _TLC

D

AT WS WS WS

Carousel Web Service

•

Prototype of Real-time collaborative service with MDPA

.

•

DAT+CTL+DTX

is designed as a Web Service

(example.1)

•

PTL

is implemented in two phases

– Markup Languages (HTML/WML)

– Layout of Client application (J2SE AWT APIs/J2ME MIDP)

•

MCL

is supported in various ways (Traditional PC

monitor(32bit true color)/ iPaq(320X240 12bit color)/

Treo300(160x160 8bit color)) based on J2ME technology

•

Real-time collaboration requires

efficient and reliable

messaging framework

4343

Distributed Event Service :NaradaBrokering

• Based on a network of cooperating broker nodes

– Cluster based architecture allows system to scale to arbitrary size

• Originally designed to provide uniform software multicast to support real-time collaboration linked to publish-subscribe for asynchronous systems.

• Now has four major core functions

– Message transport (based on performance measurement) in heterogeneous multi-link fashion

– General publish-subscribe including JMS & JXTA and support for RTP-based audio/video conferencing

– Filtering for heterogeneous clients

Event Service Portal

Aggregator

Collaborative WebServiceCollaborative_WebServiceCollaborative

WebService_{Web Services}Collaborative

Linkage with Event Service

Remote Collaborative

Web Services : integrated As portlets in Portal

Aggregator

Real-time event is processed By the event service

(NaradaBrokering)

: text message/framebuffer image/URLs/Microsoft events, etc.

Heterogeneous Users

4545

Portal Aggregator

Session Manager Collaborative

Web Services

Architecture of

CAROUSEL Web service

Event Service

Collaborative Web Services Collaborative

Web Services

Portlets

HTT _P HTT P

NB

publish/subscribe

NB

Scenario

Collaborative WS user

Session Manager Aggregative Portal

1. Login 2. Setup

3. User setup input 4. User preference/_{service request}

5. Service Topics / Available NB

Information 6. Service

Invocation 7. Service Topics

NB information

9. Service Output 8. User Event

NaradaBrokering

4747

Security Framework

•

Required in both phases of network communication

•

Within Traditional WS Architecture

– (Web Service Portal, Web Service  Web Service) – Popular Security Solution is possible (SSL, SOAP Security

Extentions (W3C), S-HTTP, etc)

•

Within Real-time messaging

– Inherit Security Framework of NaradaBorkering – Topic based encryption of message exchanges

– Key Management Center (KMC) manages key distribution – Ensures end-to-end message security

Reliability and Fault Tolerance

•

Reliability and Fault Tolerance also should be

considered in Two Phase of network communications

•

Within the traditional WS Architecture

– Portal service for Mobile Device is stateless.

•

Within the Real-time Messaging Service

– Inherit reliability and fault tolerance scheme from high-level messaging such as JMS scheme (NaradaBrokering is JMS compliant)

– Access point for the mobile device should provide reliability. Application layer protocol can ensure the reliability (e.g.

message acknowledgements, status of network communication links, etc)

4949

W Displa y W Viewe r WS Displa y WS Viewe r Master W Displa y WS Viewe r

Collaboration as a W Set up Session

We Servic e F I U O F I R O Othe Participants We Servic e F I U O F I R O We Servic e F I U O F I R O

Event

Service

Select Collaborative SVG Portlet : HTML

Select Collaborative SVG Portlet for Desktop environment

5151

Select Collaborative SVG Portlet for PDA environment

Setup the collaborative environment

Setup Operating System

Setup User Device Resolution Setup Types of Display

Setup from PDA

5353

Collaborative Browser for Users

Input URL of SVG document

Browse

Ready-to-use image from SVG content server

Part V. Contributions and Future Work

5555

Conclusions

• This dissertation presented a modular architecture for designing services in the context of the P2P Grid.

• Refined the data process as a set of modular stages.

• Categorized the user event based on the definition of stages.

• Showing the software architecture based on the Filters and Pipes

pattern, and modeled with Modular PT-net theory

• Modeling of aggregation/ collaboration for services with MDPA • Presented how the architecture is deployed with a Web service

infrastructure

• Implemented the prototype of a collaborative SVG Web service for heterogeneous environments

Contributions

• Service Design: MDPA provides a software architecture that incorporates heterogeneous user device profiles into its main service design phase. This enables the P2P Grid service designed using MDPA to support diverse user devices while ensuring maximum quality-of-service to each device.

• Refinement of Data Processing : MDPA modularizes data processing, and defines each process with distinguishable characteristics.

• User Interactive Event Model : Based on the definition of Data Processing, MDPA provides a user event model

• Interoperable Data Processing Architecture : Multiple sets of data processing can interoperate with each other.

• Collaboration: A modeling of collaboration in different degree of flexibility • An adaptable system to support Universal Accessibility : MDPA provides

an environment for developing universally accessible software with its well defined data processing stages and schemes of interoperating with other service instances.

• Prototype with a demonstrative application : Carousel Web service

5757

Future Work

• User Interface : Flexible

MDPA compliant data

object

• Intelligent Messaging

middleware: special

features for mobile

devices

• Deployment with OGSI

• Extend the adaptability to

Thank you!

Demonstration and Source Code is downloadable from

http://grids.ucs.indiana.edu/ptliupages/projects/carousel/

5959

Future Work 1. User Interface

•

For mobile devices, network/computing capability is

improving at a rapid rate.

•

User display has the obvious size limit for miniaturized

mobile devices.

•

Consider flexible accessibility to data object.

Future Work 2. Intelligent Messaging Middleware

•

Accessible from heterogeneous network communication

environments seamlessly.

•

Should support basic data compression, security,

reliability, and fault-tolerance.

•

Should ensure end-to-end service feature for mobile

devices.

•

Extend the concept of Web service infrastructure –

Service discovery/aggregation should be possible in

messaging middleware as well.

•

Provide Reliability/Fault Tolerance for mobile devices

6161

Future Work 3. Deployment with OGSI

•

OGSI (Open Grid Service Infrastructure) provides an

environment to integrate key Grid technologies with WS

mechanisms to create a distributed system framework

•

Since OGSI inherits WS, we expect MDPA to be

compliant with OGSI.

•

More refined Port type (extended WSDL) can provide

more intelligent data processing for P2P Grid service

•

Stateful WS concept of OGSI can support easier lifetime

Future Work 4. Wearable Devices

• We can extend the adaptability issue to wearable device and more advanced wireless

network communications (PAN, satellite)

• As devices get more powerful and specialized, a more

integrated and role-based design is required.

6363

Example: Shared SVG Web Service

•

SVG (Scalable Vector Graphics) is a 2D vector graphics

standard format from W3C with a structured XML

syntax

•

DOM based structure, so clearly represents object

instance in each stage of process

•

Prototype of Shared Input port model

– Every user has individual service instance.

– Collaborative/Non-collaborative user event is processed

– User preferences and device profile is defined in setup session.

Defining the Interface

• Dataflow in MDPA is performed via communication with interfaces including input and output ports

• In Modular PT-net (shared transition), input/output ports are represented as fusing transitions.

6565

Use Profile Application o Content source WSD L Web Service F I U O F I R O Render Porta (Aggregator ) Selecto r Filte r Control Channel Customized View Selectio View Control Channel Customized View

• Customized User-Facing Port -setup for device and user preference

Filtering data for PDAs and Smartphones with different message topic

(NaradaBrokering Event Service

Customizing User Facing Port

•

Based on General Purpose Web service aggregator

(Apache’s Jetspeed)

•

Supporting HTML/WML

•

Customizable Layout of portal (integrating

environment)

•

Provide environment for selecting preferences, hardware

capabilities, network service, etc..

•

Deliver the information to session manager

6767

Filtering in Event Service

• Non-collaborative/Collaborative event filtering

• Collaborative Event is processed depending on the type of devices • Master user/Participants filtering

 Filtering by the topics of

Publish/Subscribe messaging system (NaradaBrokering)

Service output for PC Service output for iPaq Service output for regular

wireless smartphone