Integration the Web 2.0 way. Florian Daniel April 28, 2009

(1)

Web Mashups

Integration the Web 2.0 way

(2)

What are we talking about?

• Mashup – possible defintions

Mashup possible defintions

• “...a mashup is a web application that combines data from

more than one source into a single integrated tool…” [wikipedia.com – March 24, 2009]

• “...you can integrate two or more […] Web APIs to create

something new and unique known as a mashup ” [*] something new and unique, known as a mashup… [ ]

• A mashup is a web application that is developed by

composing data, application logic, and/or user

co pos g data, app cat o

og c, a d/o use

interfaces originating from disparate web sources

• Similar terms: service mashups, data mashups

p ,

p

(3)

Mashup = integration the Web 2.0 way

Mashup integration the Web 2.0 way

• Young integration practice using the Web as platform

Young integration practice using the Web as platform

• Highly user‐driven

• Oftentimes the actual providers of content/functionalityOftentimes the actual providers of content/functionality are not even aware of being “wrapped”

• Google Maps example: initially skilled users hacked the

d f h li i ( lk b l ’

AJAX code of the application (talk by Google’s Lars Rasmussen at ICWE’05)

• Strong evolution: from hacking to first systematic

(4)

Let’s see an example

Let s see an example

• The HousingMaps application

The HousingMaps application

((http://www.housingmaps.comhttp://www.housingmaps.com))

composed of:

• Google Maps (http://maps.google.com)

• Craigslist (http://www.craigslist.com)

(5)

A mashup example

• HousingMaps (

GoogleMaps

http://www.housingmaps.com

)

Own application logic/UI

Craigslist

HousingMaps (

http://www.housingmaps.com

)

• http://maps.google.com • http://www.craigslist.com

(6)

Web 2.0

• Web 2.0? Again, there are lots of different (and

Web 2.0? Again, there are lots of different (and

sometimes diverging) definitions:

• “Web 2.0 is a term describing the trend in use of World

Wide Web technology and web design that aims to

enhance creativity, information sharing, and, most notably, collaboration among users ” [wikipedia com]

collaboration among users... [wikipedia.com] • “Web 2.0 is best described as a core set of patterns that are observable in applications that share the Web 2.0 label. These patterns are services, simplicity, and community…” [*] * http://www.ibm.com/developerworks/webservices/library/ws‐soa‐mashups/index.html?S_TACT=105AGX04&S_CMP=EDU

(7)

The enabling factor of Web 2.0

• Over the last years we have been witnessing two

Over the last years we have been witnessing two

main trends on the Web:

• User participation in the content creation process (e.g., communities, social networks, blogs...) • User participation in the development process (e g mashups) (e.g., mashups)

• Which are enabled or fostered by:

• Simplicity of usage: intuitive interactive applications • Simplicity of usage: intuitive, interactive applications

• Simplicity of development: novel and standardized web

(8)

Some figures

(programmableweb.com)

Some figures

(programmableweb.com)

• Most popular

categories of mashups

• Most popular

web APIs

(9)

Dynamics of the ecosystem

• Constant growth since programmableweb.com went

Constant growth since programmableweb.com went

online (over 600 days)

[by Michael Weiss, Carleton University]

(10)

Web engineering and mashups

• Evolution of web application development:

pp

p

• Manual development: static (plain HTML) and dynamic

(CGI, PHP, JSP,...) pages coded via simple text editors

Tool assisted d l t b d l t ifi

• Tool‐assisted development: web‐development‐specific tools (e.g., Dreamweaver) augment productivity by automatic web‐specific development concerns • Model‐driven development: graphical modeling tools enable developers to reason at a high level of abstraction and to “draw” an application and to automatically generate the code • Mashups: we are going toward the user‐driven development of web applicationsp pp

(11)

Developing a mashup: what does it mean?

• The mashup development scenario

Developing a mashup: what does it mean?

The mashup development scenario

develops

Component developer

publishes

chooses writes uses

Mashup user Mashup composer

discovers

and selects mashes up

Description Data sources Layouts Styles Architectures Protocols Languages Formats Mashup

component The Web

Technologies ... _Mashup

application Mashup tool or

(12)

Distribution of apps over C and S

(13)

Mashup component/API types

UI l i

S vice s nt vice s app

UI logic

App

ien

t

C/ S

ser _Clie _serv

d get s o nal We b a

App

Data

Cl

zat io n wi d idg ets vice s Con ven ti o

r

UI logic

No UI Visuali z o mple x w ent app s Ser ver ‐ Side ser v

App

D

Ser

ve

Co Clie Feeds

Data

C/S apps

(14)

The technological landscape

UI l i

The technological landscape

(D)HTML

UI logic

App

ien

t

(D)HTML Flash, Silverlight

App

Data

Cl

_AJAX JSON, XML Silverlight SOAP,

r

UI logic

SOAP, HTTP HTML, templates,...

App

D

Ser

ve

PHP, Ruby, Java, C++,... XML, Relational

Data

RSS, Atom DBs, OODBs,...

(15)

Dynamic HTML

• DHTML is a programming technique for dynamic and

DHTML is a programming technique for dynamic and

interactive Web (HTML) pages

• DHTML combines

• HTML >> document markup • JavaScript >> biz logic for interactive features • DOM (Document Object Model) >> standard access to HTML objects + HTML event model CSS (C d d St l Sh t ) >> t l d l t • CSS (Cascaded Style Sheets) >> style and layout

• Benefits

• Enhances user experience

(16)

AJAX (

Asynchronous JavaScript And XML)

AJAX (

Asynchronous JavaScript And XML)

• AJAX is a DHTML programming technique with

AJAX is a DHTML programming technique with

support for client‐server HTTP communication

• Ingredients

g

• DHTML >> UI management • XMLHttpRequest object >> communication support

• Enables asynchronous communications between

client and server >> RIA applications

• Benefits

• Enables state and UI management on the client • Distribution of application features Demo

(17)

Flash, JavaFX, Silverlight

• Flash, JavaFX, and Silverlight are light‐weight

Flash, JavaFX, and Silverlight are light weight

multimedia application platforms that run inside the

client browser

• Alternative approach to RIA applications:

• Require the installation of a dedicated browser plug‐in • Are based on simple scripting languages (JS‐like) • Break with the document‐centric approach of AJAX/DHTML

• Benefits

• Highly interactive and multimedia user interfaces featuring

desktop like user experiences desktop‐like user experiences

(18)

XML (eXtensible Markup Language)

• XML is a language for the definition of document

XML is a language for the definition of document

structures, i.e., other languages >> a meta‐language

• Ingredients: tags/elements, attributes, name spaces,

g

g /

,

p

,

DTDs, XSDs, XSLT, XSL‐FO,...

• Benefits

• Self‐describing structures >> easily human‐readable • Technology‐independent >> independent of programming language, platform, protocol, etc. • Highly portable

(19)

JSON (JavaScript Object Notation)

• JSON is a lightweight data interchange format

JSON is a lightweight data interchange format

• Text‐based, extensible data format for representing

• Collections of name/value pairsCollections of name/value pairs

• Ordered lists (e.g., associative arrays) • Basic data structures

• Benefits

• Easy parsing and handling of data structures • Supported by almost all programming languages • Less overhead (both computation and data) than XML

(20)

JSON: example

{ ”catalog”:{ The JSON version of the XML document in the ”cd”:[ { ”title”:”Empire Burlesque”, ”artist”:”Bob Dylan” XML document in the previous AJAX example

artist : Bob Dylan , ”country”:”USA”, ”company”:”Columbia”, ”price”:10.9, ” ” 198 Interpretation of ”year”:1985 }, … ] the data format in JavaScript (cf. XML example) ] } }

JSONobj = eval( "(" + http_request.responseText + ")" );

(21)

RSS and Atom

• RSS and Atom are Web feed formats (applications of

RSS and Atom are Web feed formats (applications of

XML) for the publication/syndication of frequently

updated contents over the Web

• Ingredients

• Stable structure and predefined meaning of tags • Dynamic behavior achieved through dedicated readers

• Benefits

• Simple, standardized exchange formats • Very popular in blogs, newspapers, social web sites,... Demo

(22)

PHP, Ruby,...

• Popular, simple server‐side scripting languages for

p

,

p

g

dynamic Web applications

• Benefits

• Free web server extensions (scripting engines) available for most web servers

• Full power of 3G programming languagesFull power of 3G programming languages

• Availability of powerful free code libraries (e.g., for the management of HTML templates: PHPTAL ) A il bilit f d d b d l t f k • Availability of advanced web development frameworks (e.g., Ruby on Rails with full support for MVC‐based web applications)

(23)

(24)

SOAP/WSDL web services

• Programming interfaces accessible over the Web

Programming interfaces accessible over the Web

• WSDL = Web Service Description Language • Abstract service description language (tech‐agnostic) • SOAP = Simple Object Access Protocol • XML message exchange protocol

SOA S

i

O i

t d A hit t

• SOA = Service‐Oriented Architecture

• Producer, consumer, registry (virtual marketplaces)

• Complex advanced features: security reliability

• Complex advanced features: security, reliability,

transactions, addressing,...

• Orchestration and choreography

(25)

RESTful web services

• A new architectural style of developing web services

A new architectural style of developing web services

• Principles

• Operations based on HTTP methods (Get, Post, Put, Delete)Operations based on HTTP methods (Get, Post, Put, Delete)

• Services are stateless (no session data at the server side) • Access via hierarchically structured URIs • XML or JSON over HTTP

• Benefits

• Simplicity and immediacy • No big overhead for composing and parsing messages • More efficient service implementations

(26)

(27)

Mashup development manually (1/2)

• Sceanrio 1 (at the beginning): No APIs available

Sceanrio 1 (at the beginning): No APIs available

• Developent tasks

• Read and interpret AJAX code of GMapsRead and interpret AJAX code of GMaps

• Hack into GMaps code to implement marker support • Extract data from Craigslist with regular expressions (write a wrapper) • Format extracted data and forward data to GMaps

bl

• Problems

• No stabel interfaces

• Highly error prone and time consuming

(28)

Mashup development manually (2/2)

• Scenario 2 (today): GMaps comes with AJAX API and

Scenario 2 (today): GMaps comes with AJAX API and

Craigslist provides an RSS feed

• Development tasks

p

• Instantiate GMaps component • Layout RSS feed • Set markers through GMaps API

• Problems

• Manual development for skilled programmers • Manual parsing of RSS feed b f • No common Web API format

(29)

Partially assisted development

• There are many (online) tools for

There are many (online) tools for

• Data extraction from Web pages

• Web content clipping

>> Aid the development of mashup components or APIs

(30)

Summary

• Mashups: a new way of data, application, and UI

p

y

, pp

,

integration on the Web

• An end‐user‐driven phenomenon (e.g., Google Maps)

• As for now, only few really useful applications exist

• Agreed on development frameworks and standards are still missing missing

• But there is growing interest in mashups also from

business users

• Big companies are investing large amounts of money in mashup technologies (e.g., Yahoo, Google, Microsoft, Intel, IBM) IBM)

(31)

Problems

• Web APIs often change interface/features over time

g

/

• There is no commitment to stability from providers

of clipped or extracted data

• Oftentimes they are not even aware that their contents are used in a mashup application

• Intellectual property rights and licences!Intellectual property rights and licences!

• Mashup development oftentimes is like hacking

>>

Mashup development is still a time consuming and

daunting task

(32)

Outlook

• The importance of mashups will undoubtedly be

p

y

growing over the next years in both the consumer

and the business context

h

d

l

(

d

)

• Mashup developers (even inexperienced end users)

need to be assisted in the development

• Simple models and languages are requiredSimple models and languages are required

• Dedicated tools are already being developed

• High‐quality public mashup components (i.e., APIs) are

paramount