1
Clouds and Web2.
II: Case Study and Tutorial
CTS08 Tutorial
Hyatt Regency Irvine California
May 19 2008
Geoffrey Fox, Marlon Pierce
Community Grids Laboratory, School of informatics
Indiana University
http://w
ww.infomall.org/multicore
Polar Grid: A Web 2.0 Case Study
Using Microformats, GeoRSS, REST,
and KML
More Detailed Information
•
See Marie’s blog:
http://tethealla.blogspot.com/
PolarGrid Datasets
•
A typical processed dataset consists of:
–
One plain text header file describing the overall
system and metadata of the particular chunk of
measurements.
–
Multiple JPG images processed from the
corresponding raw data.
–
Collection timestamp and fundamental properties
such as waveform and transmitting (TX) /receiving
(RX) antenna IDs are embedded in file names.
Approaches
•
Convert unstructured metadata description
into microformats.
•
Construct a relational database to enable fast
and dynamic queries.
•
Provide geographic annotation embedded into
web feeds that can be plotted by Google Maps
and Google Earth
•
Technologies:
Microformat Conversion
•
Microformats capture “semantic” metadata
information in XHTML format.
•
Principals for converted microformats:
–
Simple: capture the most basic information unit.
–
Reusable: capture shared features among data.
–
Composable: when combined, can reflect more complex
relationships among data.
•
Most of our microformat classes are prefixed with
“pg:” denoting the PolarGrid project namespace,
except when existing generic ones such as the
geo
How To Extend XHTML
•
Use either <div> or <span> tags.
•
Display of these tags can be controlled with
style sheets
•
You can also dynamically control these with
JavaScript.
Simple Example: Geo Format
<span class="geo">
<span
Polar Grid Microformat Examples
•
Simple descriptions:
<span class="pg:radar-type">MCRDS</span>
•
Measurements with unit:
<span class="pg:delay">
Microformat Examples
•
Identification for cross-references:
<span class="pg:waveform”>
<span class="pg:id">1</span>
</span>
•
Commonly shared features: (e.g. both TX and RX
antennas record same kind of information)
<span class="pg:antenna">
<span class="pg:id">0</span>
<span
class="pg:type">TX/RX</span>
<span
Microformat Examples
•
Structured collections/Arrays
:
<span class="pg:antenna-array">
<span
class="pg:array-size">2</span>
<span class="pg:antenna”>
<span class="pg:id">0</span>
<span class="pg:type">TX</span>
<span class="pg:attenuation">0</span>
</span>
<span
class="pg:antenna”>
<span class="pg:id">1</span>
<span
class="pg:type">TX</span>
<span
Microformat Examples
•
Complex objects: (e.g. spatial and temporal
information that uniquely defines a processed
data chunk)
<span class="pg:data-chunk">
<span
class="pg:name">Start/Stop</span>
<span
class="pg:utc-timestamp">1202755351.892651</span>
<span class="geo">
<span class="longitude">-2.193</span>
<span
class="latitude">52.686</span>
</span>
GeoRSS-Simple Embedded Atom Feed
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"
xmlns:georss="http://www.georss.org/georss">
<title>PolarGrid Expedition</title>
<id>tag:pg3.ucs.indiana.edu,2008:polargrid-expedition-2008-05</id>
<updated>2008-05-11T02:02:54Z</updated>
<author>
<name>Polar Grid</name>
</author>
<link type="application/atom+xml" rel="self"
GeoRSS-Simple Embedded Atom Feed
<entry>
<title>Data Chunk: 2008-03-11 11:55:41</title>
<id>tag:pg3.ucs.indiana.edu,2008:polargrid
expedition-2008-05-262</id>
<updated>2008-03-11T11:55:41Z</updated>
<content type="html">
… (data chunk information described in microformats) …
</content>
<georss:point>77.585957 -52.448461</georss:point>
<georss:point>77.580976 -52.435148</georss:point>
</entry>
Corresponding KML
<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.2"> <Document>
<name>PolarGrid Expedition</name>
<Placemark id="tag:pg3.ucs.indiana.edu,2008:polargrid-expedition-2008-05-262”>
<name>Data Chunk: 2008-03-11 11:55:41</name> <description><![CDATA[
… (data chunk information described in microformats) …
]]></description>
<Point><coordinates>-52.448461,77.585957</coordinates></Point> <Point><coordinates>-52.435148,77.580976</coordinates></Point> </Placemark>
Web 2.0 Lessons
•
Web 2.0 approaches can be applied to
management of scientific information.
•
With the right choices, you get a lot for free.
•
Microformats: simple ways to encode
name-value pairs, build up semantic descriptions.
•
GeoRSS: orders data by both time and space
Browser +
JavaScript Libraries JavaScript LibrariesBrowser + JavaScript LibrariesBrowser +
Blogs, Calendars,
Docs, etc Social GadgetContainers
Gadgets, Gadget Aggregators
Facebook Facebook Apps Server-Side
Gdata Apps
User Layer
System Cloud Layer
AJAX, JSON, REST, RSS
Map Key
•
Red blocks represent browsers and things that run in them
(JavaScript).
– This is the “user” level. – Client side mashups
•
Green blocks represent Web servers and their applications.
– This is the “developer” level. – Server-side mashups.
– These can run on any hosting environment: your web server, Amazon EC2, Google GAE, etc.
•
Blue blocks represent third party services.
– This is the “system cloud” layer.
•
Arrows represent network communications.
– Everything goes over HTTP
– REST, AJAX: communication patterns.
Using Google’s GData API
GData: the API to Google Services
•
Google provides an extensive set of online services with well-known
user interfaces.
– Docs – Spreadsheet – YouTube – Blogger/Blogspot – Calendar– Picasso Photo Sharing
– GMail
•
These have
programming
interfaces as well as
user
interfaces.
– Server side: Java, .NET, PHP, Python – Client side: JavaScript
•
Client interfaces include visual components suitable for mash-ups
Server-Side Mash-Up Use Case
•
Imagine you have a Web service that runs a large parallel finite
element code.
– Output can include images, movies, output files, metadata, etc. – Your web service manages the interaction with the queuing system
(another topic…)
•
With a server-side mash-up, your Web Service can
– Post status information about your jobs (“Step 521 Completed!”) on your blog.
– Post URLs pointing to your output files on your blog.
– Upload your visualizations to Picasso and YouTube (and post links). – Generate RSS/Atom feeds of the above.
– Post results to Google Calendar
Getting Started with GData
• Create a Blog, make sure you can login to YouTube, etc.
• You will need a clientID and associated developer key.
– Get these from http://code.google.com/apis/base/signup.html.
• Get the Java code
– http://code.google.com/p/gdata-java-client/downloads/list
• See any number of Google examples
– http://code.google.com/apis/base/javadevguide.html
• One trick: You can get your blogger ID by examining the your Blog’s Atom feed.
– It will be in the header and look something like this: <id>tag:blogger.com,1999:blog-19457310</id>
– The blog ID to use in the code below would be 19457310 in this example.
• The full working example is here
Posting Results to Your Blogspot Blog
String userName="[email protected]"; String password= "qwerty";
String content=“…”;
GoogleService myService=
new GoogleService("blogger","");
myService.setUserCredentials(userName,password);
Entry myEntry = new Entry();
myEntry.setTitle(new PlainTextConstruct(title));
myEntry.setContent(new PlainTextConstruct(content));
URL postUrl = new URL("http://www.blogger.com/feeds/" + blogID + "/posts/default");
myService.insert(postUrl, myEntry);
Create the entry to insert Login in
Replace these with your values
Getting a YouTube Feed
String feedUrl="http://gdata.youtube.com/feeds/api/videos/"; String
scarFace5sec="http://gdata.youtube.com/feeds/api/videos/egwB7hVIIEc";
YouTubeService service = new YouTubeService(clientID, developer_key); service.setUserCredentials(userName,password);
VideoFeed videoFeed =
youTubeService.getFeed(new URL(feedUrl),VideoFeed.class);
VideoEntry videoEntry = youTubeService.getEntry(new URL(scarFace5sec), VideoEntry.class);
//Loop over feed entries and display. String feedContent=
"Title: " + videoEntry.getTitle().getPlainText()+”"; feedContent+=videoEntry.getMediaGroup().
Posting To Google Calendar
CalendarService calService=new CalendarService(clientID); calService.setUserCredentials(userName,password);
URL postUrl =
new URL("http://www.google.com/calendar/feeds/"+userName+"/private/full"); CalendarEventEntry myEntry = new CalendarEventEntry();
myEntry.setTitle(new PlainTextConstruct(titleOfEvent));
myEntry.setContent(new PlainTextConstruct(contentToPost)); DateTime startTime = new DateTime(new Date());
When eventTimes = new When(); eventTimes.setStartTime(startTime); myEntry.addTime(eventTimes);
CalendarEventEntry insertedEntry = calendarService.insert(postUrl, myEntry); CalendarEventEntry myEntry2 = new CalendarEventEntry();
String now=(new Date()).toString();
myEntry2.setContent(new PlainTextConstruct("Test post at "+now)); myEntry2.setQuickAdd(true);
// Send the request and receive the response:
CalendarEventEntry insertedEntry2 =calendarService.insert(postUrl, myEntry2);
Create the event time and insert Create a new Calendar entry Log in to the calendar service
Some Other Things You Can Do with
GData Server Code
•
YouTube
–
Search movies by keyword tag, “related”, categories
–
Upload and download movies.
–
Put movies into collections
•
Calendar
–
Retrieve events as RSS feed
•
Spreadsheet
–
Remotely retrieve and insert or change row and cell data.
–
Use structured queries to retrieve data ranges.
Making FaceBook Applications
Full Examples and More Detailed
Notes
•
PHP and JavaScript notes
–
http://communitygrids.blogspot.com/2008/02/qu
ick-facebook-javascript-api-notes.html
•
JavaScript example:
Why Use FaceBook as a Portal?
•
Because it has 10’s of millions of users.
–
The largest Web 2.0-style Web portal
•
Easy to create Social Networks: groups of
friends and share applications, communicate,
share information, etc.
–
And more importantly, tools for building and
managing these networks
•
Can leverage many third party applications
–
For example, photo albums
FaceBook APIs
•
These come in two flavors
–
Embedded: make your application available through
FaceBook.
• PHP APIs
–
Embedding: use Facebook authentication and social
network data in your application.
• JavaScript API
•
In both cases, you run your application on your Web
server.
–
You application do anything you can implement.
–
And you can query FaceBook for social network
Using The JavaScript API
•
Getting Started:
– You just need a Web server to host your application. – Register as a developer
– Place Facebook’s xd_receiver.htm on your web server
• Make sure the relative path used is correct.
•
What will happen?
– Your page will run outside of Facebook.
– Anyone loading your application will be directed to Facebook to login. – User is then redirected back to your page.
•
Your application can make calls back to Facebook.
•
And you can do anything else you want.
– So you could embed this application in a portlet, interact with a local database,
HTML Page Example
<textarea style="width:500px;height:300px;"
id="_traceTextBox">
</textarea>
<script
src="http://static.ak.facebook.com/js/api_lib/Fac
ebookApi.debug.js" type="text/javascript">
var
api
= new FB.ApiClient('<your_key_here>',
'/xd_receiver.htm', null);
…
Interact with FaceBook
api
.requireLogin(function(exception) {
api
.friends_get(null,
getResults
);
var myinfo= 'last_name,
first_name,
hometown_location,
work_history,
pic_small';
api
.users_getInfo(
api
.get_session().uid,
myinfo
,
Notes on Previous Slides
•
The requireLogin, friends_get, and
users_getInfo are FaceBook API calls.
•
The getResults and getInfo methods are
callback methods that I defined.
–
Shown in next slide
•
The myinfo data structure is taken from the
Facebook XML user profile.
Developer Defined Call-Back Functions
function getResults(result,exception) {
Debug.dump(api.get_session().uid,’You');
Debug.dump(result,’Your Friends’);
}
function getInfo(result,exception) {
Debug.dump(result,’Your info');
}
These two methods receive the output returned from
Output Info Dumped to Text Area
you: 627774031 Your friends: {Array}
[Deleted]
Your Info: {Array} [0]: {Object} first_name: Marlon hometown_location: {Object} city: state: country: zip: last_name: Pierce pic_small: http://profile.ak.facebook.com/profile6/1797/98/t627774031_2463.jpg uid: 627774031 work_history: {Object}
User’s Facebook ID and the ID numbers of his friends
Facebooks’ Query Language
•
All of the FaceBook API is basically a set of
wrappers around their SQL like FQL query
language. So you can, if you prefer, make
custom methods out of FQL query strings.
var myQuery='SELECT name FROM user WHERE
uid='+api.get_session().uid;
api.fql_query(myQuery, getFQLResponse);
Working with Facebook Groups
•
To list information about groups you are a
member of, use the following.
–
You can replace the value of
uid
with any UID you
know.
–
18629081888 is the Group ID for the Open Grid
Forum’s uninteresting Facebook group
api.groups_get(api.get_session().uid,null,getGroups);
More Information
•
The JavaScript documentation is available
from here
–
h
ttp://wiki.developers.facebook.com/index.php/J
avaScript_Client_Library
•
It
is pretty minimal, but it is essentially the
same as the better documented PHP API.
–
Replace PHP’s "." with a "_".
What Is a Gadget?
Gadget Definition
•
This XML page is web accessible via
http://156.56.104.143:8080/GTLAB/MyProxyGadget.xml
•
T
his URL points to the actual application (written in JSF)
•
Google uses an HTML IFrame to load the page.
•
You can also use
content type=“html”
for more integrated
applications.
–
For example, you can send messages between gadgets in the
client.
<?xml version="1.0" encoding="UTF-8" ?> <Module>
<ModulePrefs title="MyProxy Gadget Example" scrolling="true" height="500"/>
<Content type="url"
href="http://156.56.104.143:8080/GTLAB/examples/MyProxyExample.jsf"> </Content>
Gadgets and Open Social
•
Open Social is a consortium of Facebook’s major
competitors.
–
Orkut, MySpace, Hi5, LinkedIn, Friendster, etc.
•
The Open Social API has two parts
–
JavaScript libraries for building social gadgets.
• Must run in a gadget container (server)
–
REST APIs for exchanging data between container servers.
•
The ideas:
–
JS APIs let you make portable gadgets that will work in
different containers (Hi5
LinkedIn) but data will be
different.
Getting Started
•
You need a Gadget container to host your gadgets.
–
LinkedIn is notably selective about the gadgets it will host.
•
Orkut provides a sandbox, but you have to upload your
gadgets.
•
Apache Shindig is an extremely simple Java-based
container.
–
De facto reference implementation of Open Social
–
Not suitable for production by a long ways.
–
But it will work on an air plane.
Getting Started with Shindig
• Visit the Web site for information
– http://incubator.apache.org/shindig
• Check out the code with SVN
– mkdir $HOME/shindig; cd shindig
– svn co http://svn.apache.org/repos/asf/incubator/shindig/trunk/
• Build and run it with Apache Maven
– Run “mvn” from $HOME/shindig/
– Run “mvn -Prun” from $HOME/shindig/java/server
• Point browser to
http://localhost:8080/gadgets/files/container/sample1.html and look
through other samples.
– You can also load these HTML files using file:// if you don't want to run the Jetty server.
• Periodically check for updates and rebuild
– svn update
What Do You Get?
•
Shindig actually has two major parts:
–
A container that can run social gadgets
•
This must have access to your user and social network
database.
–
A gadget aggregator that displays gadgets
•
Runs the layout manager.
•
Think of this as your own iGoogle server.
•
Access the gadget container
The gadget can be displayed
in your
http://localhost:8080
display container.
<link rel="stylesheet" href="gadgets.css">
<script type="text/javascript"
src="../../js/rpc.js?c=1&debug=1"></script>
<script type="text/javascript" src="cookies.js"></script>
<script type="text/javascript" src="util.js"></script>
<script type="text/javascript" src="gadgets.js"></script>
<script type="text/javascript"
src="cookiebaseduserprefstore.js"></script>
var my = {};
my.gadgetSpecUrls = [
'http://www.google.com/ig/modules/horoscope.xml', 'http://www.google.com/ig/modules/aue07otr.xml', 'http://www.labpixies.com/campaigns/todo/todo.xml' ];
my.LayoutManager = function() { gadgets.LayoutManager.call(this); };
my.LayoutManager.inherits(gadgets.LayoutManager);
my.LayoutManager.prototype.getGadgetChrome = function(gadget) { var chromeId = 'gadget-chrome-' + gadget.id;
return chromeId ? document.getElementById(chromeId) : null; };
Create Your Layout Manager
my.init = function() {
gadgets.container.layoutManager = new
my.LayoutManager();
};
my.renderGadgets = function() {
for (var i = 0; i < my.gadgetSpecUrls.length; ++i) {
var gadget = gadgets.container.createGadget(
{specUrl: my.gadgetSpecUrls[i]});
gadgets.container.addGadget(gadget);
gadgets.container.renderGadget(gadget);
}
};
<body onLoad="my.init();my.renderGadgets()">
<h2>Sample: Dynamic Height</h2>
<div id="gadget-chrome-0"
class="gadgets-gadget-chrome"></div>
<div id="gadget-chrome-1"
class="gadgets-gadget-chrome"></div>
<div id="gadget-chrome-2"
class="gadgets-gadget-chrome"></div>
</body>
HTML Display
Hosting Your Own Social Gadget
•
Place your widget code here:
–
$HOME/shindig/javascript/samplecontainer/example
s
•
After compilation, these will be located here:
–
./target/gadgets/files/samplecontainer/examples.
•
These will be served up from
–
http:
//localhost:8080/gadgets/files/samplecontainer/
examples/myapp.xml
http://localhost:8080/gadgets/files/samplecontainer/samplecontainer.html
