Federated Web Map Cache Server:
A “Web 2.0 Meets SOA” Case Study
Zao Liu, Marlon Pierce, Geoffrey Fox Community Grids Laboratory
Indiana University
&
Neil Devadasan Polis Center
GIS Map Servers as a Web 2.0 Case Study
Ø
There are several different products for creating on-line maps
and allowing interaction with Geographical Information System
(GIS) data bases.
•
ESRI, Autodesk, Open Geospatial Consortium
•
These follow a classic user driven request/response style model.
Ø
An interesting application of Service Oriented Architecture
principles.
Ø
Google Maps (released in 2005).
•
Highly interactive AJAX style clients replaced stodgy user-driven
request/response.
•
See the Mo
nday tutorial for mo
re information.
§
http://grids.ucs.indiana.edu/ptliupages/presentations
Building a Hybrid System
Ø
Google Maps provide a highly interactive user interface and capabilities
(geolocations, directions)
Ø
But GIS services have much more detailed local information.
• Indiana has orthophotography with much higher zoom levels than Google maps.
§ http://www.indiana.edu/~gisdata/05orthos.html
• Local county servers have many interesting map layers not in Google
§ Parcels/property lines, school district lines
• And these tie into feature services with interesting data like pinpoint addresses, tax assessments, etc.
Ø
So obviously it makes sense to adopt the Google approach but enhance
it with local data.
Browser +
Google Map API
Cass County Map Server (OGC Web Map
Server) Hamilton County Map Server (AutoDesk) Marion County Map Server (ESRI ArcIMS)
Browser client fetches image tiles for the bounding box using Google Map API.
Cache Server
Tile Server
Adapter Adapter Adapter
Cache Server requests map tiles at all zoom levels with all layers. These are converted to uniform projection, indexed, and stored. Overlapping images are combined.
Must provide adapters for each
Map Server type . The tile server fulfillsGoogle map calls with cached tiles at the requested bounding box that fill the
Sample client runs in browser, uses
Google JavaScript API.
Map of Marion/Hancock boundary (slightly panned North from
previous slide) with IU orthophotography, parcel boundaries, and
Building a Cache Server
Federating GIS Servers Around Indiana
Ø
Indiana has 92 counties
•
Approximately 15 have public GIS map servers.
Ø
Examples
• ESRI ArcIMS and ArcMap Server
§ Marion, Vanderburgh, Hancock, Kosciusco, Huntington, Tippecanoe
• Autodesk MapGuide
§ Hamilton, Hendricks, Monroe, Wayne
• WTH Mapserver™ Web Mapping Application (OGC Minnesota Map Server)
§ Fulton, Cass, Daviess, City of Huntingburg
Ø
Also there are state-wide GIS servers
•
Orthophotography from Indiana University
•
Indiana Geological Survey
Map Requests for ArcIMS and ArcMap
Ø Map image requests for ESRI ArcIMS and ArcMap are based on ArcXML
Map Requests for other type of Servers
Ø
Map requests for other type of servers are using HTTP GET method.
Ø
Trick is to figure out the correct format for the name/value pairs.
Ø
Requesting format of MapGuide and WMS is almost the same
Ø
Map image is directly returned in the HTTP response (GIF, JPG, etc)
Request for AutoDesk MapGuide Server
http://litemap.co.hamilton.in.us:8080/liteview/servlet/MapGuideLiteView?VERSION=1.1.1&REQ UEST=Gemap&LAYERS=COUNTY_PLAN.MWF\parcels&SRS=EPSG:4326&BBOX=86.0009765625 ,40.06125658140474,85.99960327148438,40.062307630891&WIDTH=256&HEIGHT=256&FOR MAT=image/png&BGCOLOR=0xFFFFFF&TRANSPARENT=TRUE&WMTVER=1.1.1&STYLES=
Request for WTH Web Map Server
Caching for Performance
County Server
Cache Server
Ø
Performance is constrained by the
performance of the individual servers.
Building a Tiling Server
Tiling Strategy
Ø Google Maps works by delivering map tiles that fill a bounding box.
Ø Google Map API 2.0 lets you point at your own tile server.
• We use this to serve up our own map data together with Google maps.
Ø To do this, all tiles should be saved as the same bounding box as Google map tiles.
• Tiles must have same size, projection, and coordinate values as underlying Google base maps.
Ø Each tile uses tile ID and zoom level as its name (ie, tile36.48.10), so no database is needed to find the tile.
• Our tile naming convention based on Google’s lat/lon to transverse Mercator projection.
• tileX.Y.Zoom, but must first convert lat and lon to rectangular coordinates. • Naming convention discussed on next slide. See
http://mapki.com/wiki/Lat/Lon_To_Tile
Converting bounding box to Google tile values
Google uses an x,y coordinate system combined with a zoom value to
specify the tiles to retrieve from the server. These coordinates are
calculated using an algorithm which can be found in GoogleMapki. See:
http://www.codeproject.com/useritems/googlemap.asp
Steps for fetching image tiles from county map servers that match Google map tiles.
Naming Tiles Example
A (-88.2, 42.4)
B(-84.6, 37.1)
Bounding box of Indiana at zoom level 10
Tile 1 (36,47)
Tile 2 (36,48)
Tile 3 (36,49)
Tile 4 (37,47)
Tile 5 (37,48)
Matching Projections: EPSG4326 to Mercator
Combine Google map with county parcel data
Ø
Map servers typically contain base maps
and optional layers.
• Parcel boundaries, roads, and
township boundaries are
layers.
Ø
We cache each layer separately.
Combine tiles at County Boundaries
Marion
County
Hancock
County
Next Steps
Ø Caching more regions in Indiana and elsewhere.
• If county uses ESRI or OGC map server, current agent plugins can be used. • We would like to do other neighboring states and California next.
Ø Build more interesting applications on top of this.
• Use to represent outputs of scientific applications.
• Contour plots, vector maps, and other types of layers for displaying results of geophysical applications.
• Dynamic (“real time” layers) to display streaming data from instruments and applications.
Ø Find a way to keep current with county servers, especially when the county server change layer id.
• Recent Monroe county example
Ø The tiling services should support multiple server styles for multiple clients
• URLs for REST/AJAX style clients
• WSDL and SOAP for formal Web Services • Support OGC and ESRI clients.
Observations and Conclusions on SOA
and Web 2.0
Ø
Web 2.0 approaches are very compatible with SOA in
general...
• Although the details are important.
Ø
But we have to do a better job making services.
• We are very good at making complicated interfaces to simple services.
• Programmable Web lists 450+ simple public APIs to complicated
services.
Ø
And it’s the data...we need to find better ways to use Grids
like the TeraGrid to populate data services instead of
concentrating on providing computing web services.
• Computations will always require expertise.
• Grid software is useful for computing experts.
• But not everyone needs this. We need to think of better ways for
More Information
Ø
[email protected]
Ø
S
ee demo:
•
ht
tp://156.56.104.164/demo/indianaViewer.html
Ø
Col
laborative version:
•
http
://156.56.104.164/samples/CollabmapUpdate/indian
aViewer.html
•
Need
a) Flash, and b) a friend to also try.
Comparison of state and county data
Ø
10 foot contours (1990)
1 foot contours (2006)
Ø
Missing local roads
Local roads (2006)
Ø
No parcels
Parcels (2006)
Ø
No point addresses
Point addresses (2006)
Basic Problem: Data Federation
Ø
Integrated GIS systems have obvious benefits but
inevitably systems are developed by various state
and local government agencies.
•
Bottom up rather than top down
Considerations
Ø
We assume heterogeneity in GIS map and feature servers.
• Must find a way to federate existing services
Ø
We must reconcile ESRI, OGC, Google Map, and other technical
approaches.
• Make a clean distinction between clients and services
• Must try to take advantage of Google, ESRI, etc rather than compete.
Ø
We must have good performance and interactivity.
• Servers must respond quickly--launching queries to 20 different map servers is very inefficient.
Developing issues
Ø Integrating GIS map servers is not trivial
• Different county map servers may use different technologies and web services.
• Interoperability of Geospatial Referencing: Different coordinate systems and projections are used by the different county web services.
• Semantic Interoperability: Different attribute names for layers are used by the different county web services.
Ø Our solution: create a virtual map server to act as an agent server
• Translates map requests from generic format to the format expected by the specific map server. • Provides a common language and programming interface for constructing clients
• Projecting the information to a single coordinate system
• Standardizing symbol
Ø The agent server by itself will work but performance is not good
• Must wait for slowest server to respond
Caching Server
Ø
The agent server runs offline to harvest map images from county map
servers.
• Images are stored as tiles.
• Tiles at county boundaries may be combined for greater storage and performance efficiency.
Ø
Clients connect to the cache server instead of the agent server.
Ø
The cache server constructs the requested image from pre-fetched
tiles.
• Inspired by Google Maps approach • Enable more interactive clients
Ø
Image construction may be parallelized/multi-threaded for greater
performance.
Two Phase Approach: Caching and Tiling
Ø Federation through caching:
• WMS and WFS resources are queried and results are stored on the cache servers. • WMS images are stored as tiles.
§ These can be assembled into new images on demand (c. f. Google Maps). § Projections and styling can be reconciled.
§ We can store multiple layers this way.
• We build adapters that can work with ESRI and OGC products; tailor to specific counties. Ø Tiling:
• Client programs obtain images directly from our tile server.
§ That is, don’t go back to the original WMS for every request. • Similar approaches can be used to mediate WFS requests.
Building Indiana Map client Using Google Map API
Ø
Google Map API V2 enables us to add custom map directly
on Google Map .
•
See
http://mapki.com/wiki/Add_Your_Own_Custom_MapØ
To utilize Google map API to build Indiana map client, there
are several steps:
• Register for using Google map API • Create a collection of copyrights.
• Give those copyright collections to all tile layers that we want to add on Google Map.
• Pass those tile layer(s) off to create a map type.
Example code for building custom map on Google Map
Ø Create a GCopyrightvar copyright = new GCopyright(1, new GLatLngBounds(new GLatLng(-90, -180), new GLatLng(90, 180)), 0, “@Indiana Geology Survey");
Ø Add the GCopyright to a GCopyrightCollection
var copyrightCollection = new GCopyrightCollection('Chart'); copyrightCollection.addCopyright(copyright);
Ø Create a GTileLayer array
var tilelayers = [new GTileLayer(copyrightCollection , 3, 11)]; tilelayers[0].getTileUrl = CustomGetTileUrl; function CustomGetTileUrl(a,b) {
var z = 17 - b;
var f = "/maps/?x="+a.x+"&y="+a.y+"&zoom="+z; return f;
}
Ø Create a GMapType
var custommap = new GMapType(tilelayers, new GMercatorProjection(12), "Chart", {errorMessage:"No data available"});
Ø Add the custom map type to the map map.addMapType(custommap);
Collaborative Indiana Map
Ø Flex data service from Adobe which is the basic technology used to support collaboration of Indiana map clients.
• Enables innovative applications to be delivered in the browser in a reliable and scalable manner. • Enables data to automatically be pushed to the client application without polling.
• Enables a client application to concurrently share data with other clients or other servers. This model enables new application concepts like “co-browsing” and synchronous collaboration. Ø The Flex module adds the following collaboration features to Google Maps:
• Map sharing: Maps are kept in sync (in real time) between users involved in a collaboration session.
• Videoconferencing (Webcam sharing and VOIP): You can share your Webcam and microphone to add video and audio to your collaboration session.
• White-board: Collaborating users can draw on the map. For example you could draw potential directions, etc. The users’ whiteboards are kept in sync in real time.
• Chat-board: Collaborating users can chat on the chat-board in real time.
• Cursor sharing: When you move your mouse, other users see the movements of your mouse and what you are pointing at.
Flex runtime architecture
Ø To enable collaboration, clients need flash player first installed.
Ø To the clients who subscribe to the same channel which configured in the flex data services, the events invoked in one client could broadcast to other clients to
enable collaboration.
Ø Clients can make direct calls to Java objects as well as subscribe to real-time data feeds, send messages to other clients, and integrate with existing Java Message Service (JMS) messaging
Storage of caching entire state
Ø Currently storing 15 counties at 13 zoom levels for 13 layers. It is takes ~250 GB.
Ø Takes about 2.5-3 TB to store the entire state to zoom level 13 this way.
• There are 48410476 tiles for zoom levels 0-13, 162561384 tiles for 0-14 levels (nearly 12 TB).
• There are ~10 layers for each scale
§ Aerial photo layer tiles take 25~30 KB
§ Other layers (parcels, roads) are much smaller: 30~36 KB for all remaining 9 layers per tile
§ So we need almost 60KB * 48410476 tiles to store all map data § Layers from Google (Hybrids, Street, Google Satellite) don’t need to
be cached.
• This is large but possible.
Ø We can easily spread our caching server over multiple hosts to store even higher magnification scales.
Current Progress
Ø Supports ESRI and OGC servers
• Now 17 counties is being cached. (Marion, Monroe are fully cached for 13 zoom levels)
Ø 7 layers has been proved that they can be easily cached.
• Aerial photo layer, street , interstate layer, parcel, parcel ID, county boundary, school).
• 3 more layers can be easily shown in client without caching. (Google Map, Google Satellite, Hybrids).
Ø Querying parcel information across boundary. ( MARION-HANCOCK boundary)
Ø Support Geocode querying.
Ø Higher resolution than Google Satellite.
Ø Google Map-like interaction.
Ø Performance and Reliability.
