Building the data cache

ArcGIS Server 9.2 introduced automated processing functions to build and maintain an optimized map cache pyramid for service pre-processed images as a map service. The ArcGIS Server Globe service would stream pre-processed 3D Globe cache imagery to an ArcGIS Explorer or ArcGIS Desktop with 3D Analysis client for 3D visualization - cached imagery would be stored at the client for high performance display. ArcGIS Server also provided a 2-D map cache service that could be used as a data source for 2D Web applications and ArcGIS Desktop or custom desktop clients developed with the ArcGIS Engine software. Figure 8-31 provide an overview of the cached map service image structure.

Figure 8-31 Cached Map Service

The cached map service would consist of a pyramid of pre-processed data imagery, starting at a single map resolution at the highest layer and breaking each image into four images at twice the resolution for each additional layer included in the pyramid. Client access to the cached data would deliver only tiles that

correspond to the resolution of the map display request. Tiles would be combined together as a single reference layer by the client application. Total pre-processing time would depend on the total number of images in the map cache and the map service time required to generate an average map display.

System Design Strategies 8 Software Performance C11144-12

Figure 8-32 can be used to get a rough estimate of the expected map cache generate time.

Figure 8-32

Generating the Map Cache

The chart above shows the estimated processing hours starting with one tile at the top layer and building the required number of layers to reach a resolution level. Three map generation times are plotted - 0.5 seconds for a simple map display, 1 second for a medium map display, and 5 seconds for a heavy map display. Charts shows about 100 hours to generate 9 layers with average map generation time of 5 seconds. It would take over 2000 hours to generate just two more layers (11 layers at 5 sec per map tile). This chart is generated simply by multiplying the time to generate one map tile by the total number of tiles required to complete the map cache pyramid.

The recommended procedure for estimating map cache generation times is to select a same area dataset that represents a small percentage of the total caching area. Build a map cache of the sample area and test the output symbology, labeling, and performance with your primary map client. Use the cache time for the small sample area to estimate processing time for the remaining map cache.

The ArcGIS 9.3 release additional options for building and maintaining map cache that make this technology adaptive to a broad range of operational scenarios.

Partial Data Cache. A partial data cache can be defined for high priority areas within your display

environment. The partial data cache can be specified by both area and level of resolution, providing optimum flexibility for pre-processing custom areas and levels of map cache.

Cache on Demand. The initial map query can be generated from a dynamic data source, and the resulting tiles can be added to the map cache. The next request for these tiles would come from the map cache, significantly improving display performance for popular map display environments.

System Design Strategies 8 Software Performance C11144-12

Figure 8-33 shows a view of the ArcGIS Server Map Service Properties Caching tab. The map cache process is compute intensive, automatically generating tile after tile based on the defined map cache properties. Map caching can be performed by several parallel service instances, and ArcGIS Server coordinates activities of these services to build and store the prescribed map cache.

Figure 8-33

Optimize number of SOC Instances for Building a Map Cache

When using a local image file data source, each map cache instance will consume a platform core. The optimum service configuration would specify 5 instances to make sure the server is operating at 100 percent capacity. The windows performance task monitor can be used to identify the machine is operating at 100 percent utilization. If your map cache include a DBMS data source, you may be able to include one or two additional service instances to reach 100 percent utilization. It is important to configure sufficient instances to take full advantage of your hardware platform processing resources.

It is a best practice to execute longer cache jobs in sections. It is good to plan cache areas that can be completed within an 8 or 10 hour period, providing an opportunity to complete the jobs in a stable platform environment.

This will also provide an opportunity to periodically reboot system between each job section to maintain a reliable platform environment.

System Design Strategies 8 Software Performance C11144-12

Figure 8-34 provides an example of taking advantage of the hardware, as described above. In this example, the cache job required 500 hours of processing. A single instance would take 500 hours to complete. Five (5) instances running on a 4 core machine will complete the same job in 125 hours. Ten (10) instances, using two 4 core machines each with 5 instances, completed the same job in 65 hours.

Figure 8-34

Sample Map Cache Processing Profile (500 Hours of Processing)

A variety of caching scenarios are being evaluated to expand the feasibility of using pre-processed data cache to improve performance and scalability of future GIS operations. Experience shows pre-processing map data can make a difference for our customers. ArcGIS Server is leading the way toward more productive and higher quality Enterprise level GIS operations.