SURFACE IMAGERY BASED MAPPING AND ROVER LOCALIZATION FOR THE
2003 MARS EXPLORATION ROVER MISSION
Kaichang Di, Fengliang Xu, Jue Wang, Xutong Niu, Charles Serafy, Feng Zhou, and Ron Li Mapping and GIS Laboratory, CEEGS, The Ohio State University
470 Hitchcock Hall, 2070 Neil Avenue, Columbus, OH 43210-1275 Tel: (614) 292-6946, Fax: (614) 292-2957
Email: (di.2, xu.101, wang.813, niu.9, Serafy.1, zhou.182, li.282)@osu.edu Larry Matthies
Jet Propulsion Laboratory, California Institute of Technology Mail Stop 125-209, Pasadena, CA 91109
Email: [email protected]
ABSTRACT
This paper presents the technology of mapping and rover localization at the two landing sites, Gusev and Meridiani, for the 2003 Mars Exploration Rover (MER) mission. The rover localization and landing site mapping technology is based on the incremental bundle adjustment of an image network formed by Pancam, and Navcam stereo images. The developed incremental bundle adjustment model supplies improved rover locations and image orientation parameters, which are critical for the generation of high quality landing site topographic mapping products. The rover localization results demonstrate that the bundle adjustment technology is able to correct position errors caused by wheel slippages, azimuthal angle drift and other navigation errors as large as 21%. The mapping products, which are generated at each stop of the rovers, include digital terrain models, orthophotos, and rover traverse maps. These maps and localization information were provided to MER mission scientists and engineers through a Web GIS.
INTRODUCTION
In the Mars Exploration Rover (MER) 2003 mission, the twin rovers, Spirit and Opportunity, carry identical Athena Instrument Payloads and engineering cameras to explore the landing sites of Gusev Crater and Meridiani Planum (Squyres et al., 2003; 2004). To support science and engineering operations, it is critical to localize the rovers and map the landing and traversing area with high accuracy (Arvidson et al., 2004). During the mission, The Ohio State University (OSU) team, collaborating with JPL, has been routinely producing topographic maps, rover traverse maps, and updated rover locations to support tactical and strategic operations. These maps and localization data were provided to MER mission scientists and engineers through a Web GIS.
Among the various instruments on board the rovers, Pancam (Panoramic Camera) and Navcam (Navigation Camera) stereo cameras are the most important for high-precision landing-site mapping and rover localization. These two stereo-imaging systems are mounted on the same stereo bar of the rover mast. The image sizes of both the Pancam and Navcam are 1,024 × 1,024 pixels. Navcam has a stereo base of 20 cm, a focal length of 14.67 mm, and an effective depth of field of 0.5 m to infinity. Its best focus is at 1 m with a field of view (FOV) of 45 degrees. Pancam has a wider stereo base (30 cm) and a longer focal length (43 mm), making it more effective for mapping medium-to-far objects in the panoramic images. The effective depth of field for the Pancam is 3 m to infinity and the FOV is 16 degrees.
The rover localization and landing-site mapping technology is based on the bundle adjustment (BA) of an image network formed by surface imagery, i.e., Pancam and Navcam stereo images. The overall technology is described in (Li et al., 2004a). Before the MER mission, the rover localization and mapping technology had been extensively tested and verified with field test data acquired on earth and actual Mars data from the 1997 Mars Pathfinder mission (Li et al., 2002; Di et al., 2002). After the landing of the two rovers, the OSU team and collaborating scientists and engineers of the mission performed lander localization using rover panoramic images, orbital images, and descent images as well as radio science based localization. The initial results of lander/rover localization, regional mapping using orbital and descent images, and detailed landing-site mapping using surface imagery are reported in Li et al. (2004b). In this paper, a detailed description of the technology of mapping and rover localization using surface imagery is given and updated mapping and localization results at the two landing sites are presented.
CAMERA MODEL AND REFERENCE SYSYTEM
As a starting point for photogrammetric processing, the camera models and reference systems must be elucidated. The original camera model of the Pancam and Navcam images is the CAHVOR model, which models the transformation from the object domain to the image domain by using vectors C, A, H, and V and corrects radial lens distortions with a vector O and a triplet R. It has an intrinsic difference with the conventional photogrammetric model but can be converted to the photogrammetric model with sufficient accuracy (Di and Li, 2004a). The model is defined in rover frame, in which the X axis points forward, the Z axis points down, and the Y axis is defined to form a right-handed system. The parameters of the CAHVOR model are stored in the image header. To facilitate rover operations in an extended landing site, individual site frames are defined along the traverse. The X axis of a site frame points to north. The Z axis points down in the normal direction. The Y axis is defined to form a right-handed system. The position and attitude of each rover frame, with respect to its site frame, is defined by three translations and a set of quaternion parameters, which are also included in the image header. The first site frame (Site 0), which is at the lander, is defined as the Landing Site Local (LSL) frame for mapping and rover localization purposes. The relative position of a site frame to its previous site frame is stored in a master file. This primary geometric information is regularly supplied by the onboard Inertial Measurement Unit (IMU) and wheel-odometry–based localization system with infrequent support by sun-finding techniques that improve the azimuth quality.
In addition to original rover images with the CAHVOR camera model, linearized images are also provided to facilitate stereo viewing and matching. Since the linearized image is resampled according to epipolar geometry, there is practically no parallax in the vertical direction. The lens distortions have also been corrected in the linearized imagery. Thus, the CAHVOR model is reduced to a CAHV camera model, which does not have lens -distortion components.
Linearized Pancam and Navcam images were used for in topographic mapping and rover localization. First, the CAHV camera model was converted to a photogrammetric model that is necessary and is commonly used for topographic mapping and remote sensing (Di and Li, 2004a). The camera model is then transformed from the rover frame to its site frame and subsequently to the LSL by sequential rotations and translations. The resultant image-orientation parameters are used as initial approximations in the BA.
Orbital images were available pre- and post-landing. After the landing of the two rovers, the Mars Global Surveyor (MGS) Mars Orbiter Camera (MOC) aimed at the two estimated lander locations and took a new, high-resolution (1 m) cPROTO (compensated Pitch and Roll Targeted Observations) image at the Gusev site on Sol 16 and a ROTO (Roll-Only Targeted Observation) image at the Meridiani site on Sol 13 (Malin, 2004). The landers can be seen from the two high-resolution images. Through the lander position, which is also the origin of the LSL, the LSL can be linked to the Mars body-fixed reference system.
ROVER LOCALIZATION BASED ON BUNDLE ADJUSTMENT OF SURFACE
IMAGE NETWORK
As indicated above, on-board rover localization is primarily performed by the IMU, wheel-odometry, and sun-finding technology. In cases where the rover experiences slippage caused by traversing loose soil or steep slopes, particularly in a crater, the onboard visual odometry (VO) technique was applied. In this mission, VO has acquired consecutive Navcam stereo pairs (<0.75 m) within a short traverse distance (<10 m). The BA technique combines the results of VO and images and builds an image network containing all panoramas and traversing images along the traverse to achieve a high-accuracy solution of rover positions along the entire traverse (see Figure 1).
Figure 1. Illustration of a rover traverse and the image network with Pancam and Navcam images
Image Network Construction
The image network is constructed by linking the panoramic and traversing images with automatically and manually selected tie points. The key to the success of the BA is to select sufficient well-distributed tie points. A systematic method to automatically select tie points from panoramic images taken at one position has been developed (Li et al., 2003; Xu, 2004). This tie-point selection method consists of five steps: interest point extraction using the Förstner operator, interest point matching, parallax verification, graph consistency verification, and, finally, tie-point selection by gridding. In matching interest points between adjacent stereo pairs, a rough digital terrain model (DTM) is generated to predict the location of conjugate points and to limit the search range. This method has been successful in selecting tie points with the same site. At most of the sites (>95%), tie points can be selected automatically. Figure 2 shows an example of automatically selected tie points at two adjacent stereos. The left stereo is shown on the left column, and the right stereo on the right. The blue crosses are intra-stereo tie points, which are the tie points within one stereo pair taken by the stereo camera. The red crosses are inter-stereo tie points, which are the tie points between adjacent stereo pairs taken at another angle.
Selection of cross-site tie points (tie points between images taken at different sites) is challenging and is done manually. The difficulty comes from the significant differences between looking angle and resolution, and the large distance between the adjacent sites. Figure 3 shows an example of manually selected cross-site tie points. The upper row shows the stereo images at one site, and the lower row shows the stereo images at the adjacent site, which is 38 m away from the first site. From the first sight, it is difficult for the human eye to identify the same features. To overcome the difficulty, a strategy and a number of interactive tools were developed to assist manual tie-point selection. For example, the projection tools can project the feature point from one stereo to the other by using the initial image-orientation parameters to give a rough location of the corresponding feature. Orthophotos can be generated from the two sites by using the initial image-orientation parameters, and overlay and compare the orthophotos to identify corresponding features. Then features from the orthophoto can be projected back to the original images, and cross-site tie points can be picked from there. Anaglyph stereo is also frequently used to help identify corresponding features (e.g., rocks) in three dimensions and locate the same point (e.g., a corner of a rock) on the feature. Overall, cross-site tie-point selection remains a challenging task and is the “bottleneck” in landing-site mapping and rover-localization work. In the future, new algorithms will be developed to automate the cross-landing-site tie-point selection process.
Figure 3. Manually selected cross-site tie points Rover Localization By Bundle Adjustment
In order to provide timely rover-localization information, the BA is preformed incrementally by fixing the previous site and adjusting only the current site. This strategy greatly reduces the computation time and still gives satisfactory results. To ensure high precision, the correlations between the image-orientations parameters of the stereo images are used as constraints for the least squares adjustment (Di et al., 2004b).
The pointing information of images from telemetry data within one panorama is generally consistent. Pointing information of images taken in adjacent site frames often has noticeable or significant inconsistencies, which are caused by wheel slippages, azimuthal angle drift, and other navigation errors. Since these inconsistencies appear to be systematic, a transformation (translation and rotation without scale change) was always applied to the last site frame based on cross-site tie points to ensure that better initial pointing information of the BA network is achieved for convergence.
Since no absolute control is available on a meaningful level at this time, the accuracy of the BA is estimated by using discrepancies computed from two-dimensional image coordinates and three-dimensional ground coordinates of bundle-adjusted cross-site and inter-stereo tie points that are projected from different stereo pairs. Specifically, the three-dimensional accuracy is derived from differences of three-dimensional ground coordinates of the cross-site tie points triangulated from different site frames. The two-dimensional accuracy is estimated by using differences between image coordinates of the same inter-stereo and cross-site tie points, which are measured in one stereo pair and projected from one or more stereo pairs (neighboring pair for inter-stereo, and opposite pair for cross-site). The BA result of site 6700 (Sol 155) and 6800 (Sol 157) at Gusev Crater site is a typical example. There are 38 images (19 stereo pairs) involved in the image network, with 7 cross-site tie points manually selected, and 99 inter- and 218 intra-stereo tie points automatically selected. By fixing the image orientation parameters of site 6700 from previous BA computation, the adjustment of site 6800 is done within a half minute. Before BA, the two-dimensional accuracy is 2798.33 pixels and three-dimensional accuracy is 29.351 m. After BA, two-dimensional accuracy is 1.24 pixels and three-dimensional accuracy is 0.760 m.
BA at Gusev Crater site was performed regularly by using full or partial Navcam panoramic images, and occasionally forward- and rear-looking Pancam images. Overall, after BA, two-dimensional accuracy is sub-pixel to 1.5 pixels and three-dimensional accuracy is tens of centimeters. In order to compare the difference between the rover traverses from the telemetry and from BA, rover-traverse maps were generated whenever a new site was added and adjusted. Figure 4 illustrates the Spirit rover traverse up to Sol 282 when the rover was at the foot of Columbia Hills and had traveled 3260 m from the lander position, which is named Columbia Memorial Station. By Sol 282, the accumulated difference between telemetry and BA position was 13.94 m. The maximum accumulated difference was 31.54 m on Sol 151 over a traveled distance of 2814.24 from the lander, generating a relative error of 1.1%. From Sol 6 to Sol 66, Spirit moved northeast toward Bonneville Crater; from Sol 66 to Sol 86 the rover traveled along the crater rim; and, after Sol 88, the rover moved sourtheast toward Columbia Hills. Along this traverse up to Sol 106, a typical error accumulation occurred. The relative error (20.62 accumulated difference over 775.7 m traveled distance) reached 2.7% on Sol 106.
Figure 4. Spirit rover traverse map up to Sol 282 (blue from telemetry; red from bundle adjustment)
In addition to the rover traverse map, a vertical profile was generated and expanded as the rover traveled. Figure 5 shows a vertical profile of Spirit up to Sol 282. The horizontal axis of the figure is the traveled distance from the lander, and the vertical axis depicts elevation (scaled). Again, the blue line is the profile computed from telemetry data, and the red line shows the BA result. The accumulated elevation difference is 16.1 m over a traveled distance of 3260 m. This difference may be attributed to wheel slippage and IMU drift.
Bonneville Crater
Figure 5. Vertical profile of the Spirit rover traverse up to Sol 282 (blue from telemetry; red from bundle adjustment)
Figure 6. Opportunity rover traverse map up to Sol 62 (blue from telemetry; red from bundle adjustment) For the Meridiani site, BA was conducted within the Eagle crater (up to Sol 62) where Opportunity landed. Figure 6 shows the Opportunity rover traverse up to Sol 62. The accumulated difference reached 21 m, or 13% of the traveled distance on Sol 62. A maximum relative error of 21% occurred on Sol 56. The significant localization errors in the telemetry data were caused mainly by wheel slippage when Opportunity traversed the crater wall on
Columbia Hills Bonneville Crater Missoula Crater
loose soil and steep slopes for 56 sols. This demonstrated that the BA was able to correct the significant localization errors.
On the way from Eagle crater to Anatolia (see Figure 11), a data gap of 100 m made a BA-based traverse impossible. After this gap, until the rover moved down to Endurance crater, a translation was applied (obtained from the BA result of Sol 62) to the rover local locations from telemetry without BA. By doing so, large features, such as Fram crater and Endurance crater, when measured from the ground images along the traverse, are generally match well with their positions on the MOC image. This indicates that after exiting Eagle crater, the rover did not experience significant slippages as in Eagle crater. After the rover moved inside Endurance crater and performed investigations on the crater wall, significant slippages occurred again. Rover localization errors were corrected by comparing the features in orthophoto patches at each stop with a base orthophoto generated by using two Pancam panoramas taken at the crater rim (see Figure 10). This adjustment method enabled us to provide the Opportunity traverse in a timely manner.
Traverse image maps were also produced by back-projection of rover positions onto the image mosaics. Figure 7 is a rover traverse image map, which vividly shows the Spirit’s track across the Martain surface as it climbed Husband Hill from Sol 149 to Sol 233.
Figure 7. Spirit rover traverse image map (Sol 149 to Sol 233), up to Husband hill
TOPOGRAPHIC MAPPING
Topographic products, such as DTM, orthophoto, and three-dimensional models were generated routinely at the stops of the two sites by using Pancam and Navcam images. The orthophotos from Pancam cover an area of 60 × 60 m and that from Navcam covers an area of 30 × 30 m, both having a resolution of 0.01 m. The steps of DTM and orthophoto generation include: dense interest-point matching between intra-stereo images, three-dimensional 3D position calculation of the matched points, TIN (triangular irregular network) construction and DTM interpolation, and orthophoto generation through projection between the images and DTM. Contour maps were also produced for craters. The details of the mapping algorithms can be found in Xu (2004). Based on DTM interpolation and othrophoto generation, three-dimensional models were generated for big features, e.g., craters. Figure 8 shows an orthophoto of Laguna Hollow at the Spirit site on Sol 45 Figure 8. Spirit orthophoto of Laguna Hollow on
generated by using Pancam images. Figure 9 shows the orthophoto (a) and a three-dimensional model (b) of Eagle crater at the Opportunity site generated by using Pancam images. Figure 10 shows an image mosaic and a three-dimensional model of the Endurance crater, which has a diameter of 156 m. The DTM were generated by adjusting and integrating the two Pancam panoramas taken at two opposite locations (site locations P2002 and P2809 appear in the figure) at the crater rim.
By Sol 282 of Spirit and Sol 263 of Opportunity, we generated timely topographic products including 76 orthophotos and three-dimensional DTM, and five three-dimensional crater models, as well as periodical rover traverse maps and vertical profiles.
a. Orthophoto b. 3D model
Figure 9. Orthophoto and three-dimensionalmodel of Eagle crater at Opportunity site
Figure 10. An image mosaic and a model of Endurance crater at the Opportunity site. P2002 and P 2809 are the two site positions where the panoramas were taken
WEB GIS
A web-based landing site GIS system was established at the OSU Mapping and GIS Laboratory to update and disseminate the localization and topographic information daily to support tactical and strategic operations of the mission. The web GIS was developed by using both HTML and ESRI’s ArcIMS. All mapping and localization products are included and organized in different layers or hyper-linked web pages and can be explored by using tools such as zoom, pan, identification, measurement, and hyperlink. The original rover images can also be retrieved through hyperlinks of the image pointing lines. The three-dimensional interactive model can be viewed and manipulated on the web through an embedded VRML viewer. This internal Web GIS proved to be valuable and efficient for mission scientists and engineers to track the two rovers and the local terrain along the traverses.
Figure 11. Interfaces of the web GIS for accessing Spirit (left) and Opportunity (right) traverse information and local topographic products
SUMMARY
This paper presents the surface imagery based technology of mapping and rover localization at the two landing sites of the MER mission and summarizes the achievements by Sol 282 of Spirit and Sol 263 of Opportunity. The camera models and coordinate frames, automatic tie-point selection and image network construction, BA, topographic product generation, and Web GIS, are described systematically. The rover localization results demonstrate that the BA technology is able to correct position errors as large as 21%. Timely generation of various topographic products and localization information and their dissemination through a Web GIS greatly supported tactical and strategic operations of the mission.
As of December 1, 2004, both rovers are “healthy” and are continuing to explore the two landing sites. The surface operations have far exceeded the nominal mission of 90 days from January to April. The reported activities will continue to support the mission operations.
ACKNOWLEDGMENTS
Funding for the research by the Mars Exploration Program of NASA is acknowledged. The collaboration with the science and engineering team of the MER mission is greatly appreciated.
REFERENCES
Arvidson, R.E., R.C. Anderson, P. Bartlett, J.F. Bell, III, D. Blaney, P.R. Christensen, P. Chu, L. Crumpler, K. Davis, B.L. Ehlmann, R. Fergason, M.P. Golombek, S. Gorevan, J.A. Grant, R. Greeley, E.A. Guinness, A.F.C. Haldemann, K. Herkenhoff, J. Johnson, G. Landis, R. Li, R. Lindemann, H. McSween, D.W. Ming, T. Myrick, L. Richter, F.P. Seelos, IV, S. W. Squyres, R.J. Sullivan, A. Wang, J. Wilson (2004). Localization and Physical Properties Experiments Conducted by Spirit at Gusev Crater. Science, Special Issue on MER 2003 Mission, 305(5685): 821-824.
Di, K., F. Xu, R. Li, L. Matthies, C. Olson (2002). High precision landing site mapping and rover localization by integrated bundle adjustment of MPF surface images. In: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (IAPRS), Vol. 34, Part 4, pp.733-737.
Di, K., and R. Li (2004a). CAHVOR camera model and its photogrammetric conversion for planetary applications,
J. Geophys. Res., Planets, 109(E4), E04004 [doi:10.1029/2003JE002199].
Di, K., F. Xu and R. Li (2004b). Constrained Bundle Adjustment of Panoramic Stereo Images for Mars Landing Site Mapping. Proceedings of the 4th International Symposium on Mobile Mapping Technology, Kunming, China, March 29-31, 2004.
Li, R., F. Ma, F. Xu, L.H. Matthies, C.F. Olson, and R.E. Arvidson (2002). Localization of Mars rovers using descent and surface-based image data. J. Geophys. Res., Planets, FIDO Special Issue (R.E. Arvidson, editor), 107(E11): FIDO 4.1-4.8.
Li, R., K. Di, and F. Xu (2003). Automatic Mars landing site mapping using surface-based images, ISPRS WG IV/9: Extraterrestrial Mapping Workshop on Advances in Planetary Mapping, March 22, 2003, Houston, Texas. URL: http://www.lpi.usra.edu/meetings/lpsc2003/lpsc2003.download.html.
Li, R., K. Di, L.H. Matthies, R.E. Arvidson, W.M. Folkner, and B.A. Archinal (2004a). Rover localization and landing site mapping technology for 2003 Mars Exploration Rover mission. Photogrammetric Engineering & Remote Sensing, 70(1): 77-90.
Li, R., S. W. Squyres, R. E. Arvidson, B. A. Archinal, J. Bell, Y. Cheng, L. Crumpler, D. J. Des Marais, K. Di, T. A. Ely, M. Golombek, E. Graat, J. Grant, J. Guinn, A. Johnson, R. Greeley, R. L. Kirk, M. Maimone, L. H. Matthies, M. Malin, T. Parker, M. Sims, L. A. Soderblom, S. Thompson, J. Wang, P. Whelley, and F. Xu (2004b). Initial Results of Rover Localization and Topographic Mapping for the 2003 Mars Exploration Rover Mission. Photogrammetric Engineering and Remote Sensing, submitted.
Malin, M. (2004). MOC cPROTO and ROTO images with Spirit and Opportunity landers, http://www.msss.com/mer_mission/index.html, Malin Space Science Systems, San Diego, CA.
Xu, F. (2004). Mapping and Localization for Extraterrestrial Robotic Explorations. Ph.D. Dissertation, The Ohio State University, Columbus, Ohio.
Squyres, S.W., R.E. Arvidson, E.T. Baumgartner, J.F. Bell, III, P.R. Christensen, S. Gorevan, K.E. Herkenhoff, G. Klingelhöfer, M.B. Madsen, R.V. Morris, R. Rieder, and R.A. Romero (2003). Athena Mars rover science investigation. J. Geophys. Res., 108(E12): 8062 [doi:10.1029/2003JE002121].
Squyres, S.W., R.E. Arvidson, J.F. Bell, III, J. Brückner, N.A. Cabrol, W. Calvin, M.H. Carr, P.R. Christensen, B.C. Clark, L. Crumpler, D.J. Des Marais, C. d’Uston, T. Economou, J. Farmer, W. Farrand, W. Folkner, M. Golombek, S. Gorevan, J.A. Grant, R. Greeley, J. Grotzinger, L. Haskin, K.E. Herkenhoff, S. Hviid, J. Johnson, G. Klingelhöfer, A. Knoll, G. Landis, M. Lemmon, R. Li, M.B. Madsen, M.C. Malin, S.M. McLennan , H.Y. McSween, D. W. Ming, J. Moersch, R.V. Morris, T. Parker, J. W. Rice, Jr., L. Richter, R. Rieder, M. Sims, M. Smith, P. Smith, L.A. Soderblom, R. Sullivan, H. Wänke, T. Wdowiak, M. Wolff, and A. Yen (2004). The Spirit rover's Athena science investigation at Gusev Crater, Mars. Science, Special Issue on MER 2003 Mission, 305(5685): 794-799.
