Received 11 March 2009; accepted 23 May 2009
Corresponding author. Tel: +1-814-8631620; E-mail address: [email protected]
Zigbee/Google Earth based assisted driving system in mining
SUN En-ji
1, NIETO Antonio
21Department of Mining and Mineral Engineering, University of Science and Technology Beijing, Beijing 100083, China 2Department of Energy and Mineral Engineering, Penn State University, PA 16802, USA
Abstract: The Assisted Driving System (ADS) for haul trucks operating in surface mining and construction sites is to reduce acci-dents related to low visibility conditions. This system is based on the GPS, Zigbee, and the Google-Earth engine as the graphic interface and mine-mapping server. The system has the capability to pin-point and track vehicles in real time using a 3D interface, which is based on user-based AutoCAD mine maps using the Google-Earth graphics interface. All equipped vehicles are shown in a 3D mine map stored in a local server through a wireless network. When low visibility conditions are present, the system indicates available exit/escape routes for driver safety. The ADS potentially increases reliability and reduces uncertainty in open pit mining operations.
Keywords: 3D simulation; GPS; mining; assisted driving system; Google Earth
1 Introduction
According to the report of the U.S. Mine Safety and Health Administration (MSHA), in the past few decades, there were a significant number of accidents in open pit mining involving off-highway haul trucks. According to the analysis of surface powered haulage accidents, MSHA reported 1300 fatal accidents from 1990 to 2005. Of these, 163 were considered off- highway truck haulage accidents, and at least 23 in-volved collisions of an off-highway truck with a neighboring vehicle or a worker on foot[1]. In general, an average of 18 non-fatal accidents and 3 fatal acci-dents occurred in open pit mining per year when equipment backed over the edge of an embankment, stockpile, or dump point.
The specific problems that need to be solved can be determined by carefully studying the accidents involving haulage trucks[2]. Solutions to haulage truck safety must consider the human factors affecting each task even when engineering solutions seem most ap-propriate.
All of these affects can be attributed partially to the lack of visibility[3]. Driver visibility and communica-tions are key issues to reduce mining truck accidents. Analysis of fatalities and injuries involving mining equipment demonstrates the need to develop new intervention and control strategies to mitigate the risks associated with mining equipment operation. Ongoing development of these and other new
strate-gies is required to achieve a reduction of injury in the mining industry[4].
2 GPS assisted real-time communication
architecture
The Global Positioning System (GPS) is a satel-lite-based navigation system that was developed by the U.S. Department of Defense (DoD) in the early 1970s. Initially, GPS was developed to fulfill U.S. military needs. However, it was later made available to civilians and is now a dual-use system that can be accessed by both military and civilian users. GPS provides continuous positioning and timing informa-tion, anywhere in the world under any weather condi-tions. Because it serves an unlimited number of users, and is still used for security reasons, GPS is a one-way-ranging (passive) system[5]. In other words, users can only receive a satellite signal[6–7].
The Assisted Driving System (ADS) software was written using Visual Studio, and is based on the GE 3D graphical engine. The ADS dynamically updates locations based on the truck GPS or any GPS- equipped vehicle. ADS aim to help truck drivers find and follow safe routes under severe weather condi-tions when low visibility is present. ADS features a GIS layer interface to store different route paths, and it can display other layers of information such as buildings, ramps, and pre-recorded vehicle diagnos-tics sensors (vibration, temperature, etc.).
Mining Science and Technology 19 (2009) 0626–0630
SCIENCE AND TECHNOLOGY
In its present form, the ADS run in portable rugged computers (trucks and server). The ADS receives geo-positional information from the GPS receiver mounted on the truck. After the ADS calculates the GPS position, it sends out the GPS data to the system server through IEEE 802.11 wireless radios. A critical factor for the ADS is the wireless component and its ability to wirelessly transmit GPS positioning data from each enabled vehicle to a remote client/server[8]. After the server receives GPS data; it sends the real-time truck position, local 3D map, and 3D model to the ADS and displays them on the driver’s com-puter screen. The driver then chooses the safest route by simply clicking on the touchable display screen. The system zooms into the truck’s current location using the 3D GE interface; it then loads the corre-sponding road information layer over the mine map. The system tracks the truck’s current location, the mine map, and the suggested safe route as a real-time assisted driving reference. A brief description of how the system works is illustrated in (Fig. 1)
6DWHOOLWH 6DWHOOLWH 6DWHOOLWH 6DWHOOLWH '$6 GLVSOD\ /HJHQG *36GDWD ,SUDGLR 'DWDWDQVIRUP $FFHVV SRLQW /RFDO VHUYHU :LUHOHVVQHWZRUN :LUH OHVVGDWDWU DQVIRUP 6\VWHP PDQDJHU
Fig. 1 GPS assisted real-time communication architecture
The system manager handles the wireless network and server remotely, which includes updating the lo-cal 3D mining map, analyzing the updated GPS data, and setting safe guide routes to truck drivers. All other vehicles equipped with ADS are displayed on the system manager’s computer server screen. Under the wireless component, each vehicle is assigned with an IP address creating a mobile wireless network functioning under a peer-to-peer scheme.
The main advantage of the Assisted Driving Sys-tem (ADS) is that it works using commonly available technologies: the GPS as the geo-reference solution to pinpoint truck locations using satellites, and the Google Earth (GE) graphics engine as a user-friendly, yet powerful 3D graphical interface including basic Geographic Information System (GIS) capabilities. The Assisted Driving System (ADS) uses this func-tion to generate and simulate the geometry of any surface mine and is able to display 3D moving ob-jects equipped with GPS and radios, such as trucks or smaller vehicles. The ADS is capable of dynamically updating the location of those trucks or vehicles in
real-time and show map layers of geographic infor-mation such as roads and ramps.
ADS users can add their own graphical information to the mine map and make the information available through various online sources. ADS is able to dis-play all kinds of geographical information overlaid on the surface of the virtual Mine. ADS is a web base map client featuring GIS capabilities which are used to store and display any kind of sensorial information produced by the mine equipment. ADS supports and manages three-dimensional geospatial data using the Keyhole Markup Language (KML), the standard markup language used in GE.
ADS and its GE interface feature a combination of dynamic zooming based on the GE 3D model of the Earth using surface images taken from satellite im-agery and aerial photography. When zooming into a specific mine, ADS switches from GE maps to AutoCAD maps previously loaded in the computer server. This process improves and enhances the qual-ity of the graphical information given to the operator and the system manager. GE is a novel free software and likely to be a powerful geographical information platform for applications such as ADS. The ADS is the first system proposed for mining and earth mov-ing operations that integrates GE with dynamic equipment GPS positioning, using local AutoCAD maps to augment graphic information in GE.
3 ADS system integration and 3D graphics
engine integration
3.1 Google Earth based graphical display The ADS Windows-based software was developed using Visual C++2005. The system is integrated with GE and the user is required to install the free-edition of GE to launch its 3D graphic engine.
First, the ADS loads GE 3D graphic engine as the main graphics interface. To achieve high resolution 3D interaction control at every zooming level, ADS converts AutoCAD 3D mine maps to GE format in order to augment geographic information on local mine sites. ADS then loads the local 3D mine map, initiates GPS tracking and records the location of the truck or vehicle in real-time. In case of low visibility conditions, and If the user requests it, ADS displays safe routes, roads, paths, and ramps based on the us-er’s current location.
Once the ADS software is started, the computer windows interface consists of two sections. On the left side is the user control panel with command but-tons. On the right side is the 3D display area with the full 3D control functions, including zoom, tilt, drag, rotate, and commands that allow the view to drift dy-namically across the virtual 3D model of the Earth. The GE 3D graphics control of the ADS is powerful and convenient. ADS users can take advantage of all the GE control functions to navigate through the mine
site and pinpoint roads and trucks in 3D view in an effortless process.
There are two types of ADS users: the system manager working at the central control office, and the vehicle (truck) driver. The ADS uses the same GE graphics engine for both users; however, the system relies on two different user interfaces as ‘server’ and as ‘client’, both with two distinct purposes according to the tasks performed respectively by the system manager and the truck driver.
The system manager using the ADS server inter-face can update local mine maps in 3D, inquire the status of the trucks through their ID-IP network numbers, check GPS locations for each truck, and review working conditions by selecting trucks dis-played on the screen by using the touch screen. The system manager can also request recorded (historical) vehicle tracking routes, check wireless network nodes, and set up safe routes, roads, and ramps, according to updated mine maps, mine geometry, and current ve-hicle locations.
In case of low visibility conditions, the truck driver can display his current location with respect to the mine map by simply selecting the ‘find truck’ button on the ADS screen as seen in Figs. 2 and 3. The driver can also display safe routes and roads by ask-ing the ADS to track the vehicle position automati-cally. The ADS will alert the driver if the truck diverts from the pre-defined route with visual and audio warnings.
Fig. 2 Top down view of the mining truck
Fig. 3 Tilted view of two mining trucks
3.2 3D mining map customization
Accurately customizing the local 3D mine map is critical in ADS. As the mining operation advances, roads and ramps within the open pit mine are con-stantly shifting and moving. As a result, it is impor-tant to keep an updated model of the mine geometry loaded in the ADS by continuously updating mine and road maps. The ADS can then transmit new in-formation to every equipped vehicle in the mine.
The system manager uses the ADS software to convert the AutoCAD maps into GE maps, then up-loads the new converted mine maps to the ADS server. The system manager must confirm that the mine map and road information is in the correct scale and in the correct coordinate system. According to the ADS map conversion wizard, the system manager must name the converted mine map by using a geographic refer-ence and the current date in the description field text for future reference.
All the extra layers of information usually embed-ded in AutoCAD mine maps, such as text, colors, and borders, are not necessarily needed by the ADS. The system manager can filter unneeded information us-ing the ADS software and select just those AutoCAD map entities that will be needed for assisted driving purposes. Although it is a straightforward process to convert the entire map in one step, it is not recom-mended to do so due to the potential of generating very large map files. The size of already converted map files must always be considered when using a wireless network. Data transmission efficiency be-tween the server and clients will be directly impacted by the amount of data being broadcast through the network. A small size mine and road map will im-prove the efficiency and response time of the ADS.
Note that some AutoCAD map layers of embedded spatial information, such as rock, soil, and geology, in some cases could be useful to the system manager and operators. In this case the system manager can select this information from the AutoCAD map and import the layers into the ADS. These layers of in-formation, in addition to assisted driving purposes, can be transmitted to trucks and vehicles according to their operational tasks.
The key issue behind the local 3D map conversion is to accurately adjust the AutoCAD mine map to the GE coordinate system to ensure that the local 3D mine model precisely overlays and displays the GE geographic model.
To generate and import a 3D mine map in ADS, a geo-reference point consistent with both maps (UTM and Geodetic) must be defined according to an equivalent point in the WGS84 coordinate model. From the ADS map-conversion window, the user
se-lects a point in the AutoCAD DWG map (X, Y) and
then defines the geodetic location with its equivalent longitude and latitude.
If the mine map is rotated, the software allows the user to correct the inclination and change the orienta-tion by defining the rotaorienta-tion angle of the map as shown in (Fig. 4).
Fig. 4 Convert AutoCAD map
Once both maps have been adjusted, the system manager runs the map conversion routine to convert the mine map into GE format. The system manager then uploads the new converted map wirelessly to the local map server. The map server location is defined by the user.
After the system manager has preprocessed the local 3D mining map, the next step is to upload the local 3D mine map to the system server and to each vehicle. There are two approaches to upload files according to the host receiving the data (maps, roads, etc). One is to locally upload the mine map to the local server using the networked intranet. The second approach is to remotely upload the mine map to every equipped vehicle in the mine using the wireless network.
Once the mine map is transformed from AutoCAD to GE, the map is automatically renamed with an extension KMZ format. KMZ files are Keyhole Markup Language (KML) used by XML language to represent geographic locations. KMZ are KML files which have been compressed or zipped.
When a KMZ file is unzipped, a single KML file is decompressed along with any overlay and icon images referenced in the KML geographic map. KML uses a tag-based structure with nested elements and attributes and is based on the standard XM language.
Once the 3D mine map is loaded, the ADS will automatically zoom to the mine location. The ADS map keeps all original requested AutoCAD informa- tion and its attributes, such as color, lines and polylines.
3.3 Wireless mesh network data transform The Wireless Mesh Network (WMN) is a self- organizing and self-configuring network of mesh routers and clients connected to each other over wireless links. The wireless communication capabi- lity consists primarily of a wireless Ethernet system
based on the IEEE-802.11 standards. An antenna should be deployed around the mining areas to provide point-to-point communication. The primary function of the wireless communication is to provide instrumentation configuration and control and to facilitate data monitoring functions.
An IEEE 802.15.4 network is built using several types of devices, including simple nodes (end devices), routers, star coordinators and PAN coordinators. The 802.15.4 standard has two protocol stack versions. First, there is a full function device (FFD) stack, including all network standard functionalities, such as sending/receiving data, routing and coordinating. Second, there is a lighter version or reduced function device (RFD) stack. This stack is used by end devices. With this stack, several functionalities can be removed, like encryption, routing functions, etc. An RFD stack requires little memory and CPU. It also reduces energy use. Generally, FFD devices are connected to the main power source (main powered) while RFD devices are embedded with the batteries.
The GPS includes measurements of latitude, longitude, altitude, direction and ground speed and the data is acquired at a frequency of 1 Hz. Long distance data delivery between controlling center and internet through GPRS is realized from the monitoring terminals of different geological positions. This system connects TCP/IP of data delivery between long-distance terminal GPRS Modem and controlling centers, while the monitoring center order is received by this connection or sends information data of checking equipment. Once the data transform begins, the uplink and downlink data format is uniform as shown in (Fig. 5).
Synchronization code Byte length Source MAC Target MAC Frame cushion number Control word GPS data CRC code End code
Fig. 5 Signaling format code frame
The data segments of begin and end signaling frame are both FFH. The synchronization code is AAH, 55H. The byte length (1B) usually will be less than 64. The target MAC address is the ADS terminal address. The value of frame cushion number will be OOH when the data is transforming. The control word (1B) will identify the control command or the data. The end code is set to 55H, AAH.
3.4 Real-time vehicle tracking and safe route monitoring
The objective of the ADS is to provide guidance to mine vehicle operators and truck drivers by displaying safe routes when low visibility is present if the driver or operator requests it. If the ADS system is turned on, a line representing the safe route is shown on the mine map as an added roadmap layer. From
the touch screen onboard computer the driver selects the safe route according to the truck’s current location. The driver can choose between different path alternatives using the computer interface. Besides having full control of the assisted driving system from inside the vehicle’s cab, the system manager can remotely load and turn the safe routes and maps to provide route information if requested by the operator.
One of the most powerful features within GE is the network link. The network link enables users to access geographic data on a local machine, or on an internet networked computer. This geographic information is based on the KML file which is the standard format supported by GE. The ADS running at the server uses the network link function to update the truck’s GPS coordinates. The truck’s GPS location is then displayed by the ADS client. The network link can automatically refresh or can be updated on demand by the user depending on operating conditions.
After starting the real-time GPS tracking function, a green dot is used by the ADS to represent the truck position. Real-time latitude, longitude, and altitude are shown/displayed on the panel.
The automatic follow-view option provided in the ADS is another useful camera view function. If the driver loses the current view, this feature helps the driver to quickly regain his current position. The ADS also provides a GPS data record function useful for historic operational data analysis. The entire vehicle data is stored in the central server for further analysis. Safe routes are calculated based on the entire set of historic GPS data using updated mine maps as the main geographic reference (Fig. 6).
Fig. 6 Real-time truck positioning and safe route
4 Conclusions
The Assisted Driving System (ADS) is a reliable safety and warning complement in mining and construction operations. The system is designed to be used by haul truck drivers and mobile equipment operators working under low visibility conditions to provide safer working environments. This system is based on the GPS, mesh-wireless networks, and the Google-Earth (GE) engine as the graphic interface and mine-mapping server. The system has the capability to pin-point and track vehicles in real-time, using the GE-3D interface, based on user-based AutoCAD mine. The ADS system potentially increases safety reliability and reduces decision- making uncertainty when low visibility conditions exist (due to snow, rain, dust, etc.) in open pit mining operations.
References
[1] Nieto A, Dagdelen K. Development of a dump edge and vehicle proximity warning system based on GPS and wireless networks to improve safety in open pit mines.
SME Transactions, 2006(320): 11–20.
[2] Ronald V M, James M H, Shaughnessy M P. Coal Mine Safety and Health Report of Investigation, 2004(9): 1–7. [3] Pyles J, Woodrome M, South J, Zenone M, Blanton A, Fredland J, Skrabak R, Vamossy S, Conrad D. Report of Investigation—Fatal Powered Haulage Accident. Ar-lington, VA: Mine Safety and Health Administration, 2003.
[4] Groves W A, Kecojevic V J, Komljenovic D. Analysis of fatalities and injuries involving mining equipment.
Journal of Safety Research, 2007, 38(4): 461–470. [5] Langley R B. Why is the GPS signal so complex? GPS
World, 1999, 1(3): 56–59.
[6] Ruff T M, Holden T P. Preventing collisions involving surface mining equipment: a GPS-based approach.
Journal of Safety Research, 2003, 34(2): 175–181. [7] Ruff T. Evaluation of a radar-based proximity warning
system for off-highway dump trucks. Accident Analysis and Prevention, 2006(38): 92−98.
[8] Nieto A, Miller S, Miller R. GPS proximity warning system for at-rest large mobile equipment. International Journal of Surface Mining, Reclamation and Environ-ment, 2005, 19(1): 75–84.
