Relative observations from the aforementioned sensors can provide sufficient accuracy, but they contain shortcomings such as a constrained FOV, limited range, susceptibility to dy- namic lighting, and susceptibility to clutter. Information from sensors with complimentary characteristics are often combined, to obtain a more complete description of the quantity of interest while negating individual shortcomings. This process is known as multi-sensor data fusion.
A popular data fusion algorithm is the recursive Bayesian estimator, known as the Kalman filter. It provides the optimal state estimate for a linear system that is corrupted by Gaus- sian noise with known covariance. Application of the Kalman filter has been extended to nonlinear systems by the aptly name extended Kalman filter (EKF), which linearises about an estimate of the current mean and covariance. A more recent variation on the Kalman filter, known as the UKF [41], deterministically samples a set of points about the mean, then propagates these sigma points through the nonlinear function, from which the mean and covariance are recovered. The UKF addresses some of the limitations of the EKF by capturing higher-order nonlinearity and avoiding the derivation or numerical computation of Jacobians. Another data fusion algorithm, known as the particle filter, takes the idea of the UKF further by propagating a large number of randomly selected points through the nonlinear function. This means that the particle filter can represent any distribution, even multi-modal, at the expense of computational effort.
2.2.1 Airborne Relative Navigation
Due to the high nonlinearity of the relative navigation problem and constraints due to limited onboard processing, estimation techniques have generally used the EKF or UKF.
2.2. Multi-Sensor Data Fusion 15 Despite the UKF being touted as having better convergence properties and improved han- dling of nonlinearity over the EKF, the differences are usually small so the decision is often personal preference. The more important aspect is the structure of the filter, the pro- cess models, the observation models and the handling of sensor measurements to ensure reliability. The specific estimation algorithm simply provides the framework.
The structure of a visual-GNSS/INS filter can be categorised as loosely coupled or tightly coupled. Loosely coupled methods extend vision-only algorithms by fusing a vision-only rel- ative pose estimate with GNSS and inertial data from one or more vehicles [9, 52, 54]. These methods add reliability through extra information, but are constrained by the requirements of the vision-only algorithm, such as a minimum number of feature points. Further, the measurement covariance depends on the dynamic relative state, as well as sensor noise. A tightly coupled visual-GNSS/INS architecture fuses the raw observations directly, rather than preprocessing them in a separate algorithm [30, 59, 66, 93]. This approach allows vi- sual information to be used, even when the complete 6DOF relative pose cannot be uniquely resolved from the visual measurements. An example is the CORSE algorithm in [39] where azimuth and elevation measurements to a single point can provide relative positioning infor- mation but no notion of relative attitude. Work in [66] used these centre-only measurement as well as the line segment joining the wingtips of the target to provide information about target size. These observations were fused with onboard inertial and GNSS measurements within a UKF and successfully verified by post-processing flight data and comparing to the differenced GNSS/INS estimate. Avoiding communications and a priori knowledge of the target is desirable in a military context but foregoes reliability and necessitates assump- tions about target dynamics. Communications were allowed in [93], where an indirect EKF fused both leader and follower GNSS/INS sensor data with a marker based vision system. High-fidelity hardware-in-the-loop simulation yields estimator accuracies that are sufficient for large-scale boom-based aerial docking.
2.2.2 Drogue Motion Estimation
Accurate knowledge of the state of the drogue, relative to the follower, is essential for docking. This is a challenging problem because the drogue adds an extra layer of dynamics to an already highly dynamic problem. The propagation of the follower-drogue relative
state is particularly nonlinear, because it is the superposition of three bodies that are each susceptible to disturbances. Although there are many other ways to dock two aircraft, this work considers the existing probe and drogue technique because reuse of hardware and processes is the natural way to introduce the technology.
Approaches to drogue relative navigation follow that of aircraft relative navigation. Chen and Stettner [10] used flash LIDAR 2D intensity to segment the drogue, then calculated the relative pose using the 3D point cloud. Monocular template matching, image segmentation, and image registration were used to detect and track a drogue in [57]. Known geometric characteristics of the drogue and the contrast of the hub were used in [71] to resolve the range. Marker based methods using VisNav have also been used [59, 85].
As with all vision-only relative navigation, the entire system is at the mercy of the inherently inconsistent relative observations which often dropout [80]. This can normally be partially alleviated by instrumenting the target, however, drogues are not often instrumented (with the exception of [64]). Instead, our approach is to use known geometric and inertial charac- teristics of the system, knowledge of the leader’s state, and a dynamic model for the drogue that propagates the system. When air-to-air observations are available, the state of the drogue is corrected, and parameters within the model are refined.
The dynamic model assumes the tether is a rigid member because the tether used in this work has negligible mass and aerodynamic drag, but this assumption does not always hold. Numerous methods of varying fidelity and complexity have been proposed to model the hose. A common approach is to approximate the bending structure with n connected linkages, each subject to gravitational and aerodynamic forces due to tanker wake, steady wind and atmospheric turbulence [68, 86]. Regardless of the number of linkages used, experiments with an instrumented drogue in [76] showed that the steady state of a drogue is most sensitive to the tether length, drag coefficient, and drogue mass. We therefore estimate the tether length and drag coefficient online using the air-to-air vision and measure the drogue’s mass offline.
As a difficult problem practically, experiments with drogue estimation in-flight have been limited. An exception is the first automated docking with manned aircraft [21]. During the final approach, feedback was switched from the precision GNSS/INS system to azimuth, elevation and range measurements to the drogue, from a camera and off-the-shelf video tracking processor. Work in [64] used LOS visual feedback to the drogue centroid to attempt
2.3. Relative Guidance 17