Sensor fusion architectures

Sensor fusion is a very wide topic, treated by multiple disciplines. For this reason, definitions and categorizations are very diverse, depending on the source. This section will detail some of the concepts and divisions related with sensor fusion.

Several types of sensor fusion exists, depending on the concept of the fusion used for the classification, as explained in [24].

Sensor fusion categorization according to the level of abstraction

One of the ways for sensor fusion categorization is depending on the level of abstraction for the fusion, as in [19] and [22].

• Low level sensor fusion

Also called direct fusion or pixel-level fusion, it combines unprocessed data from different sources in order to create a more complex dataset [26], in principle of more quality than the individual inputs. This sensor fusion is dependent on the particular sensors used. An example of this type of fusion are stereoscopic cameras, in which two sensors (monoc-ular cameras) are fused in order to obtain tridimensional information from bidimensional information using the adequate algorithms. This level involves the greatest computational cost, and provides the highest potential detection performance [22].

• Medium level sensor fusion

Also known as characteristics level fusion or feature level fusion, it com-bines edges, corners, lines, textures or positions [26] in a characteristics map, ready of use in segmentation and detection [19]. These charac-teristics are extracted for each individual sensor, combining them later by means of neural networks, state vectors, etc, in a common decision space.

As it is a intermediate level fusion, information from several sensors can be used, and advantage of the possibilities of each of the different sensors can be taken, but detections are presumed to be independent for each sensor. Nevertheless, as pointed in [19], the usual training process in these cases makes the addition of new sensors more difficult, as a new training process with the new characteristics from the new sensors is needed.

• High level sensor fusion

Also called decision level fusion, it combines decisions from the dif-ferent experts involved in the system. This fusion uses voting systems,

2.2 Sensor Fusion 21

fuzzy logic and statistical methods. The final decision is made as a function of the decisions of each of the sensors and its reliability. This type of fusion is less complex, as it is based on previously established subsystems. Two different methods can be used for making classification decisions: hard decisions, that is, the optimum choice, and soft decisions, allowing some level of uncertainty that can be combined in subsequent stages of the fusion process, as in the work [22]. This level of fusion uses generally Bayesian methods [19] [26]

In this case, the purpose of the fusion process is to add reliability to the detections coming from the aforementioned subsystems, obtaining a final combination of these information. An important advantage of this kind of fusion is its scalability, as the addition of new sensors increases the whole system confidence, usually with no complexity addition. This level requires the lowest computational cost of the three levels.

Sensor fusion categorization according to the sensor configuration

Some authors categorize the sensor fusion depending on the type of configu-ration of the sensors involved in the fusion [27] [26], considering the diverse, non excluding possibilities, that can even be found in a hybrid way:

• Complementary sensors

In this case, the sensors are not dependent from each other, but they complement them in order to offer a more complete information of the observed phenomenon. An example is the case of several cameras focusing on disjoint zones in an operations theater [26]. The informa-tion fusion coming from complementary sensors is simple, as the new information is just added to the preexistent.

• Competitive sensors

Competitive sensors are those supplying independent measures of the same object, providing fault tolerance and robustness to the system. This is the case of the so called fault tolerance systems, where the compliance of the standards of service must be ensured even in the case of fail-ure. Alternatively, and as an inferior security level system, competitive sensors allow to offer a degraded behavior in case of failure, adding robustness to the system [27].

• Cooperative sensors

In this case, the fusion uses information from independent sensors in order to obtain results that would not be available in the case of single sensor use, such as in stereoscopic vision. This type of results are the less certain and more difficult to obtain, as they depend directly on the proper functioning of all of the sensors involved. Unlike competitive sensors, these sensors reduce precision and reliability [27].

These three categories of sensor configuration are not mutually exclusive, as diverse hybrid architectures exist, for example where multiple cameras can cover a common area in a competitive or cooperative way, while configuration would be complementary in the areas covered by just one camera.

Sensor fusion categorization according to the point of decision

Topology is a key characteristic in sensor fusion systems, that is, the way how the different nodes communicate and it function in the results delivery. In line with these criteria, some works as in [22], [28] and [29] propose the division of sensor fusion systems in centralized and distributed systems.

2.2 Sensor Fusion 23

• Centralized systems

In this case, all the nodes in the system send the information to a central node, where the final fusion is performed. It is important to note that some aspects of the intermediate sensor fusion may have been performed in a cooperative way in some of the nodes, as is the case when one of the sensors is a stereo camera. In centralized systems, the central node con-centrates all the information from all of the sensors, providing reliability to the system, while the fault tolerance of the system is limited. An ex-ample of this type of systems if shown in [30], where a central node in a vehicle receives information from a differential GPS and inertial sensors, fusing the entire set of information with road maps, in order to obtain a more precise position than the one available using just a differential GPS.

• Distributed systems

Systems where each node performs the fusion in a local way using in-formation from the same node and, in some cases, adjacent nodes, are called distributed systems. A differential GPS fusing by itself informa-tion from its own sensor and the differential system is an example of distributed system. When fusing also information from inertial sensors, it would be considered a distributed multisensor system. These are fault tolerant and easily scalable systems, but the lack of global information reduces the effectiveness of the sensor fusion performed. An example of this strategy can be found in [31].

Figure 9 shows a centralized fusion of the vehicle information, with a central node in charge of the final fusion, opposed to a distributed fusion with local node fusion.

Fig. 9 Centralized vs distributed sensor fusion [32]