Where Sensors and Data Fusion Methods Meet

Author: Liam Critchley

In these days of big data, data originates in multiple sources. To turn this data into useful information it needs to be integrated. Data fusion, or data integration, is a method that integrates a series of data sources, and in turn, provides a much more comprehensive and accurate outlook on a system than a single data source can. Data integration is the third dimension, out of eight dimensions that affect information quality (Kenett and Shmueli, 2016). Other dimensions include data resolution, data structure, temporal relevance and chronology of data and goal. There are different classes of data fusion, but sensors fall under the low-level data fusion, which is the combination of a series of raw data to produce new raw data.

Sensors are becoming smarter, and the recent advances in big data and the creation of industry 4.0 rely on a series of sensors to make processes automated. Behind this network of sensors are a whole host of data algorithms that collate this information together to create usable and monitorable data, minus all the noise that comes with such a vast amount of data.

Applications where sensors meet data fusion are vast, but some of the most common examples include LiDAR sensors in autonomous vehicles, in data processing systems on board different types of satellites and in the detection of remote objects.


So where does data fusion fit in?

There are different sensor environments. Multiple sensor systems that use identical sensors are easy to process and the analysis can be done with little effort. However, it is the systems where there are many different types of sensors in use, which require the data to be formatted into a common form and aligned in the relevant time domain. This is where data fusion is useful.

Data from sensors can either be combined when it arrives into the system, or it can be combined at a designated point within the fusion process. There are wide range of sensors that can be integrated into these processes, from sensors that take images, to measuring relative humidity, to sensing when an object has passed by a given point (and many examples in between).

There are often three different steps in the fusion process, i.e. post-sensing. These are object refinement, situation assessment and threat assessment, and these three steps are refined into a usable output. Object refinement is concerned with aligning the data, data association, object estimation and in identifying objects. The situation assessment turns objects into events by constructing a picture from the data in the first step. The final step, the threat assessment, analyses the advantages and disadvantages of each potential course of action using the built-up picture and data as evidence. The refinement process monitors all these different steps to find ways of enhancing the information and optimising the sensors.

There are currently a wide range of algorithms that can be employed to turn a vast amount of sensor data into coherent usable data. These include intensity-hue-saturation (IHS), high-pass filtering, principal component analysis (PCA), Brovey transform image fusion, pyramid algorithms, continuous wavelet transform (CWT), Bayesian networks (BN) and Artificial Neural Networks (ANNs).

What are the Challenges?

It is of no surprise that trying to coordinate a large sensor network can be problematic, even for the most advanced systems. Because even with the more advanced systems, if anything does go wrong, the algorithms are much more complex to fix. Throughout multi-sensor data fusion process, there are a number of things that can commonly go wrong.

One of the key challenges is imperfect data. Sensors are not perfect and can provide inaccurate data. Data fusion technologies need to be able to understand which data points are correct, and which data points are false readings; this also extends to outlier data points which arise from noise within the measurements. Another key challenge is minimising the noise. Yet another challenge is integrating data with different levels of bias. Special algorithms based on Bayesian networks have been proposed for achieving this. This is especially true for data that is combined from the same environment, where the sensors are exposed to the same noise. In a lot of systems, the same noise is not accounted for and this can sometimes lead to an under/over confidence in the results. However, as sensors and data technologies advance, these challenges will become less prevalent.

Khalegi B. et al, Multisensor data fusion: A review of the state-of-the-art, Information Fusion, 14(1), (2013), 28-44.
Dong J. et al, Advances in Multi-Sensor Data Fusion: Algorithms and Applications, Sensors, 9, (2009), 7771-7784.
Esteban J. et al, A Review of data fusion models and architectures: towards engineering guidelines, Neural Computing & Applications, 14, (2005), 273-281.
Chao W. et al, DATA FUSION, THE CORE TECHNOLOGY FOR FUTURE ON-BOARD DATA PROCESSING SYSTEM, Pecora 15/Land Satellite Information IV/ISPRS Commission I/FIEOS 2002 Conference Proceedings.
Kenett, R.S. and Shmueli, G., Information Quality: The Potential of Data and Analytics to Generate Knowledge, John Wiley and Sons, 2016.
Dalla Valle, L. and Kenett, R.S., Social Media Big Data Integration: A New Approach Based on Calibration, Expert Systems with Applications, 111, (2018). 76–90.
Kedem, B., de Oliveira and Sverchkov, M., Statistical Data Fusion, World Scientific Pub, 2017
Trends in Sensor and Data Fusion: http://www.ifp.uni-stuttgart.de/publications/phowo05/300roth.pdf