Deep Learning Enhanced Multisensor Data Fusion for Building Assessment Using Multispectral Voxels and Self-Organizing Maps

Abstract

Efforts in the domain of building studies involve the use of a diverse array of geomatic sensors, some providing invaluable information in the form of three-dimensional point clouds and associated registered properties. However, managing the vast amounts of data generated by these sensors presents significant challenges. To ensure the effective use of multisensor data in the context of cultural heritage preservation, it is imperative that multisensor data fusion methods be designed in such a way as to facilitate informed decision-making by curators and stakeholders. We propose a novel approach to multisensor data fusion using multispectral voxels, which enable the application of deep learning algorithms as the self-organizing maps to identify and exploit the relationships between the different sensor data. Our results indicate that this approach provides a comprehensive view of the building structure and its potential pathologies, and holds great promise for revolutionizing the study of historical buildings and their potential applications in the field of cultural heritage preservation.