Visual Information to enhance Time Difference of Arrival Based Acoustic Localization

In this paper, we investigate the problem of reliable detection and localization of active sound source using a new fusion approach of the vision and the acoustic data that enables reducing errors of acoustic localization caused mainly by synchronisation between sensors in addition to reverberation and multipath propagation. In this context, we propose combining the two heterogeneous modalities of data by using the RANSAC technique. This technique allows filtering out erroneous acoustic measurements, as a consequence, only a selected set of inputs are kept are able to fit with time difference of arrival (TDOA) function used to model the geometric relationship between the sound source and the time of detecting of the acoustic events. The proposed solution provides a robust tool for estimating the location of the acoustic source; therefore it enables accurate localization of some targets of interest such as the acoustic moving object. Evaluation results through an implementation version on Wireless Sensors Network (WSN) of the fusion approach confirms the performance improvement in the proposed detection and localization solution. This approach enabled a better detection of the moving acoustic source in comparison with the acoustic information only.