Multiple Sound Source Localization Exploiting Robot Motion and Approaching Control

Sound source localization (SSL) and approaching are essential capabilities for robots with auditory sensing. However, most existing methods for SSL only provide the direction of arrival (DoA) of the source, without the source distance, and cannot cope with time-varying number of sound sources. In this article, a novel framework that integrates multi source state estimation and source approaching control is proposed to address these issues. First, auditory probability hypothesis density (A-PHD) method is proposed, which can estimate both the source direction and distance by leveraging the robot motion information along with DoA estimation. A-PHD designs a new state update and merge strategy based on the characteristics of auditory perception, enabling more accurate source number estimation and real-time performance improvement. Second, a new source approaching control method aiming at both improving SSL accuracy and approaching the source is proposed, which utilizes the entropy to quantify the uncertainty of SSL. This method establishes the explicit relationship between the entropy and the robot motion, achieving efficient and smooth source approaching while improving SSL accuracy. Third, based on the methods proposed above, an organic framework consisting of source localization and approaching is formed. In this framework, the results of A-PHD serve as input of the control method, and in turn, the control method utilizes certain robot motions to improve the accuracy of A-PHD and obtains more precise source locations as input. In this process, more accurate SSL and more efficient source approaching are achieved. A series of experiments such as multisource state estimation with various DoA errors, robot motions, missing and false measurements, and source approaching are conducted on a mobile robot with a six-microphone array, verifying the effectiveness of our methods.