CNN-Based Assistive Technology Platform for Hearing Impairments Individuals

Being conscious of what goes on around a person's surrounding highly depends on the person's ability to perceive and identify sounds. However, hearing impairment can affect the ability of a person to obtain situational awareness and can interfere with their everyday life. There are numerous technologies available, ranging from body-worn and implant devices to devices which alerts the deaf and hard-of-hearing. As beneficial as they are, they are also expensive and cannot treat all types of hearing loss. In this paper, we propose Digital Assistance System for Deaf and Hard-of-Hearing (DHH) People which is essentially a mobile application that will help people with hearing impairment be aware about the sounds in their domestic environment. Audio samples from a benchmark dataset are converted to spectrograms on which a Convolutional Neural Networks (CNN) model is trained on. The model is deployed in a mobile application and the app displays the predicted class the sound belongs to.