DEMO: multimodal sensing system for hearing enhancement and research

Directivity in hearing aid systems is normally achieved with the use of multiple microphones in an array fashion that aims to enhance signal processing capabilities and performance. Despite the use of binaural information (hearing with two ears), natural hearing systems (e.g. human) also rely on movements of the head to retrieve directional information and the origin of sounds [1]. This kind of natural sensing technique, which combines sound with the movements and position of the body (Fig. 1 & 2) has previously been investigated, and showed encouraging results with improved localization performance [2]. However, the concept has not been explored or exploited yet in any standard commercial hearing aid devices. Creating a hearing aid system that can also include motion sensing would be of great benefit to the hearing impaired in order to enhance directional perception of sound with other functionalities such as frequency selectivity (Fig. 3 & 4) on a wearable hearing device.