Starkey Research & Clinical Blog

Award-Winning Study: Perceptual and Neural Representation of Consonants in Hearing Impaired Listeners

In 2017, Yaqing Su, a summer intern with Starkey Hearing Technologies studied the perception of spoken consonants by hearing impaired listeners at the Acoustical Society of America. Su’s research was selected for a special session titled “Acoustics Outreach: Planting Seeds for Future Clinical and Physiological Collaborations ‘18” in the spring 2018 meeting of the Acoustical Society of America. We caught up with Su’s mentor Jayaganesh Swaminathan to talk about the study and its importance.

Background on the issue of perception

In order to perceive meaningful speech, the auditory system must recognize different consonants amidst a noisy and variable acoustic signal. Difficulties in perception of consonants have often been associated with speech perception challenges for people with hearing loss. For example, imagine a retired couple conversing over their favorite television program. The wife tells her husband, “I think you need a hearing test.” The husband responds angrily, “Why the heck do I need a hairy chest?” This scenario demonstrates the classic problem people with hearing loss face: mishearing a few consonants in noise may lead to a completely different interpretation of the original message.

Part of the challenge in perceiving consonants correctly arises from the impoverished representation of consonants in the auditory pathway (also referred to as neural mechanisms) due to hearing loss.

A quick overview of the study

The overall goals of this study were to use surface level electro-encephalography (EEG) recordings to get a better understanding of the neural mechanisms that contribute to the difficulties in consonant perception in hearing impaired listeners.

Both perceptual and EEG responses were measured for a set of nonsense vowel-consonant-vowel (VCV) speech in listeners with moderate hearing loss. Responses were measured in unamplified and amplified conditions. A state-of-the-art machine-learning classifier was trained to discriminate the EEG signal evoked by each consonant. The performance of the classifier was compared to the listeners’ psychophysical performance.

For all the listeners, performance in the perceptual discrimination task was poor, even with sound amplification.[ Referring to “sound amplification”: Not with hearing aids but just with linear gain amplification. The amplification was provided to the speech stimuli based on a listeners’ audiogram and the amplified stimuli were presented over headphones. The linear amplification “mimics” the sound amplification provided by hearing aids.] EEG waveforms from the listeners also showed different patterns of responses for each consonant. A simple classifier could decode consonants from the EEG signal. For certain consonant categories, a relationship between the neural and perceptual representation of consonants could be established in the listeners. The results from this study have implications for technologies aiming to use neural signals to guide hearing aid processing.

Talking with Starkey Research Scientist Jayaganesh Swaminathan about the value of this study

Starkey Hearing Technologies: This study is incredible. In your opinion, what specifically influenced ASA to honor Su with an award for it?
Jayaganesh Swaminathan: This is really a beautiful study as it had a physiology component, a psychophysical component and a quantitative component in addressing speech perception difficulties in hearing impaired listeners. Su’s project at Starkey is highly translational in nature and lays a solid scientific foundation to develop improved hearing aid fitting and speech enhancement strategies to improve the lives of people with hearing loss.

SHT: Why was this study so important to conduct?
JS: There has been a recent surge in interest in developing hearing aids that incorporate EEG signals as an input to guide signal processing to facilitate robust speech perception in the hearing impaired (HI). However, the vast majority of research on this topic has been conducted in listeners with normal hearing and there is a paucity of EEG data from HI listeners. The results from Yaqing’s study is critical to advance both basic and applied research aimed at using EEG signatures to better understand the effects of hearing impairment on neural coding of speech and to design better hearing aid technologies.

SHT: What is the value of the data from this study for today’s hearing aid technologies?
JS: First and foremost, the data from this study establishes the feasibility of recording meaningful EEG signals in response to consonants from HI subjects that are relatable to their perceptual responses. This is encouraging for research efforts aimed at developing speech enhancement strategies in hearing aids based on signals measured from the brain. I do not think we are (technologically) ready to right away transfer the knowledge from Yaqing’s study to improve current hearing aid technologies yet. For example, we still do not have the technology to reliably record EEG signals from a hearing aid.

SHT: What are the implications of this study for future efforts to improve speech enhancement?
JS: In my opinion, this is the first step in a long chain of events which may one day lead to the development of hearing aid technologies that can be modulated based on neural responses.

SHT: What do you feel is the next step along this research pathway?
JS: Here are a few immediate research steps to follow through, although this list is not exhaustive:
1) Replication with a larger HI population
2) Measure EEG responses to consonants with more complex speech sentences across different listening environments
3) Study the effects of some of the hearing aid signal processing components (such as compression, noise reduction, etc.) on the fidelity of the neural signals measured in response to speech

 

 

 

 

 

Starkey Hearing Technologies helping to shape the ears, eyes, and minds of elderly Europeans

Starkey Hearing Technologies is the only American partner, and the only member of the hearing industry, to participate in an ambitious 6.5€ million European project that aims to improve the mental well-being of numerous elderly Europeans.

The project, known as SENSE-Cog, is designed to improve understanding of how sensory impairments relate to cognitive and mental health. The European Union has funded this project with the goal being to translate new knowledge of the interrelationships of sensory and cognitive health into daily clinical practice, policy and community impact. The SENSE-Cog project is working on this multi-faceted goal via the following efforts: (1) the creation of clinical tools, (2) the development and deployment of approaches for raising public awareness, and (3) the documentation of the economic impact of sensory and cognitive impairment.

“Amongst the project outcomes targeting daily clinical practice, a crucial one is the design and testing of an intervention protocol to support vision and hearing function of elderly dementia sufferers,” a Starkey Hearing Research Center Director, said. “And this is where Starkey comes in.”

The sensory support intervention is a centerpiece of the project. “A well-designed and successful sensory support intervention protocol has the potential to greatly impact the cognitive well-being of many elderly people and thereby significantly reduce one of the top 10 public health burdens of Europe,” he said. “Starkey has played a crucial role in both the specification of the sensory support intervention and in the design of the research study to determine the efficacy of the intervention.”

The SENSE-Cog team originally called upon the world-leading expertise of Starkey Research in the clinical deployment of hearing aids and in the design of clinical trials of hearing aid interventions. To help with the SENSE-Cog project, the Starkey team will be focused on deploying randomized controlled trials addressing the efficacy of the sensory support intervention over the next few months. These trials will be conducted across five different countries and involve an estimated 200 patients. The trials, the Starkey team said great promise of yielding a wealth of knowledge that could impact the care of elderly people facing sensory and cognitive challenges worldwide.

But, Starkey’s important contributions in this research project extend beyond the design of the sensory support intervention and its testing. The Starkey research team will also be helping with other practical elements of project deployment. Specifically, our partners at Starkey UK have already provided hearing aids for use in the research study and have also given expert guidance to the project on proper hearing aid usage and deployment.

The SENSE-Cog project is currently still in continuation.