Starkey Research & Clinical Blog

Do hearing aid wearers benefit from visual cues?

Wu, Y-H. & Bentler, R.A. (2010) Impact of visual cues on directional benefit and preference: Part I – laboratory tests. Ear and Hearing 31(1), 22-34.

This editorial discusses the clinical implications of an independent research study. The original work was not associated with Starkey Laboratories and does not reflect the opinions of the authors. 

The benefits of directional microphone use have been consistently supported by experimental data in the laboratory (Valente et al. 1995; Ricketts & Hornsby 2006; Gravel et al. 1999; Kuk et al. 1999). Similarly, hearing aid users have indicated a preference for directional microphones over omnidirectional processing in noise in controlled environments (Preves et al. 1999; Walden et al. 2005; Amlani et al. 2006). Despite the robust directional benefit reported in laboratory studies, field studies have yielded less impressive results; with some studies reporting perceived benefit (Preves et al. 1999; Ricketts et al. 2003) while others have not (Walden et al. 2000; Cord et al. 2002, 2004; Palmer et al. 2006).

One factor that could account for reduced directional benefit reported in field studies is the availability of visual cues. It is well established that visual cues, including lip-reading (Sumby & Pollack 1954) as well as eyebrow (Bernstein et al. 1989) and head movements (Munhall et al. 2004), can improve speech recognition ability in the presence of noise. In field studies, the availability of visual cues could result in a decreased directional benefit due to ceiling effects. In other words, the benefit of audio-visual (AV) speech cues might result in omnidirectional performance so close to a listener’s maximum ability that directionality may offer only limited additional improvement.  This could reduce both measured and perceived directional benefits.  It follows that ceiling effects from the availability of AV speech cues could also reduce the ability of auditory-only (AO) laboratory findings to accurately predict real-world performance.

Few studies have investigated the effect of visual cues on hearing aid performance or directional benefit. Wu and Bentler’s goal in the current study was to determine if visual cues could partially account for the discrepancy between laboratory and field studies of directional benefit. They outlined three experimental hypotheses:

1. Listeners would obtain less directional benefit and would prefer directional over omnidirectional microphone modes less frequently in auditory-visual (AV) conditions than in auditory-only (AO) conditions.

2. The AV directional benefit would not be predicted by the AO directional benefit.

3. Listeners with greater lip-reading skills would obtain less AV directional benefit than would listeners with lesser lip-reading skills.

Twenty-four adults with hearing loss participated in the study. Participants were between the ages of 20-79 years, had bilaterally symmetrical, downward-sloping, sensorineural hearing losses, normal or corrected normal vision and were native English speakers. Participants were fitted with bilateral, digital, in-the-ear hearing instruments with manually-accessible omnidirectional and directional microphone modes.

Directional benefit was assessed with two speech recognition measures:  the AV version of the Connected Speech Test (CST; Cox et al., 1987) and the Hearing in Noise Test (HINT; Nilsson et al., 1994). For the AV CST the talker was displayed on a 17” monitor. Participants listened to sets of CST sentences again in a second session to evaluate subjective preference for directional versus omnidirectional microphone modes. Speech stimuli were presented in six signal-to-noise (SNR) conditions ranging from -10dB to +10dB in 4db steps. Lip-reading ability was assessed with the Utley test (Utley, 1946), an inventory of 31 sentences recited without sound or facial exaggeration.

Analysis of the CST scores yielded significant main effects for SNR, microphone mode and presentation mode (AV vs. AO) as well as significant interactions among the variables. The benefit for visual cues was greater than the benefit afforded by directionality. As the authors expected, for most SNRs the directional benefit was smaller for AV conditions than AO conditions with the exception of the poorest SNR condition, -10dB.  Scores for all conditions (AV-DIR, AV-OMNI, AO-DIR, AO-OMNI) plateau at ceiling levels for the most favorable SNRs; meaning that both AV benefit and directional benefit decreased as SNR improved to +10dB.  HINT scores, which did not take into account ceiling effects, yielded a significant mean directional benefit of 3.9dB.

Participants preferred the directional microphone mode in the AO condition, especially at SNRs between -6dB to +2dB. At more favorable SNRs, there was essentially no preference. In the AV condition, participants were less likely to prefer the directional mode, except at the poorest SNR, -10dB. Further analysis revealed that the odds of preferring directional mode in AO condition were 1.37 times higher than in the AV condition. In other words, adding visual cues reduced overall preference for the directional microphone mode.

At intermediate and favorable SNRs there was no significant correlation between AV directional benefit and the Utley lip-reading scores. For unfavorable SNRs, the negative correlation between these variables was significant, indicating that in the most difficult listening conditions, listeners with better lip-reading skills obtained less AV directional benefit than those participants who were less adept at lip-reading.

The outcomes of these experiments generally support the authors’ hypotheses. Visual cues significantly improved speech recognition scores in omnidirectional trials close to ceiling levels, reducing directional benefit and subjective preference for directional microphone modes.  Auditory-only (AO) performance, typical of laboratory testing, was not predictive of auditory-visual (AV) performance. This is in agreement with prior indications that AO directional benefit as measured in laboratory conditions doesn’t match real-world directional benefit and suggests that the availability of visual cues can at least partially explain the discrepancy.  The authors suggested that directional benefit should theoretically allow a listener to rely less on visual cues. However, face-to-face conversation is natural and hearing-impaired listeners should leverage avoid visual cues when they are available.

The results of Wu and Bentler’s study suggest that directional microphones may provide only limited additional benefit when visual cues are available, for all but the most difficult listening environments. In the poorest SNRs, directional microphones may be leverages for greater benefit.  Still, the authors point out that mean speech recognition scores were best when both directionality and visual cues were available. It follows that directional microphones should be recommended for use in the presence of competing noise, especially in high noise conditions. Even if speech recognition ability is not significantly improved with the use of directional microphones in many typical SNRs, there may be other subjective benefits to directionality, such as reduced listening effort, distraction or annoyance that listeners respond favorably to.

It is important for clinicians to prepare new users of directional microphones to have realistic expectations. Clients should be advised that directionality can reduce competing noise but not eliminate it. Hearing aid users should be encouraged to consider their positioning relative to competing noise sources and always face the speech source that they wish to attend to.  Although visual cues appear to offer greater benefits to speech recognition than directional microphones alone; the availability of visual speech cues may be compromised by poor lighting, glare, crowded conditions or visual disabilities, making directional microphones all the more important for many everyday situations. Thus all efforts should be made to maximize directionality and the availability of visual cues in day-to-day situations, as both offer potential real-world benefits.

References

Amlani, A.M., Rakerd, B. & Punch, J.L. (2006). Speech-clarity judgments of hearing aid processed speech in noise: differing polar patterns and acoustic environments. International Journal of Audiology 12, 202-214.

Bernstein, L.E., Eberhardt, S.P. & Demorest, M.E. (1989). Single-channel vibrotactile supplements to visual perception of intonation and stress. Journal of the Acoustical Society of America 85, 397-405.

Cord, M.T., Surr, R.K., Walden, B.E., et al. (2002). Performance of directional microphone hearing aids in everyday life. Journal of the American Academy of Audiology 13, 295-307.

Cord, M.T., Surr, R.K., Walden, B.E., et al. (2004). Relationship between laboratory measures of directional advantage and everyday success with directional microphone hearing aids. Journal of the American Academy of Audiology 15, 353-364.

Cox, R.M., Alexander, G.C. & Gilmore, C. (1987). Development of the Connected Speech Test (CST). Ear and Hearing 8, 119S-126S.

Gravel, J.S., Fausel, N., Liskow, C., et al. (1999). Children’s speech recognition in noise using omnidirectional and dual-microphone hearing aid technology. Ear and Hearing 20, 1-11.

Kuk, F., Kollofski, C., Brown, S., et al. (1999). Use of a digital hearing aid with directional microphones in school-aged children. Journal of the American Academy of Audiology 10, 535-548.

Lee L., Lau, C. & Sullivan, D. (1998). The advantage of a low compression threshold in directional microphones. Hearing Review 5, 30-32.

Leeuw, A.R. & Dreschler, W.A. (1991). Advantages of directional hearing aid microphones related to room acoustics. Audiology 30, 330-344.

Munhall, K.G., Jones, J.A., Callan, D.E., et al. (2004). Visual prosody and speech intelligibility: head movement improves auditory speech perception. Psychological Science 15, 133-137.

Nilsson, M., Soli, S. & Sullivan, J.A. (1994). Development of the Hearing in Noise Test for the measurement of speech reception thresholds in quiet and in noise. Journal of the Acoustical Society of America 95, 1085-1099.

Palmer, C., Bentler, R., & Mueller, H.G. (2006). Evaluation of a second order directional microphone hearing aid: Part II – Self-report outcomes. Journal of the American Academy of Audiology 17, 190-201.

Preves, D.A., Sammeth, C.A. & Wynne, M.K. (1999). Field trial evaluations of a switched directional/omnidirectional in-the-ear hearing instrument.  Journal of the American Academy of Audiology 10, 273-284.

Ricketts, T. (2000b). Impact of noise source configuration on directional hearing aid benefit and performance. Ear and Hearing 21, 194-205.

Ricketts, T. & Hornsby, B.W. (2003). Distance and reverberation effects on directional benefit. Ear and Hearing 24, 472-484.

Ricketts, T. & Hornsby, B.W. (2006). Directional hearing aid benefit in listeners with severe hearing loss. International Journal of Audiology 45, 190-197.

Ricketts, T., Henry, P. & Gnewikow, D. (2003). Full time directional versus user selectable microphone modes in hearing aids. Ear and Hearing 24, 424-439.

Sumby, W.H. & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America 26, 212-215.

Valente, M., Fabry, D.A. & Potts, L.G. (1995). Recognition of speech in noise with hearing aids using dual microphones. Journal of the American Academy of Audiology 6, 440-449.

Walden, B.E., Surr, R.K., Cord, M.T., et al. (2000). Comparison of benefits provided by different hearing aid technologies. Journal of the American Academy of Audiology 11, 540-560.

Walden, B.E., Surr, R.K . Grant, K.W., et al. (2005). Effect of signal-to-noise ratio on directional microphone benefit and preference. Journal of the American Academy of Audiology 16, 662-676.

Wu, Y-H. & Bentler, R.A. (2010) Impact of visual cues on directional benefit and preference: Part I – laboratory tests. Ear and Hearing 31(1), 22-34.