Written by: Rachel Hottle
The roar of a train, the laughter of
children, the swell of a symphony. All are colorful aspects of daily life that
are communicated to us via our sense of hearing. Hearing is an important asset
that helps us respond to and interact with the world around us. Age-related
hearing loss is a pervasive problem that often affects older adults’ quality of
life. Such hearing loss can make it difficult to perform everyday tasks, such
as determining where a sound is coming from, distinguishing speech from
background noise, and understanding emotions conveyed in speech—all of which
can lead to isolation and depression. Over the last few years, researchers in
the SMART (Science of Music, Auditory Research, and Technology) lab at Ryerson
University have become especially interested in the interaction between hearing
loss and perception of emotion from auditory cues.
|
Hearing is an important asset to help us respond to and interact with the world. |
Perceiving and correctly identifying emotion
in speech is important for interacting with and responding to others, and the
loss of this ability can cause social isolation in older adults. Emotion is
conveyed in speech through features such as pitch contour and dynamic
contrasts. For example, emotions such as anger and fear tend to be conveyed in
a loud voice, while sadness and tenderness are conveyed via softer tones. In
hearing impaired individuals, constrictions in the dynamic range may cause
“loud” emotions to be less well differentiated from “quiet” emotions such as
fear. This dynamic range is often limited in individuals with sensorineural
hearing loss due to a problem known as recruitment. In sensorineural hearing
loss, some of the hair cells in the inner ears die and are no longer able to
convey sound information to the brain. Our brains “recruit” adjacent hair cells
to vibrate at the frequency of the dead hair cells, as well as their original
frequency. This causes the vibration reaching the brain to sound louder than
usual. It can also cause hearing distortion, as hair cells for multiple
frequencies can be vibrating at the same time due to their now double or triple
function. As a result, hearing aids can be uncomfortable for individuals with
severe recruitment.
|
It is important to investigate the effects of hearing aids on perception of emotion in speech. |
To deal with the problem of recruitment,
hearing aids must both amplify quiet sounds and compress loud sounds to make
them less uncomfortable and minimize distortion. Since emotion in speech often
relies on a wide dynamic range, it is important to investigate the effects of
hearing loss as well as hearing aids on perception of emotion in speech. Our
lab has found small, but promising, results regarding the effect of hearing
aids on emotional perception. A study led by PhD student Gabe
Nespoli found that hearing aided individuals have similar emotional responses
to emotional speech as normal hearing individuals (as measured by skin
conductance), but are slower and less accurate at identifying emotions
presented in speech (although still better than hearing impaired individuals
without hearing aids) (Nespoli, Singh, & Russo, 2016). A study conducted by Dr. Huiwen
Goy, a postdoctoral fellow in the lab, found that hearing aids yielded small
but significant improvements in emotion identification in speech compared to hearing
impaired individuals without hearing aids (Goy, Pichora-Fuller, Singh, & Russo, 2016). While hearing aids may not be able
to increase emotion recognition levels to that experienced by normal hearing
listeners, we see encouraging results when compared with individuals with
hearing loss who do not use hearing aids. Nonetheless, there is certainly room
for improvement.
|
Musical emotion differs from spoken emotion in some important ways. |
Although self-report questionnaires that
measure hearing loss and its associated limitations do exist, until recently,
no scale existed to assess the experiences of individuals with hearing loss with
respect to emotional communication. In collaboration with Dr. Gurjit Singh and
Stefan Launer, research scientists at Phonak, we developed the EmoCheQ (Emotional Communication
in Hearing Questionnaire) to address this gap (Singh, Liskovoi, Launer, & Russo, submitted). Our seventeen-item questionnaire
includes questions relating to characteristics of talkers, speech production,
listening situations, and socio-emotional wellbeing. After validating our
scale, we tested it with older adults with normal hearing, hearing aids, and
hearing impairments without hearing aids. We found that, in the domain of
talker characteristics, individuals with normal hearing reported significantly
less handicap than those with hearing aids or hearing loss, while in the domain
of situational factors, there was no difference between the performance of
individuals with normal hearing and those with hearing aids. This provides yet
more evidence that hearing aids in their current form can help with some
features of emotional perception in speech, but not others.
While musical emotions rely on some of the
same prosodic cues as spoken emotions (dynamic range, pitch contour, speed),
musical emotion differs from spoken emotion in some important ways. Emotions in
music are often redundantly coded, meaning that multiple emotional features are
present at once. For example, a “sad” musical passage may be simultaneously
soft, slow, and low in pitch. Since instrumental musical passages are lacking
in the semantic content of emotional speech (the words we speak, which may give
clues to the emotion we are attempting to convey), the emotions presented in
music need to be more exaggerated to be distinguishable to the audience. All
this might mean that it should be easier for hearing impaired and hearing aided
individuals to parse musical emotion compared with spoken emotion. However, it
is also possible that recruitment and dynamic range compression may make it
more difficult for hearing impaired and hearing aided individuals to appreciate
the emotion conveyed by music, which, especially in the classical genre, relies
on a wide dynamic range from pianissimo to fortissimo.
|
Parsing emotion is a key part of speech communication and musical enjoyment |
A new preliminary study led by undergraduate
Domenica Fanelli found promising results regarding the parsing of musical
emotion by hearing aided individuals (Fanelli, Good, & Russo, unpublished). Her study involved the presentation of
music stimuli judged to be either happy, sad, angry, fearful, or tender/calm to
older adults who were either hearing impaired, hearing aided, or had normal
hearing. She found that the hearing aided group was slightly better than the
other two groups at distinguishing low arousal emotions such as sadness and
tenderness, and performed just as well as the normal hearing group at judging
high arousal emotions of happiness, anger, and fear.
While these results are preliminary, it
appears from our recent work that hearing aids are beneficial for those with
hearing loss in understanding emotional cues from speech and music. This is
encouraging, as parsing emotion is a key part of speech communication as well
as musical enjoyment. More research remains to be done to investigate if there
is a way to improve hearing aid technology to further increase emotional
understanding.
References:
Fanelli, D., Good, A., & Russo, F.A. (unpublished). Perception of
Emotion in
Music by Hearing-Impaired and Hearing-Aided
Listeners. Paper
submitted in 2017 for partial fulfillment of
the requirements of an
undergraduate thesis project at Ryerson
University. Toronto, ON.
Goy, H., Pichora-Fuller, M. K., Singh, G., & Russo, F. A. (2016).
Perception
of emotional speech by listeners with hearing
aids. Proceedings of Acoustics
Week in Canada. Canadian Acoustics, 44, 182-183,
Vancouver, BC.
Nespoli, G., Singh, G., & Russo, F.A. (2016). Skin conductance responses
to
emotional speech in hearing-impaired and
hearing-aided listeners.
Proceedings of Acoustics Week in Canada.
Canadian Acoustics, 44,
184-185, Vancouver, BC.
Singh, G., Liskovoi, L., Launer, S., & Russo, F.A (submitted). The
Emotional
Communication in Hearing Questionnaire
(EMO-CHeQ): Development
and validation.
Rachel is a fourth-year undergraduate at Swarthmore College in
Philadelphia, Pennsylvania, where she is studying music and biology.
During summer 2017 she volunteered as a research assistant in the SMART
Lab.
Edited by: Dr. Frank Russo
Formatted by: Fran Copelli
Also published in HearingHealthMatters.org on Tuesday, July 25, 2017.