Repository hosted by TU Delft Library

Home · Contact · About · Disclaimer ·

User evaluation of a communication system that automatically generates captions to improve telephone communication

Publication files not online:

Author: Zekveld, A.A. · Kramer, S.E. · Kessens, J.M. · Vlaming, M.S.M.G. · Houtgast, T.
Institution: TNO Defensie en Veiligheid
Source:Trends in Amplification, 1, 13, 44 - 68
Identifier: 181173
doi: doi:10.1177/1084713808330207
Keywords: Acoustics and Audiology · Assistive text display · Automatic speech recognition · Communication device for hearing impaired · User evaluation · accuracy · adult · aged · article · assistive technology · auditory stimulation · automatic speech recognition · clinical article · controlled study · female · follow up · hearing impairment · human · information processing · Internet · low frequency noise · male · rating scale · simulation · speech discrimination · task performance · telecommunication · telephone · Adult · Aged · Aged, 80 and over · Cognition · Communication Aids for Disabled · Comprehension · Computer Systems · Female · Hearing Loss, Mixed Conductive-Sensorineural · Hearing Loss, Sensorineural · Humans · Male · Memory · Middle Aged · Noise · Perceptual Masking · Questionnaires · Rehabilitation of Hearing Impaired · Speech Perception · Speech Reception Threshold Test · Speech Recognition Software · Telephone · Time Factors · Visual Perception · telephone · communication


This study examined the subjective benefit obtained from automatically generated captions during telephone-speech comprehension in the presence of babble noise. Short stories were presented by telephone either with or without captions that were generated offline by an automatic speech recognition (ASR) system. To simulate online ASR, the word accuracy (WA) level of the captions was 60% or 70% and the text was presented delayed to the speech. After each test, the hearing impaired participants (n = 20) completed the NASA-Task Load Index and several rating scales evaluating the support from the captions. Participants indicated that using the erroneous text in speech comprehension was difficult and the reported task load did not differ between the audio + text and audio-only conditions. In a follow-up experiment (n = 10), the perceived benefit of presenting captions increased with an increase of WA levels to 80% and 90%, and elimination of the text delay. However, in general, the task load did not decrease when captions were presented. These results suggest that the extra effort required to process the text could have been compensated for by less effort required to comprehend the speech. Future research should aim at reducing the complexity of the task to increase the willingness of hearing impaired persons to use an assistive communication system automatically providing captions. The current results underline the need for obtaining both objective and subjective measures of benefit when evaluating assistive communication systems. © 2009 SAGE Publications.