Evaluating an ASR Pipeline for a Social Robot

Evaluating an ASR Pipeline for a Social Robot

Sparreboom, Jamey (TU Delft Electrical Engineering, Mathematics and Computer Science)

Scharenborg, Odette (mentor)
Hindriks, Koen (graduation committee)
Oertel genannt Bierbach, Catharine (graduation committee)

Delft University of Technology

2019-08-28

There has been a big increase in the use of social robots, such as Pepper, which use verbal communication as the main method of interacting with a human. Verbal communication with a robot is performed using Automatic Speech Recognition (ASR) to recognize words from an audio stream containing speech.
These social robots are being more frequently used in noisy environments.
As such, this thesis investigates 1) whether Pepper's built-in keyword spotter can be replaced by an ASR system able to recognize continuous speech in Dutch; 2) whether Pepper's ASR pipeline can be made more robust against noise, without changing its hardware.
To that end, Pepper's built-in keyword spotter and Sound Source Localization (SSL) algorithm are evaluated against an ASR pipeline based on a Delay-and-Sum beamformer, MUSIC Sound Source Localization, and Google Cloud Speech-to-Text.

The proposed pipeline showed a significant decrease in Keyword Error Rate of 6.2\% compared to Pepper's built-in keyword spotter, and a significant decrease of Word Error Rate (WER) of 21.4\% on Dutch continuous speech in clean listening conditions. A decrease in WER of 13.3\% was observed in an SNR of 8 dB, and a decrease in WER persisted throughout lower Signal-to-Noise ratios (SNR).

As such, it has been shown that Pepper's speech recognition can be improved and made more robust against noise by preprocessing the audio using MUSIC SSL and a Delay-and-Sum beamformer, and transcribing the speech (in Dutch) using Google Cloud Speech-to-Text.

ASR
Social Robotics
Beamforming

http://resolver.tudelft.nl/uuid:1bb48563-6ef0-4571-aea3-9a0bed34a161

Student theses

master thesis

© 2019 Jamey Sparreboom