The representation of speech and its processing in the human brain and deep neural networks

The representation of speech and its processing in the human brain and deep neural networks

Scharenborg, O.E. (TU Delft Multimedia Computing)

Salah, Albert Ali (editor)
Karpov, Alexey (editor)
Potapova, Rodmonga (editor)

2019

For most languages in the world and for speech that deviates from the standard pronunciation, not enough (annotated) speech data is available to train an automatic speech recognition (ASR) system. Moreover, human intervention is needed to adapt an ASR system to a new language or type of speech. Human listeners, on the other hand, are able to quickly adapt to nonstandard speech and can learn the sound categories of a new language without having been explicitly taught to do so. In this paper, I will present comparisons between human speech processing and deep neural network (DNN)-based ASR and will argue that the cross-fertilisation of the two research fields can provide valuable information for the development of ASR systems that can flexibly adapt to any type of speech in any language. Specifically, I present results of several experiments carried out on both human listeners and DNN-based ASR systems on the representation of speech and lexically-guided perceptual learning, i.e., the ability to adapt a sound category on the basis of new incoming information resulting in improved processing of subsequent speech. The results showed that DNNs appear to learn structures that humans use to process speech without being explicitly trained to do so, and that, similar to humans, DNN systems learn speaker-adapted phone category boundaries from a few labelled examples. These results are the first steps towards building human-speech processing inspired ASR systems that, similar to human listeners, can adjust flexibly and fast to all kinds of new speech.

Adaptation
Deep neural networks
Human speech processing
Non-standard speech
Perceptual learning
Speech representations

http://resolver.tudelft.nl/uuid:3b886ba2-1c3e-4ff2-b91c-7ff32e2bcf3c

https://doi.org/10.1007/978-3-030-26061-3_1

Springer, Cham

978-3-030-26060-6

Speech and Computer: 21st International Conference, SPECOM 2019, Proceedings

SPECOM 2019, 2019-08-20 → 2019-08-25, Istanbul, Turkey

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 0302-9743, 11658 LNAI

Institutional Repository

conference paper

© 2019 O.E. Scharenborg