Improving Automatic Speech Recognition for Dutch Children with Developmental Language Disorder Using Cross-Lingual Child-to-Child Voice Conversion

Abstract

Automatic Speech Recognition (ASR) systems perform poorly on speech of children with Developmental Language Disorder (DLD) due to a scarcity of training data and the high variability in speech of these children. This study investigates whether speech created through cross-lingual child-to-child voice conversion (VC) can improve ASR performance for Dutch children with DLD. To this end, multiple data augmentation pipelines were developed using the MeanVC zero-shot VC model, generating synthetic child speech with different source-target speaker pairing strategies (most similar, ran- dom, least similar) and varying augmentation sizes. The augmented datasets were used to fine-tune a Whisper-based ASR model. The results show that fine-tuning the ASR on a combination of synthetic speech and source speech improved ASR performance over an out-of-the-box baseline, but did not lead to better performance than fine-tuning it on real speech of children with DLD alone. Interestingly, speaker-dissimilar VC pairings yielded the best ASR performance. Subjective Mean Opinion Score (MOS) evaluations suggest that synthetic speech was perceived as less natural, less intelligible, and less child-like than real speech and that AMOS predictions only partially align with human judgments and ASR performance. These findings suggest that while cross-lingual voice-converted speech can have value as supplementary training data, improvements in speech quality and voice conversion methods are needed before it can serve as an effective replacement for real speech of a child with DLD in ASR training.