Repository hosted by TU Delft Library

Home · Contact · About · Disclaimer ·

NIST and NFI-TNO evaluations of automatic speaker recognition

Publication files not online:

Author: Leeuwen, D.A. van · Martin, A.F. · Przybocki, M.A. · Bouten, J.S.
Institution: TNO Defensie en Veiligheid
Source:Computer Speech and Language, Campell J.P.Mason J.Ortega-Garcia J., 2-3 SPEC. ISS., 20, 128-158
Identifier: 239206
doi: doi:10.1016/j.csl.2005.07.001
Keywords: Acoustics and Audiology · Evaluation · Pattern recognition systems · Speech synthesis · Statistical methods · Automatic speaker recognition · Speech duration · Training handsets · Speech recognition


In the past years, several text-independent speaker recognition evaluation campaigns have taken place. This paper reports on results of the NIST evaluation of 2004 and the NFI-TNO forensic speaker recognition evaluation held in 2003, and reflects on the history of the evaluation campaigns. The effects of speech duration, training handsets, transmission type, and gender mix show expected behaviour on the DET curves. New results on the influence of language show an interesting dependence of the DET curves on the accent of speakers. We also report on a number of statistical analysis techniques that have recently been introduced in the speaker recognition community, as well as a new application of the analysis of deviance analysis. These techniques are used to determine that the two evaluations held in 2003, by NIST and NFI-TNO, are of statistically different difficulty to the speaker recognition systems.