A Hybrid Approach to Sign Language Recognition (extended abstract)

Abstract

Methods commonly used for speech and sign language recognition often rely on outputs of Hidden Markov Models (HMM) or Dynamic TimeWarping (DTW) for classification, which aremerely factorized observation likelihoods. Instead, we propose to use Statistical DTW (SDTW) only for warping, while classifying the synchronized features with either of two proposed discriminants. This hybrid approach is shown to outperform HMM and SDTW. However, we have found that combining likelihoods of multiple models in a second classification stage degrades performance of the proposed classifiers, while improving performance with HMM and SDTW. A proof-of-concept experiment, combining DFFM mappings of multiple SDTW models with SDTW likelihoods, shows that also for model-combining, hybrid classification can provide significant improvement over SDTW.