Sign Language Recognition by Combining Statistical DTW and Independent Classification

Sign Language Recognition by Combining Statistical DTW and Independent Classification

Lichtenauer, J.F.
Hendriks, E.A
Reinders, M.J.T.

Electrical Engineering, Mathematics and Computer Science

Mediamatics

2008-05-13

To recognize speech, handwriting, or sign language, many hybrid approaches have been proposed that combine Dynamic Time Warping (DTW) or Hidden Markov Models (HMMs) with discriminative classifiers. However, all methods rely directly on the likelihood models of DTW/HMM. We hypothesize that time warping and classification should be separated because of conflicting likelihood modeling demands. To overcome these restrictions, we propose using Statistical DTW (SDTW) only for time warping, while classifying the warped features with a different method. Two novel statistical classifiers are proposed—Combined Discriminative Feature Detectors (CDFDs) and Quadratic Classification on DF Fisher Mapping (Q-DFFM)—both using a selection of discriminative features (DFs), and are shown to outperform HMM and SDTW. However, we have found that combining likelihoods of multiple models in a second classification stage degrades performance of the proposed classifiers, while improving performance with HMM and SDTW. A proof-of-concept experiment, combining DFFM mappings of multiple SDTW models with SDTW likelihoods, shows that, also for model-combining, hybrid classification can provide significant improvement over SDTW. Although recognition is mainly based on 3D hand motion features, these results can be expected to generalize to recognition with more detailed measurements such as hand/body pose and facial expression.

time series analysis
face and gesture recognition
3D/stereo scene analysis
statistical dynamic programming
Markov processes
classifier design and evaluation
real-time systems

http://resolver.tudelft.nl/uuid:a3e063bf-742b-4a86-b952-93d51be0b775

IEEE

0162-8828

IEEE Transactions on Pattern Analysis and Machine Intelligence, 30 (11), 2008

Institutional Repository

journal article

(c) 2008 Lichtenauer, J.F. ; Hendriks, E.A. ; Reinders, M.J. ; IEEE