Automatic Cattle Lameness Assessment with Markerless Cattle Pose Estimation

Abstract

Lameness is characterized by abnormal gait and is an indicator of various hoof diseases in cattle. Not only does this raise animal welfare issues, it also causes significant economic loss from reduced milk yield and fertility. Despite that, prevalence of lameness in dairy farms is high because farmers are unable to dedicate time and labor to identify lame cattle. Many automatic lameness detection solutions have been proposed in literature. Machine vision solutions using cameras are especially attractive because cameras do not require much space and is relatively low cost. However, none of the machine vision solutions so far have been robust enough to be useful to dairy farmers. This thesis attempts to remedy that by applying deep learning methods to the pose estimation of cattle to analyze their gait and detect the presence of lameness.
313 videos of cattle walking were recorded at a dairy farm. Images were randomly extracted from those videos and 17 body parts were manually annotated on the images to fine-tune a deep neural network pretrained on ImageNet. The fine-tuned network is then used to automatically find the trajectories of the 17 body parts in all 313 videos. 84 gait features were extracted from each video based on these trajectories. Each video was also manually given a locomotion score between 1-5 by 2 experts. Due to the small number of locomotion score 5 cows, locomotion score 4 and 5 were merged into one group for analysis. Two experiments were conducted with these gait features and locomotion scores: a) Data analysis to test significant differences between locomotion score groups and b) Automatic locomotion score classification.
Data analysis was done using ANOVA followed by Bonferroni correction. Stance time related features were the best at differentiating locomotion score groups, but were unable to differentiate between locomotion score pair 2<>3. Step length related features were also relatively good at differentiating different locomotion score groups, but have trouble differentiating between locomotion score pairs 1<>2 and 2<>3.
For the automatic classification, the 84 gait features were first reduced to 3 features using LDA. Then, various classifiers were trained with these 3 features and locomotion score as labels. The linear discriminant classifier achieved the highest classification rate at 85.6%, but this number was heavily skewed by the high classification rate of locomotion score 1 group (95.3%), which also makes up the largest portion of the dataset. To correct this imbalance in the dataset, the prior probabilities were set equal for each locomotion score group and the classifiers were trained again. This resulted in much better classification rates with the linear discriminant classifier for locomotion score 2 and 3 (71% and 84% respectively) at the expense of lowering the classification rate of locomotion score 1 (83.4%).