Improving robustness of pose estimation in AR using motion segmentation

Abstract

Recent improvements in mobile technology have allowed computationally intensive augmented reality systems to become increasingly ubiquitous. As will be shown in this thesis an increasing number of companies have started developing markerless AR systems that can be used professionally. Robust pose estimation is essential for such systems to convey users into a believable AR experience by excluding outlying and erroneous tracked features. Robust statistics can detect and reject large percentages of outliers but in highly dynamic scenes more dedicated motion segmentation is required. In this thesis a motion segmentation algorithm is presented that is designed to increase robustness in a vision based marker-less stereo AR framework, to improve pose estimation and mapping. This is based on a motion segmentation method that has successfully been tested on PTAM [1]. We believe that by expanding this to stereo vision, improvements can be made by utilizing the extra visual information obtained by the second camera. By modeling the stereo triangulation error we could use the Mahalanobis distance for clustering features and correct for their error distribution, which increases with distance from the camera. Initial visual inspection showed that this improved the dynamic feature rejection with respect to the Euclidean distance and with that also the accuracy of pose estimation. This is shown in the test results. By using the 3D distribution in the scoring of each set, the correct labeling of a set of clustered features as being static was improved. This is because static features are more likely to be arbitrarily spread over the 3D scene while dynamic features belonging to the same moving object are more likely to be grouped in a relatively planar configuration. Stereo vision provides 2.5 D information (in contrast with X-ray images) and moving objects are generally smaller than the 3D scene in which the user resides itself.