High-Quality, Real-Time HD Video Stereo Matching on FPGA

Abstract

Stereo matching is an important computer vision technique, which extracts the depth information of the scene by matching a pair of stereo images. It has numerous applications, such as view-point interpolation, 3DTV, object detection, etc. In the past decades, many algorithms have been proposed to improve the matching quality or to increase the speed. Due to the high computational complexity, it is still quite challenging to attain high matching-quality at real-time speed. In this thesis, we propose a hardware design of stereo matching, which is capable of producing high-quality disparity maps at real-time speed. A high-quality stereo matching algorithm is efficiently implemented and hardware-oriented optimized, attaining huge speedup by parallel computing. The whole algorithm is implemented in a single EP3SL150 FPGA. The experimental results show that our design is capable of matching high-definition videos at real-time speed, i.e. 60 frame per second at 1024×768 resolution. In terms of matching quality, our design is among the leading real-time methods, evaluated in the Middlebury stereo benchmark. As an application of the stereo matcher, we also build up a depth-scaling system for 3DTV, working together with a view synthesis module. The SoC system synthesizes high-quality virtual views at real-time speed.