Free Viewpoint 3D TV Rendering Platform

Abstract

Over the past decade, products enabled by 3D technology have been increasingly adopted in the consumer market. The current challenge in the field is to explore the methods of free-viewpoint interpolation for 3D TVs. Free-viewpoint interpolation enhances the user experience significantly by allowing the viewer to select and alter the desired viewpoint of the scene interactively. This thesis covers the development of the free-viewpoint rendering platform for the European iGLANCE project. The proposed FTV platform is powered by a 7-issue slot VLIW architecture combining scalar and vector data paths. Our processor is based on an image signal processor (ISP) template from Silicon Hive. We vectorized the free-viewpoint algorithm developed in the context of the iGLANCE project, and mapped it onto this processor. The performance of our implementation is compared to an out-of-the-box implementation and previous vectorization work using the same architecture template. In order to address irregular memory accesses, identified as the bottleneck by the previous work, we used scatter-gather unit and a customized memory transfer scheme. This allowed us to apply several classical vectorization methods to fully utilize data level parallelism. In addition, instruction level parallelism is improved by applying further optimizations (loop transformations, data mapping, extending ISA). As a result of all above, a speed-up of a factor 6x is achieved over the selected baseline, which is equivalent to 78x over the out-of-the box code, and the ILP is improved by 17% as compared to the previous work. We set up a demonstration system to validate the results in a real-time environment by mapping our design to an FPGA running at 50 MHz frequency. The achieved frame rate is 6.75 fps in a 1280x720 resolution. This result indicates that when our design is mapped to silicon, running at about 10x the FPGA frequency and with extended processor resources, we will be able to achieve performance levels required in current-day consumer applications, which is Full HD resolution at 30 fps per eye.