Multiplane Images (MPIs) are among the simpler scene representations used for novel view synthesis. They consist of stacks of partially transparent RGB images that create parallax when viewed from different viewpoints. As MPIs are a method of adding depth to images, one might expect them to have been extensively studied in conjunction with images obtained from depth cameras (RGB-D images), but the contrary is true.

We propose a method for generating MPIs from RGB-D images using only classical algorithms, as opposed to machine learning-based methods. This makes the approach fast to run and therefore suitable for real-time use, such as building MPIs from live video streams captured by depth cameras. Our method is evaluated in terms of performance using scenes adapted from the VKITTI 2 dataset. Results show that, on a modern laptop, our method far exceeds the speed required to ingest real-time data. The process can still run at interactive frame rates when generating MPIs with more than 400 planes.

Since visual artifacts remain clearly visible in the generated MPIs, more sophisticated algorithms should be investigated to alleviate these artifacts while maintaining real-time performance. ... Read more