Improving Multiplane Images with Deformable Layers

Abstract

Multiplane Images (MPIs) are an efficient representation for single-view novel-view synthesis, but their fronto-parallel planes limit how well they can represent slanted or non-planar scene geometry. This paper investigates whether MPI-based single-view view synthesis can be improved by replacing flat MPI planes with alpha-weighted deformable mesh layers. The proposed method takes a predicted MPI as input, converts each RGBA layer into a triangular mesh, and displaces its vertices using an expected disparity estimate derived from the MPI alpha distribution. The goal is to add local geometric flexibility while preserving the layered visibility structure of the original MPI.

The method is evaluated on RealEstate10K and LLFF using PSNR, SSIM, and LPIPS, with additional runtime measurements and a controlled synthetic Blender scene. On predicted MPIs, the deformation produces only marginal changes and does not yield a meaningful improvement over the flat MPI baseline, while also increasing rendering cost. However, in the controlled RGB-D experiment, where the MPI is constructed from accurate depth and uses fewer layers, the deformable representation better follows slanted geometry and improves over the flat baseline. These results suggest that deformable MPI layers can be beneficial when the input layered geometry is reliable, but are not sufficient as a standalone post-processing step for learned single-view MPI predictions.