Using the raw data from consumer-level RGB-D cameras as input, we propose a deep-learning-based approach to efficiently generate RGB-D images with completed information in high resolution. To process the input images in low resolution with missing regions, new operators for adaptive convolution are introduced in our deep-learning network that consists of three cascaded modules - the completion module, the refinement module, and the super-resolution module. The completion module is based on an architecture of encoder-decoder, where the features of input raw RGB-D will be automatically extracted by the encoding layers of a deep neural network. The decoding layers are applied to reconstruct the completed depth map, which is followed by a refinement module to sharpen the boundary of different regions. For the super-resolution module, we generate RGB-D images in high resolution by multiple layers for feature extraction and a layer for upsampling. Benefited from the adaptive convolution operators proposed in this article, our results outperform the existing deep-learning-based approaches for RGB-D image complete and super-resolution. As an end-to-end approach, high-fidelity RGB-D images can be generated efficiently at the rate of 22 frames/s. Note to Practitioners - With the development of consumer-level RGB-D cameras, industries have started to employ these low-cost sensors in many robotic and automation applications. However, images generated by consumer-level RGB-D cameras are generally in low resolution. Moreover, the depth images often have incomplete regions when the surface of an object is transparent, highly reflective, or beyond the distance of sensing. With the help of our method, engineers are able to 'repair' the images captured by consumer-level RGB-D cameras in high efficiency. As the typical deep-learning networks are employed in this approach, the proposed approach fits well with the GPU-based hardware architecture of deep-learning computation - therefore, it potentially can be integrated into the hardware of RGB-D cameras.

Handle-driven image warping based on linear blending is widely used in many applications because of its merits on intuitiveness, efficiency, and ease of implementation. In this paper, we develop a method to compute high-quality weights within a closed domain for image warping. The property of C₂ continuity in weights is guaranteed by the carefully formulated basis functions. The efficiency of our algorithm is ensured by a closed-form formulation of the computation for weights. The cost of inserting a new handle is only the time to evaluate the distances from the new handle to all other sample points in the domain. A virtual handle insertion algorithm is developed to allow users to freely place handles within the domain while preserving the satisfaction of all expected criteria on weights for linear blending. Experimental examples for real-time applications are shown to demonstrate the effectiveness of this method.