Acquiring material properties of objects for tactile simulation through point cloud scans

Abstract

Tactile internet enables communication in a new layer of immersion, touch. It has the potential to transform the landscape of digital communication. However, the achievable scale of tactile internet is severely limited by its 1 ms round-trip latency. We propose a workaround for the latency through tactile simulation, which can bypass the requirement by having the user interact with locally simulated force feedback instead of real ones. A real-time material estimation method is required to create such a simulation, as it needs material information such as friction and tactile texture. We, therefore, investigate whether material is estimable from point cloud scans since it provides readily available environmental data. We also explore how friction and tactile texture could be extracted from the material. Past material estimation methods rely heavily on point intensity; however, most existing point clouds do not have intensity information. Therefore, we study whether mocking intensity information could be sufficient, exchanging it with grayscale. The results demonstrate that without point intensity, the material estimation method can only discern object colour and not material properties. This finding implicates the importance of intensity and suggests future exploration of the viability of material estimation with intensity data for indoor objects.