Remote communication applications have become a necessity in a globalised and connectedworld exemplified by the popularity of video conferencing applications. In recent years, virtual reality (VR) remote communication applications have emerged that aim to deliver a greater sense of co-presence and immersion in a shared virtual space where users are able to navigate freely while employing both verbal and non-verbal communication. Such applications require a volumetric user representation and point clouds have emerged as a popular format to represent real-time user reconstructions. However, volumetric point clouds are challenging to deliver over bandwidth-limited networks owing to the large volume of data required for dynamic streams. Adaptive streaming is the process of segmenting an object spatially and temporally in order to optimize the delivery of content by prioritizing the quality of spatial segments that are visible from a given viewport. In this thesis, we embed our research in the application scenario of VR remote communication with real-time point cloud user reconstructions and explore adaptive delivery optimizations. Previous work in the field mainly focused on using the entire point cloud object as the unit of bandwidth allocation in scenes containing multiple point clouds. Other recent work has relied on computationally complex surface estimation in order to spatially segment the point cloud that is unsuited to real-time applications. This thesis focuses on investigating if a user-centred approach combined with a low complexity adaptive streaming method can improve the quality of experience of interacting with point cloud user reconstructions in VR remote communication. We focus on optimizing the delivery of a remote user’s reconstruction with spatial segmentation and surface orientation estimation in real-time combinedwith an auxiliary utility function to allocate the available bandwidth across available segments. The utility is defined based on the position and surface orientation of the point cloud segment and the position and orientation of the user’s viewport.@en

The need to capture environments and objects in 3 dimensions to produce a high quality digital representation is proving to be useful in many applications in the world, where there is an increasing dependence on digital spaces. Point Clouds are a data type to represent 3D objects and scenes. During the processing of the point cloud data, undesired changes to the point cloud can be introduced or information can get lost. Automatic detection and classifying of the quality of a point cloud is needed, to regulate this data type for use in 3D environments, and speed up the processing of point cloud objects in real-time applications.

In this thesis, we look into the problem of no-reference point cloud quality assessment. Full-reference quality metrics compare a distorted point cloud with its original, while no-reference or blind metrics only look at a single point cloud. This task is more challenging than full-reference, as there is less direct information available to work with. We approach the challenge of how the perceptual quality of a point cloud can be estimated when a high-quality reference point cloud is not available, by analyzing if any point cloud properties correlate with the perceptual quality.

We try to find point cloud properties that are effective to use in a model to predict a user given quality score. We select multiple local properties to transform into global descriptors, by taking multiple statistical attributes of the data on all the points to combine into scalars. We analyze the created global descriptors to find linear correlations with the quality score, and found many features that to some measure linearly correlate with some distortion types. With the selected global descriptors, we use Support Vector Regression to find the fitting of the features to the quality score. Varying performances are achieved by training the models on different data subsets, and different fitting kernel functions.

The results show some distortions on unknown objects are properly predictable using fitted weights on this distortion. However, a single defining quality descriptor is not yet found. While the results show potential, our model is not robust and fast enough to perform a reliable assessment of quality in a real-time environment. We are just at the beginning of exploring no-reference quality assessment, and while the current methods are not yet appliable in real-time scenarios, future work can fill many knowledge gaps to reach this goal.