Visualising High-Dimensional Data using Non-Euclidean t-SNE

Abstract

Dimensionality reduction methods such as t-SNE are standard for visualising high-dimensional data, yet almost universally embed into Euclidean space. Representation learning has shown that hyperbolic and spherical spaces fit hierarchical and cyclic data with lower distortion, but those gains are measured on downstream tasks, not on what an analyst reads from a two-dimensional scatter plot. This thesis asks how the curvature of the embedding space influences the quality of such visualisations.
To compare the visualisations, we generalise t-SNE to the Euclidean, hyperbolic and spherical geometries by using the geodesic distance in the low-dimensional kernel, and optimise it with Riemannian stochastic gradient descent. Because visualisation quality is multidimensional and t-SNE is sensitive to its hyperparameters, geometries are compared through visualisation corpora: Pareto fronts over six established quality metrics for local structure, distance preservation and label-aware structure, produced by multi-objective Bayesian optimisation over the hyperparameters, curvature included. The comparison covers synthetic datasets sampled from known constant-curvature manifolds and four real datasets.

The central hypothesis, that an embedding geometry matching the sign of the data's curvature improves visualisation quality, is not supported on the high-dimensional synthetic datasets. A curvature fit on their pairwise distances does not distinguish the generating geometries, whereas on two-dimensional versions of the same datasets it does, and there the visualisations suggest that the matched geometry is the most faithful. The geometry changes distance preservation only slightly, and label-aware differences change sign with the dataset rather than following matching. In the hyperbolic geometry, beyond a moderate dimensionless curvature (the curvature normalised by the squared characteristic length of the embedding) the metrics decline and global structure is visibly lost. Overall, curvature improves a high-dimensional visualisation little and not through matching, but can degrade it considerably. We recommend starting from Euclidean t-SNE, trying a curved geometry mainly when the curvature of the data can be recovered from its distances, and reporting its dimensionless curvature next to a Euclidean visualisation.