Gaussian process regression (GPR), is a powerful non-parametric approach for data modeling, which has garnered considerable interest in the past decade, however its widespread application is impeded by the significant computational burden for larger datasets. The computational complexity for both inference and hyperparameter learning in GPs lead to O(N³) for N training points. The current state-of-the-art approximations, such as structured kernel interpolation (SKI)-based methods e.g., Kernel Interpolation for Scalable Structured Gaussian Process (KISSGP), have emerged to mitigate this challenge by providing a scalable inducing point alternatives. However, the choice of the optimal number of grid points, which influences the accuracy and efficiency of the model, typically remains fixed and is chosen arbitrarily. In this work, we introduce a novel approximation framework, Malleable KISSGP (MKISSGP), which dynamically adjusts grid points using a new hyperparameter of the model called density, which adapts to the changes in the kernel hyperparameters in each training iteration. In comparison with the state-of-the-art KISSGP and irrespective of changes in hyperparameters, our proposed MKISSGP algorithm exhibits consistent error levels in the reconstruction of the kernel matrix, and offers reduced computational complexity. We present extensive simulations to validate the improved performance of the proposed MKISSGP, and give directions for future research. ... Read more

Gaussian process regression (GPR), a potent non-parametric data modeling tool, has gained attention but is hindered by its high com- putational load. State-of-the-art low-rank approximations like struc- tured kernel interpolation (SKI)-based methods offer efficiency, yet lack a strategy for determining the number of grid points, a pivotal factor impacting accuracy and efficiency. In this thesis, we tackle this challenge.
We explore existing low-rank approximations that facilitates the computation, dissecting their strengths and limitations, particularly SKI-based methods. Subsequently, we introduce a novel approxima- tion framework, MKISSGP, which dynamically adjusts grid points us- ing a new hyperparameter of the model: density, according to changes in the kernel hyperparameters in each training iteration.
MKISSGP exhibited consistent error levels in the reconstruction of the kernel matrix, irrespective of changes in hyperparameters. This robust performance forms the bedrock for achieving accurate approx- imations of kernel matrix-related terms. When employing our rec- ommended density value (i.e., 2.7), MKISSGP achieved a comparable level of precision to that of precise GPR, while requiring only 52% of the time compared to the current state-of-the-art method. ... Read more