Resource-Efficient Distributed Gaussian Processes

Abstract

Multi-agent systems operating under limited communication and computational resources require efficient distributed learning methods. In many such systems, communication is the primary bottleneck, yet existing distributed Gaussian process (GP) approaches have largely overlooked communication efficiency. This thesis addresses this gap by developing a resource-efficient framework for recursive training of a distributed multi-output Gaussian process.

A distributed recursive GP (RGP) problem formulation is developed, from which the ADMM-RGP and PDMM-RGP algorithms are derived. The former adapts a communication-efficient alternating direction method of multipliers (ADMM) scheme, while the latter is obtained by applying the primal–dual method of multipliers (PDMM) to the distributed RGP problem. Both methods enable recursive, distributed training of a sparse multi-output GP.

Extensive simulations on a two-dimensional wind field dataset demonstrate that the proposed algorithms better approximate the centralized model and converge faster than the state-of-the-art Consensus-RGP algorithm. Compared to Consensus-RGP, ADMM-RGP reduces inter-agent communication by up to 30%, while PDMM-RGP achieves reductions of up to 40%, and is particularly effective in sparse network topologies. These gains come at the cost of slightly increased local computation and storage, which is a favorable trade-off in communication-constrained systems.