Machine learning and data processing techniques relying on covariance information are widespread as they identify meaningful patterns in unsupervised and unlabeled settings. As a prominent example, Principal Component Analysis (PCA) projects data points onto the eigenvectors of their covariance matrix, capturing the directions of maximum variance. This mapping, however, falls short in two directions: it fails to capture information in low-variance directions, relevant when, e.g., the data contains high-variance noise; and it provides unstable results in low-sample regimes, especially when covariance eigenvalues are close. CoVariance Neural Networks (VNNs), i.e., graph neural networks using the covariance matrix as a graph, show improved stability to estimation errors and learn more expressive functions in the covariance spectrum than PCA, but require training and operate in a labeled setup. To get the benefits of both worlds, we propose Covariance Scattering Transforms (CSTs), deep untrained networks that sequentially apply filters localized in the covariance spectrum to the input data and produce expressive hierarchical representations via nonlinearities. We define the filters as covariance wavelets that capture specific and detailed covariance spectral patterns. We improve CSTs’ computational and memory efficiency via a pruning mechanism, and we prove that their error due to finite-sample covariance estimations is less sensitive to close covariance eigenvalues compared to PCA, improving their stability. Our experiments on age prediction from cortical thickness measurements on 4 datasets collecting patients with neurodegenerative diseases show that CSTs produce stable representations in low-data settings, as VNNs but without any training, and lead to comparable or better predictions w.r.t. more complex learning models. ... Read more

Covariance Neural Networks (VNNs) perform graph convolutions on the empirical covariance matrix of signals defined over finite-dimensional Hilbert spaces, motivated by robustness and transferability properties. Yet, little is known about how these arguments extend to infinite-dimensional Hilbert spaces. In this work, we take a first step by introducing a novel convolutional learning framework for signals defined over infinite-dimensional Hilbert spaces, centered on the (empirical) covariance operator. We constructively define Hilbert coVariance Filters (HVFs) and design Hilbert coVariance Networks (HVNs) as stacks of HVF filterbanks with nonlinear activations. We propose a principled discretization procedure, and we prove that empirical HVFs can recover the Functional PCA (FPCA) of the filtered signals. We then describe the versatility of our framework with examples ranging from multivariate real-valued functions to reproducing kernel Hilbert spaces. Finally, we validate HVNs on both synthetic and real-world time-series classification tasks, showing robust performance compared to MLP and FPCA-based classifiers. ... Read more

We study the problem of generating graph signals from unknown distributions defined over given graphs, relevant to domains such as recommender systems or sensor networks. Our approach builds on generative diffusion models, which are well established in vision and graph generation but remain underexplored for graph signals. Existing methods lack generality, either ignoring the graph structure in the forward process or designing graph-aware mechanisms tailored to specific domains. We adopt a forward process that incorporates the graph through the heat equation. Rather than relying on the standard formulation, we consider a time-warped coefficient to mitigate the exponential decay of the drift term, yielding a graph-aware generative diffusion model (GAD). We analyze its forward dynamics, proving convergence to a Gaussian Markov random field with covariance parametrized by the graph Laplacian, and interpret the backward dynamics as a sequence of graph-signal denoising problems. Finally, we demonstrate the advantages of GAD on synthetic data, real traffic speed measurements, and a temperature sensor network. ... Read more

CoVariance Neural Networks (VNNs) perform convolutions on the graph determined by the covariance matrix of the data, which enables expressive and stable covariance-based learning. However, covariance matrices are typically dense, fail to encode conditional independence, and are often precomputed in a task-agnostic way, which may hinder performance. To overcome these limitations, we study Precision Neural Networks (PNNs), i.e., VNNs on the precision matrix—the inverse covariance. The precision matrix naturally encodes statistical independence, often exhibits sparsity, and preserves the covariance spectral structure. To make precision estimation task-aware, we formulate an optimization problem that jointly learns the network parameters and the precision matrix, and solve it via alternating optimization, by sequentially updating the network weights and the precision estimate. We theoretically bound the distance between the estimated and true precision matrices at each iteration, and demonstrate the effectiveness of joint estimation compared to two-step approaches on synthetic and real-world data. ... Read more

Learning deep representations from covariance in-formation via coVariance Neural Networks (VNNs) has shown an improved performance and insights with respect to Principal Component Analysis (PCA)-based alternatives and better stability in finite-sample regimes. VNNs extend the PCA transform by learning end-to-end the spectral processing function on the principal directions of the data in each layer. However, VNNs operate on the pre-computed sample covariance matrix, which is prone to estimation errors, sensitive to outliers, and not adapted to the task at hand. To overcome this limitation, we propose Robust coVariance Neural Networks (RVNNs), a framework that simultaneously learns a robust estimator of the covariance matrix and the VNN parameters in an end-to-end manner, leading to a fully task-aware pipeline. We prove that RVNNs combine the robustness to outliers with the finite-sample stability of VNNs, and we show that their end-to-end robust covariance learning leads to better prediction performance compared to robust PCA-based approaches on simulated and real-world data from brain recordings and human motion sensor measurements. ... Read more

This paper proposes a scalable method for identifying interactions in higher-order networks from observations of nodal processes. Finding such dependencies is important in many disciplines, including neuroscience, social influence modeling, and beyond. However, current approaches are either limited to extracting pairwise dependencies or struggle with scalability, as estimating higher-order dependencies becomes computationally prohibitive. To overcome these challenges, we introduce a tensorbased graph Volterra model that leverages low-rank decomposition techniques to estimate higher-order interactions efficiently. Our approach not only reduces computational and storage complexity but also acts as an implicit regularizer, improving network estimation in ill-posed settings. We validate our method through simulations and real data experiments, demonstrating competitive performance and enhanced scalability compared to existing techniques. ... Read more

Topological Deep Learning (TDL) has emerged as a paradigm to process and learn from signals defined on higher-order combinatorial topological spaces, such as simplicial or cell complexes. Although many complex systems have an asymmetric relational structure, most TDL models forcibly symmetrize these relationships. In this paper, we first introduce a novel notion of higher-order directionality and we then design Directed Simplicial Neural Networks (Dir-SNNs) based on it. Dir-SNNs are message-passing networks operating on directed simplicial complexes able to leverage directed and possibly asymmetric interactions among the simplices. To our knowledge, this is the first TDL model using a notion of higher-order directionality. We theoretically and empirically prove that Dir-SNNs are more expressive than their directed graph counterpart in distinguishing non-isomorphic directed graphs. Experiments on a synthetic source localization task demonstrate that Dir-SNNs outperform undirected SNNs when the underlying complex is directed, and perform comparably when the underlying complex is undirected. ... Read more

Computer and social networks can be effectively represented as complex temporal graphs where entities (nodes) keep interconnecting through various relationships (edges), forming evolving structures. Anomaly Detection (AD) in such networks consists of identifying patterns diverging from what is expected or normal. This task is fundamental for the detection of potential threats - e.g. suspicious connections (edge AD) or misbehaving entities (node AD), and challenging due to the lack of a common definition of anomaly. However, the literature is scarce about solutions to detect node anomalies on temporal graphs. This work addresses three challenges in AD as found in computer and social networks: fast-evolving graph structure, lack of ground truth, and simultaneous presence of anomalous nodes and edges. For this, we propose to use temporal Graph Neural Networks (tGNNs) coupled with specialised AD blocks trained in a self-supervised way. We also embed an attention mechanism providing interpretability to the decision process. We extensively validate the tGNNs on synthetic and real-world datasets showing that they successfully detect both node and edge anomalies simultaneously (≈0.9 of average AUC). ... Read more