Manifold-Aware Regularization for Masked Autoencoders

Masked Autoencoders (MAEs) represent a significant shift in self-supervised learning (SSL) due to their independence from augmentation techniques for generating positive (and/or negative) pairs as in contrastive frameworks. Their masking and reconstruction strategy also aligns well with SSL approaches in natural language processing. Most MAEs...

Protein Structure and Sequence Co-Design through Graph Based Generative Diffusion Modeling

Proteins are fundamental biological macromolecules essential for cellular structure, enzymatic catalysis, and immune defense, making the generation of novel proteins crucial for advancements in medicine, biotechnology, and material sciences. This study explores protein design using deep generative models, specifically Denoising Diffusion...

An Experimental Assessment of the Stability of Graph Contrastive Learning

The Deep Neural Network (DNN) has become a widely popular machine learning architecture thanks to its ability to learn complex behaviors from data. Standard learning strategies for DNNs however rely on the availability of large, labeled datasets. Self-Supervised Learning (SSL) is a style of learning that allows models to also use unlabeled data...

BECLR: Batch Enhanced Contrastive Unsupervised Few-Shot Learning

There exists a fundamental gap between human and artificial intelligence. Deep learning models are exceedingly data hungry for learning even the simplest of tasks, whereas humans can easily adapt to new tasks with just a handful of samples. Unsupervised few-shot learning (U-FSL) aspires to bridge this gap, without relying on costly annotations....

MAPL: Model Agnostic Peer-to-Peer Learning

Current methods in Federated and Decentralized learning presume that all clients share the same model architecture, assuming model homogeneity. However, in practice, this assumption may not always hold due to hardware differences. While prior research has addressed model heterogeneity in Federated Learning, it remains unexplored in fully...

Self-Supervised Few Shot Learning: Prototypical Contrastive Learning with Graphs

A primary trait of humans is the ability to learn rich representations and relationships between entities from just a handful of examples without much guidance. Unsupervised few-shot learning is an undertaking aimed at reducing this fundamental gap between smart human adaptability and machines. We present a contrastive learning scheme for...

TRIDENT: Transductive Variational Inference of Decoupled Latent Variables for Few Shot Classification

The versatility to learn from a handful of samples is the hallmark of human intelligence. Few-shot learning is an endeavour to transcend this capability down to machines. Inspired by the promise and power of probabilistic deep learning, we propose a novel variational inference network for few-shot classification (coined as TRIDENT) to decouple...

LAB: Learnable Activation Binarizer for Binary Neural Networks

Binary Neural Networks (BNNs) are receiving an upsurge of attention for bringing power-hungry deep learning towards edge devices. The traditional wisdom in this space is to employ sign(.) for binarizing featuremaps. We argue and illustrate that sign(.) is a uniqueness bottleneck, limiting information propagation throughout the network. To...