Self-supervised contrastive learning is a popular way to pre-train vision foundation models. So far, it has mostly been studied with large pre-training datasets, and it is most accessible to organizations with massive computational resources. In this work we evaluate the data-efficiency of one such method, Momentum Contrast (MoCo), and investigate how to make it work better when less data is available. We pre-train a Vision Transformer with MoCo on subsets of Tiny-ImageNet ranging from 1,000 to 100,000 images, and evaluate the learned representations on a diverse set of downstream tasks using linear probing. We investigate how the training parameters of MoCo should be chosen for a given amount of data, how the downstream accuracy scales with the amount of pre-training data, and how this scaling differs across types of downstream tasks. We find that the best parameters depend on the amount of data: the optimal number of negatives used for the contrastive objective grows with the size of the dataset, while the momentum coefficient has no single best value. We also find that pre-training is beneficial even with very little data, the downstream accuracy grows approximately log-linearly with the size of the pre-training set, and the data-efficiency growth rate is larger for tasks that are similar to the pre-training data. ... Read more