Spurious correlation can be detrimental to performance of machine learning solutions on data that it has never seen before. This could be disastrous in situations where the prediction is important and the situation is ever changing. This paper investigates whether adding a masking layer can improve the accuracy of machine learning models on visual data, where the background is different during training and testing. A new dataset is created that overlays MNIST numbers on CIFAR-10 images, where during training only a distinct subset of background images for every label is presented to the model. The performance is measured by supplying the model with an unseen foreground sample on a background that was not encountered during training. This paper then shows that adding a masking layer can improve the performance of a model. This improvement depends on the implementation and the dataset on which the model is trained and tested.