This paper shows how the current state of the art in image classification performs on LEGO bricks. Currently the standard image classification models with deep learning are single label image classifiers. In this paper we will convert them to work on multi-label images and subsequently evaluate how well they perform. We show how well the classifiers will work on three different types of datasets. Experiments will be conducted on these three types of datasets to compare the performance of three different multi-label image classifiers. The main research question accompanying this paper is ``How well does the state of the art in image classification work on LEGO bricks?''. Three subquestions are set up to answer this question. The first will regard the existence of the image classifiers. The second subquestion will regard how big the influence is of real life aspects, such as deterioration of the LEGO bricks. The final subquestion will be about the performance on the datasets. After answering these questions and conducting the experiments, we came to the conclusion that the ResNext model performed the best on almost all of the categories. Based on the numbers of the results we can also conclude that the models should perform well with multi-label images of LEGO bricks.

Color information has been shown to provide useful information during image classification. Yet current popular deep convolutional neural networks use 2-dimensional convolutional layers. The first 2-dimensional convolutional layer in the network combines the color channels of the input images, which produces feature maps per channel with only spatial dimensions, height and width, getting rid of the color dimension. In this work we introduce Full Color Deep networks which use 3-dimensional convolutions to retain the color dimension beyond the first layer. The 3D kernels convolve over the color and spatial dimensions. The network can extract features from all three dimensions in all layers which are subsequently used by the classifier. We show that the Full Color Deep networks perform at least as well as the current CNNs but outperform them in learning color information and using that information in other downstream tasks.