Exploration of Deep Learning-based Computer Vision for the detection of floating plastic debris in waterways

Abstract

Plastic pollution is one of the most challenging global environmental problems. Currently, more than 1000 rivers transport approximately 80% of the plastic influx into the oceans. Naturally, more and more companies are interested in tackling this problem. One of them is Noria Sustainable Innovators, a company based in Delft (Netherlands). It is focussed on the detection, removal, and reuse of plastic from Dutch waterways. The company has the ambition to automate the detection of plastic for a wide range of applications. The quantification of plastic and understanding its spatiotemporal variability are crucial for the mitigation of plastic pollution. Current monitoring methods (e.g., visual counting) are tedious, time-consuming, and labour-intensive. Furthermore, the detection of different plastic debris objects could provide more insight about the source of plastic pollution. This thesis explores the feasibility of automating plastic detection in waterways using modern deep learning (DL) algorithms named convolutional neural networks (CNNs) with image classification and object detection techniques. To train these models, a large dataset is required. Due to the unavailability of data, images were gathered in a controlled environment with two GoPros and a Huawei P30. The data was aggregated during sunny and cloudy conditions, different camera heights (2.7m and 4.0m) and angles (0 and 45 degrees). For the simplest case (2.7m/0 degrees), a maximum accuracy of 87.6% was obtained for the multiclass classification of plastic debris in images, using the DenseNet121 model. By applying a majority vote for the three best performing models (DenseNet121, ResNet50 and InceptionV3), the accuracy could be increased to 91%. A qualitative and quantitative analysis found that the following factors influence the model performance negatively: presence of organic material, wind, transparent objects, submerged objects, small objects, overlapping and occluding plastic debris, sun glint and reflection of other objects on the water surface. Sunny conditions yielded a lower accuracy (79%) than cloudy conditions (90%), explained by the presence of sun glint. By applying object detection, the error sources influencing the model performance could be reduced. For training and testing data from 2.7m/0 degrees on one class (‘plastic debris’), the YOLOv4 model yielded an accuracy of 95.61% (GoPro). For four classes (e.g., plastic bottles, other plastic, paper, metal tins) an average accuracy of 66.04% was found, indicating that the model experienced difficulties distinguishing different floating debris in water. Furthermore, it was also shown, that the use of a different image source (Huawei P30), does not have a negative effect on the accuracy (96.63%) compared to the original image source (GoPro). Furthermore, due to height differences, discrepancy in object sizes and different camera settings, the trained model had large difficulties generalizing to a dataset from Indonesia (12.23%). On the other hand, training on the dataset from Indonesia and testing on the dataset from 2.7m/0 degrees achieved an accuracy of 63.51%. Although the error sources could be reduced, the model was still negatively impacted by small, transparent objects, submerged objects and the presence of sun glint. This study clearly showed that Deep Learning-based computer vision can detect floating plastic debris with a high accuracy and have the potential to automate the process of plastic detection in the future. Future work would comprise the following aspects: sensor improvements (polarising filter for sun glint and multispectral sensor for continuous monitoring), data collection from the natural environment and different image sources, implementation of guidelines for Citizen Science platforms, addition of an object tracking module for monitoring (YOLOv4) and focussing on the detection of specific plastic debris objects after the removal from waterways.