Object detection heavily relies on accurate annotations, which are costly to obtain but crucial for model performance. Annotation errors can severely impact the reliability of detection models. In response to this challenge, we introduce EnsembAudit (EA), a novel framework designed to autonomously identify and rectify common labeling errors, thereby reducing annotation efforts. EA leverages ensemble techniques such as Threshold Voting for error identification and Non-Maximum Suppression for error rectification.

This paper evaluates EA across various noise levels and types of labeling errors to assess its effectiveness. Our experiments demonstrate that EA excels in detecting and rectifying errors in datasets with significant noise, achieving an approximate 20% reduction in errors. However, its efficacy diminishes when applied to datasets with minimal noise. This study highlights EA's potential in enhancing annotation quality and improving the robustness of object detection applications.

Pseudo-labeling involves training models on a small amount of labeled data and then using those models' predictions on unlabeled data as labels for further training, which therefore decreases the required labeling effort. In this paper, we investigate the effects of pseudo-labeling using the robust YOLOv8 object detector. The research establishes a naive basis pseudo-labeling approach and explores improvement methods, such as object ratio based dynamic thresholds to reduce class imbalance issues, confidence scaling to retain more information when predicting a single class with less confusion, and an ensemble approach to leverage performance benefits offered by model ensembling. Experiments do not show significant improvements and mostly decreasing performance is seen on the VOC2012 dataset. However, improvements relative to the basis are observed with all proposed methods.

The annotation effort associated with object detection is extremely costly. One option to reduce cost is to relax the demands on annotation quality, effectively allowing annotation noise. Current research primarily focuses on noise correction before or during training. However, there remains a gap in the research regarding the impact of specific types of human annotation noise on object-detector performance. This research aimed to determine how sensitive object detectors are to human annotation noise. A systematic methodology was developed to generate and quantify the effects of four noise types: missing annotations, extra annotations, inaccurate bounding boxes, and wrong classification labels. Additionally, evaluations were conducted on YOLOv8 and Faster R-CNN using the PASCAL VOC 2012, VisDrone, and Brain-Tumor datasets. The experiments demonstrated that adding noise to smaller datasets adversely affects the performance of object detectors trained on these datasets more than it does for those trained on larger datasets. Similarly, annotation noise in small objects affects detector performance more than large objects. Furthermore, YOLOv8 is resilient to low levels of missing annotations and inaccurate bounding boxes but is sensitive to all levels of incorrect classification labels. Interestingly, extra annotations seem to have a regularization effect on YOLOv8. In contrast, Faster R-CNN is generally more susceptible to annotation noise compared to YOLOv8, particularly concerning extra annotations, though both models display similar trends regarding inaccurate bounding boxes.

A way to reduce labelling effort and improve accuracy for object detection is class grouping. In this research, we experiment with creating hierarchical tree structures of grouped classes (super-classes). Our objective is to find out what the effects are of grouping classes in terms of accuracy and labelling effort. First we show what accuracy improvement can be gained from different grouping strategies. Then we show what the difference is in accuracy gain for a hierarchical tree structure on FasterRCNN, RetinaNet and YOLOv8. Next up we introduce a new layer in the tree structure with new super-classes and we show the difference in the tree with two and with three layers. After that we compare the accuracy for predicting only super-classes. Lastly, we want to take predicting super-classes one step further by showing that class grouping can reduce labelling effort for real life applications like autonomous cars.

Highly performing object detectors require large training datasets, which entail class and bounding box annotations. To reduce the labelling effort of curating such datasets, Weakly Supervised Object Detection is concerned with training object detectors from only class labels. The most performant weakly supervised detectors (MIL-based) have high inference times, while faster methods (CAM-based) have been primarily studied in the context of localizing just one object in an image. This research proposes an extension to weakly supervised CAM-based detectors that allows them to detect multiple objects in an image and asseses their performance at localizing the full extent of objects with bounding boxes, as well as their general location with pin-points. VGG16 and a novel FPN-based classifier are experimented with as the backbone of the network, followed by GradCAM++ which indicates through heatmaps the locations of the objects predicted by the classifiers. Additionally, the proposed method is used to create pseudo-labels on which any fully supervised detector could be trained on. Results show that while the proposed method is not suitable for detecting the full extent of objects, it can accurately pin-point their general location in near real-time, thus showing the Object is Roughly There (ORT).

Deep convolutional neural networks (CNNs) have achieved current state-of-the-art in image denoising, but require large datasets for training. Their performance remains limited on smaller real-noise datasets. In this paper, we investigate robust deep learning denoising using transfer learning. We explore the impact of dataset sizes, CNN parameter updates, and noise distribution similarity. Our findings demonstrate that finetuning the decoder while fixing the encoder of an encoder-decoder network architecture avoids overfitting during transfer learning. Moreover, we introduce the concept of noise similarity in transfer learning, showing that reducing the similarity distance between pre-training and finetuning noise significantly enhances CNN performance in denoising. Our results are demonstrated on multiple datasets and various noise camera models. Finally, we validate the robustness and applicability of our approach on real-noise images. All our results and analyses hold and generalize across different datasets, highlighting our insights into the potential of transfer learning for image denoising.

In this paper, we combine image denoising and classification, aiming to enhance human perception of noisy images captured by edge devices, like security cameras. Since edge devices have little computational power, we also optimize for efficiency by proposing a novel architecture that integrates the two tasks. Additionally, we alter a Neural Architecture Search (NAS) method, which searches for classifiers, to search for the integrated model while optimizing for a target latency, classification accuracy, and denoising performance. Our NAS architectures outperform our manually designed alternatives in both denoising and classification, offering a significant improvement to human perception. Moreover, our approach empowers users to construct architectures tailored to domains like medical imaging, surveillance systems, and industrial inspections.

Automated imaging systems, critical in domains like medical imaging, autonomous driving, and security, experience noise from camera sensors and electronic circuits in bad or dark lighting conditions. This impacts downstream tasks, including object detection. However, an analysis of strategies combining denoising and object detection is lacking. This study addresses this gap by analyzing diverse strategies for optimizing both image quality and detection performance. Results reveal that isolating denoiser network optimization and training a detector on its outputs yields the best overall performance. Combining detection and denoising enhances detection outcomes. The results offer valuable insights to make educated decisions on how to combine denoising and detection in modern imaging systems.

Occlusion degrades the performance of human pose estimation. In this paper, we introduce targeted keypoint and body part occlusion attacks. The effects of the attacks are systematically analyzed on the best-performing methods. In addition, we propose occlusion specific data augmentation techniques against keypoint and part attacks. Our extensive experiments show that human pose estimation methods are not robust to occlusion and data augmentation does not solve the occlusion problems.

Video understanding has received more attention in the past few years due to the availability of several large-scale video datasets and improvement in the computational power of computers. However, annotating large-scale video datasets are cost-intensive due to their complexity. In this work, we propose a time-efficient video annotation method using spatio-temporal feature similarity and t-SNE dimensionality reduction to make the annotation process more efficient. Placing the same actions from different videos near each other in the two-dimensional space based on feature similarity helps the oracle to group label the video clips. We evaluate the performance of our method on two subsets of the ActivityNet (v1.3) dataset. We show that our method can outperform conventional video labeling tools time-wise while maintaining a reasonable test accuracy on video classification task compared to the ground-truth labels. To further evaluate the generalization of our method, we test our performance on Sports-1M and Breakfast datasets.

This thesis presents a novel self-supervised approach of learning visual representations from videos containing human actions. Our approach tackles the complex problem of learning without the need of labeled data by exploring to what extent the ideas successfully used for images can be transferred, adapted and extended to videos for action recognition purposes. We begin by giving a brief introduction to the topic of learning features without having access to a labeled corpora, providing the motivation of our work. We continue with presenting the related research in terms of contrastive learning, action recognition from videos with 3D convolutions and self-supervised techniques for both images and videos. Next, we formalize our approach with regards to the sampling method, the types of spatial and temporal transformations and the contrastive loss used. We evaluate videoSimCLR proposed method in terms of linear evaluation, fully fine-tuning and video retrieval using two popular action recognition datasets, HMDB51 and UCF101. We also explore the extension of another contrastive learning approach to videos, videoMOCO, and compare it with videoSimCLR by means of linear evaluation.

Creating big datasets is often difficult or expensive which causes people to augment their dataset with rendered images. This often fails to significantly improve accuracy due to a difference in distribution between real and rendered datasets. This paper shows that the gap between synthetic and real-world image distributions can be closed by using GANs to convert the synthetic data to a dataset which has the same distribution as the real data. Training this GAN requires only a fraction of the dataset traditionally required to get a high classification accuracy. This converted data can subsequently be used to train a classifier with a higher accuracy than a classifier trained only on the real dataset.

Object detectors, much like humans, perform less well on small than on large objects. Because of this, the object size distribution of a dataset influences the average precision a network achieves on that dataset. Therefore, the object size/precision curve of a network might be a better way to compare convolutional object detectors than the average precision over an entire dataset. In this paper we measure the relationship between object size and accuracy
for a modern mobile convolutional object detector. We verify that this relationship holds for a different dataset, and that the dataset object size distribution influences the average precision over the entire dataset. We conclude that the object size/accuracy curve might contain more information about a network’s performance than the average precision over an entire dataset.