In this paper, we combine image denoising and classification, aiming to enhance human perception of noisy images captured by edge devices, like security cameras. Since edge devices have little computational power, we also optimize for efficiency by proposing a novel architecture that integrates the two tasks. Additionally, we alter a Neural Architecture Search (NAS) method, which searches for classifiers, to search for the integrated model while optimizing for a target latency, classification accuracy, and denoising performance. Our NAS architectures outperform our manually designed alternatives in both denoising and classification, offering a significant improvement to human perception. Moreover, our approach empowers users to construct architectures tailored to domains like medical imaging, surveillance systems, and industrial inspections.

Deep convolutional neural networks (CNNs) have achieved current state-of-the-art in image denoising, but require large datasets for training. Their performance remains limited on smaller real-noise datasets. In this paper, we investigate robust deep learning denoising using transfer learning. We explore the impact of dataset sizes, CNN parameter updates, and noise distribution similarity. Our findings demonstrate that finetuning the decoder while fixing the encoder of an encoder-decoder network architecture avoids overfitting during transfer learning. Moreover, we introduce the concept of noise similarity in transfer learning, showing that reducing the similarity distance between pre-training and finetuning noise significantly enhances CNN performance in denoising. Our results are demonstrated on multiple datasets and various noise camera models. Finally, we validate the robustness and applicability of our approach on real-noise images. All our results and analyses hold and generalize across different datasets, highlighting our insights into the potential of transfer learning for image denoising.

Occlusion degrades the performance of human pose estimation. In this paper, we introduce targeted keypoint and body part occlusion attacks. The effects of the attacks are systematically analyzed on the best-performing methods. In addition, we propose occlusion specific data augmentation techniques against keypoint and part attacks. Our extensive experiments show that human pose estimation methods are not robust to occlusion and data augmentation does not solve the occlusion problems.

Automated imaging systems, critical in domains like medical imaging, autonomous driving, and security, experience noise from camera sensors and electronic circuits in bad or dark lighting conditions. This impacts downstream tasks, including object detection. However, an analysis of strategies combining denoising and object detection is lacking. This study addresses this gap by analyzing diverse strategies for optimizing both image quality and detection performance. Results reveal that isolating denoiser network optimization and training a detector on its outputs yields the best overall performance. Combining detection and denoising enhances detection outcomes. The results offer valuable insights to make educated decisions on how to combine denoising and detection in modern imaging systems.

Highly performing object detectors require large training datasets, which entail class and bounding box annotations. To reduce the labelling effort of curating such datasets, Weakly Supervised Object Detection is concerned with training object detectors from only class labels. The most performant weakly supervised detectors (MIL-based) have high inference times, while faster methods (CAM-based) have been primarily studied in the context of localizing just one object in an image. This research proposes an extension to weakly supervised CAM-based detectors that allows them to detect multiple objects in an image and asseses their performance at localizing the full extent of objects with bounding boxes, as well as their general location with pin-points. VGG16 and a novel FPN-based classifier are experimented with as the backbone of the network, followed by GradCAM++ which indicates through heatmaps the locations of the objects predicted by the classifiers. Additionally, the proposed method is used to create pseudo-labels on which any fully supervised detector could be trained on. Results show that while the proposed method is not suitable for detecting the full extent of objects, it can accurately pin-point their general location in near real-time, thus showing the Object is Roughly There (ORT).

Pseudo-labeling involves training models on a small amount of labeled data and then using those models' predictions on unlabeled data as labels for further training, which therefore decreases the required labeling effort. In this paper, we investigate the effects of pseudo-labeling using the robust YOLOv8 object detector. The research establishes a naive basis pseudo-labeling approach and explores improvement methods, such as object ratio based dynamic thresholds to reduce class imbalance issues, confidence scaling to retain more information when predicting a single class with less confusion, and an ensemble approach to leverage performance benefits offered by model ensembling. Experiments do not show significant improvements and mostly decreasing performance is seen on the VOC2012 dataset. However, improvements relative to the basis are observed with all proposed methods.

Pseudo-labeling involves training models on a small amount of labeled data and then using those models' predictions on unlabeled data as labels for further training, which therefore decreases the required labeling effort. In this paper, we investigate the effects of pseudo-labeling using the robust YOLOv8 object detector. The research establishes a naive basis pseudo-labeling approach and explores improvement methods, such as object ratio based dynamic thresholds to reduce class imbalance issues, confidence scaling to retain more information when predicting a single class with less confusion, and an ensemble approach to leverage performance benefits offered by model ensembling. Experiments do not show significant improvements and mostly decreasing performance is seen on the VOC2012 dataset. However, improvements relative to the basis are observed with all proposed methods.

A way to reduce labelling effort and improve accuracy for object detection is class grouping. In this research, we experiment with creating hierarchical tree structures of grouped classes (super-classes). Our objective is to find out what the effects are of grouping classes in terms of accuracy and labelling effort. First we show what accuracy improvement can be gained from different grouping strategies. Then we show what the difference is in accuracy gain for a hierarchical tree structure on FasterRCNN, RetinaNet and YOLOv8. Next up we introduce a new layer in the tree structure with new super-classes and we show the difference in the tree with two and with three layers. After that we compare the accuracy for predicting only super-classes. Lastly, we want to take predicting super-classes one step further by showing that class grouping can reduce labelling effort for real life applications like autonomous cars.

A way to reduce labelling effort and improve accuracy for object detection is class grouping. In this research, we experiment with creating hierarchical tree structures of grouped classes (super-classes). Our objective is to find out what the effects are of grouping classes in terms of accuracy and labelling effort. First we show what accuracy improvement can be gained from different grouping strategies. Then we show what the difference is in accuracy gain for a hierarchical tree structure on FasterRCNN, RetinaNet and YOLOv8. Next up we introduce a new layer in the tree structure with new super-classes and we show the difference in the tree with two and with three layers. After that we compare the accuracy for predicting only super-classes. Lastly, we want to take predicting super-classes one step further by showing that class grouping can reduce labelling effort for real life applications like autonomous cars.

Object detection heavily relies on accurate annotations, which are costly to obtain but crucial for model performance. Annotation errors can severely impact the reliability of detection models. In response to this challenge, we introduce EnsembAudit (EA), a novel framework designed to autonomously identify and rectify common labeling errors, thereby reducing annotation efforts. EA leverages ensemble techniques such as Threshold Voting for error identification and Non-Maximum Suppression for error rectification. This paper evaluates EA across various noise levels and types of labeling errors to assess its effectiveness. Our experiments demonstrate that EA excels in detecting and rectifying errors in datasets with significant noise, achieving an approximate 20% reduction in errors. However, its efficacy diminishes when applied to datasets with minimal noise. This study highlights EA's potential in enhancing annotation quality and improving the robustness of object detection applications.

Object detection heavily relies on accurate annotations, which are costly to obtain but crucial for model performance. Annotation errors can severely impact the reliability of detection models. In response to this challenge, we introduce EnsembAudit (EA), a novel framework designed to autonomously identify and rectify common labeling errors, thereby reducing annotation efforts. EA leverages ensemble techniques such as Threshold Voting for error identification and Non-Maximum Suppression for error rectification. This paper evaluates EA across various noise levels and types of labeling errors to assess its effectiveness. Our experiments demonstrate that EA excels in detecting and rectifying errors in datasets with significant noise, achieving an approximate 20% reduction in errors. However, its efficacy diminishes when applied to datasets with minimal noise. This study highlights EA's potential in enhancing annotation quality and improving the robustness of object detection applications.

Object detectors, much like humans, perform less well on small than on large objects. Because of this, the object size distribution of a dataset influences the average precision a network achieves on that dataset. Therefore, the object size/precision curve of a network might be a better way to compare convolutional object detectors than the average precision over an entire dataset. In this paper we measure the relationship between object size and accuracy for a modern mobile convolutional object detector. We verify that this relationship holds for a different dataset, and that the dataset object size distribution influences the average precision over the entire dataset. We conclude that the object size/accuracy curve might contain more information about a network’s performance than the average precision over an entire dataset.

This thesis presents a novel self-supervised approach of learning visual representations from videos containing human actions. Our approach tackles the complex problem of learning without the need of labeled data by exploring to what extent the ideas successfully used for images can be transferred, adapted and extended to videos for action recognition purposes. We begin by giving a brief introduction to the topic of learning features without having access to a labeled corpora, providing the motivation of our work. We continue with presenting the related research in terms of contrastive learning, action recognition from videos with 3D convolutions and self-supervised techniques for both images and videos. Next, we formalize our approach with regards to the sampling method, the types of spatial and temporal transformations and the contrastive loss used. We evaluate videoSimCLR proposed method in terms of linear evaluation, fully fine-tuning and video retrieval using two popular action recognition datasets, HMDB51 and UCF101. We also explore the extension of another contrastive learning approach to videos, videoMOCO, and compare it with videoSimCLR by means of linear evaluation.

Creating big datasets is often difficult or expensive which causes people to augment their dataset with rendered images. This often fails to significantly improve accuracy due to a difference in distribution between real and rendered datasets. This paper shows that the gap between synthetic and real-world image distributions can be closed by using GANs to convert the synthetic data to a dataset which has the same distribution as the real data. Training this GAN requires only a fraction of the dataset traditionally required to get a high classification accuracy. This converted data can subsequently be used to train a classifier with a higher accuracy than a classifier trained only on the real dataset.

A good action proposal method should generate proposals with high recall and high temporal overlap with groundtruth. The quality of the proposals relies on the labeled data available during training. Obtaining labeled data for untrimmed videos is a time consuming, expensive and error-prone task. The labels obtained are also subjective and the temporal bounds are inconsistent between different human annotators. We propose using a single key frame label for each action instance instead of the start and end point labels to generate temporal proposals. This reduces the number of labeled action frames in the dataset leading to class imbalance. To overcome this, we replace the learning setting with a PU-learning setup. We demonstrate that using key frames as labels give high quality proposals and yield results comparable to using full annotations while being faster to annotate as the exact temporal bounds no longer need to be annotated. We evaluate our method on THUMOS'14 and ActivityNet v1.2 dataset. Further experiments indicate that by combining existing action classifier on our proposals, our method is able to achieve high mean average precision (mAP) for action localization.