Enhancing Deep Networks through Customized Iterative Hierarchical Data Augmentation

Abstract

Artificial Neural Networks (ANNs) have emerged as a powerful tool for classification tasks due to their ability to outperform traditional methods. Nevertheless, their effectiveness relies heavily on the availability of large, varied, and labeled datasets, which are often not available. To counter this constraint, data augmentation techniques have emerged, leveraging existing data to generate additional, variant data. Extending these techniques to multi-dimensional time series data, such as the transportation mode detection data considered in this thesis, however, introduces challenges. In response, generative models such as Variational Autoencoders (VAEs) have shown promising advancements.

In this context, this thesis investigates the application of the Iterative Hierarchical Data Aug- mentation (IHDA) algorithm for ANNs, which represents a VAE-based data augmentation technique. The IHDA method utilizes VAEs not only to generate new data samples but also to map existing data to a lower-dimensional latent space, which is then utilized for identifying samples that might require additional training. The proponents of this method, Khan and Fraz, reported an accuracy elevation for the considered transportation mode detection classifier from 83% to 92%. However, due to the absence of publicly accessible code for this algorithm, the initial step of this thesis involved implementing the IHDA algorithm. Further, this research proposed and incorporated advancements like the σ-VAE, aimed to improve the generative capacity of the VAE and to refine its latent space mapping. Additionally, the Kullback-Leibler (KL) divergence was introduced as a similarity metric, aiming to optimize the identification process of samples that require retraining.

Unfortunately, the results reported by Khan and Fraz could not be reproduced in this study. Furthermore, despite the potential shown by the σ-VAE to improve the generative capacity and refine the latent space mapping, along with the enhanced sample identification through the KL divergence, these enhancements did not lead to an overall improvement in the IHDA algorithm. This was primarily attributed to the low generative performance of the VAEs utilized, which also hindered a thorough evaluation of the effectiveness of the IHDA algorithm.

Given these outcomes, it is suggested that future work should focus on employing more complex VAE models with the potential to enhance their generative performance, which, in turn, could improve the IHDA algorithm’s overall effectiveness.