System identification is a mature field in physical sciences and an emerging field in social sciences, with a vast range of applications. Nevertheless, it remains of great focus in academia. The main challenge is the efficient use of data to generate good model fits. System identification involves multi-disciplinary techniques from statistical, mathematical and computational sciences. The typical approaches for dynamic system identification include fuzzy models, non-linear auto regressive models, state-space models, subspace identification models and many others. In this thesis, artificial neural networks are evaluated, among these, as black-box methods known to be capable of universal approximation. With no essential prior information, the identification problem exhibits more difficult challenges. These include the complexity of the resulting models, choice of regressors, and uncertainty quantification. Specifically in this thesis, a Sparse Bayesian Learning approach is proposed, as a solution to these challenges. A practical iterative Bayesian procedure is derived and set to identify six benchmark datasets of three non-linear mechanical processes: Cascaded Tanks, Coupled Electric Drives, Bouc-Wen hysteresis model as well as of three linear mechanical processes: Heat Exchanger, Glass Tube Manufacturing and Hair Dryer.

Human pose estimation, a challenging computer vision task of estimating various human body joints' locations, has a wide range of applications such as pedestrian tracking for autonomous cars, baby monitoring, video surveillance, human action recognition, virtual reality, gaming, gait analysis, etc. A majority of the research on the development of models for the task of human pose estimation has been focused on improving the accuracy of the task which also increases the complexity of the models. These models demand devices with high computational power to be deployed for real-world applications. Even though a lot of research has been focused on estimating the human pose from monocular images taken from cameras, the complexity of the models makes them impossible to be implemented on edge devices and embedded devices like mobile phones that have built-in cameras. This reduces the scope of applications where human pose estimation can be used. To address the issue, the research focuses on improving the performance of a baseline human pose estimation architecture by reducing the model size(number of parameters) and thereby its inference time without a significant loss in the accuracy. To improve the performance of the model, a structured Bayesian compression algorithm is used and the network is compressed by engineering the model based on the uncertainty of the parameters. The results show that the Bayesian compression method reduces the model size by around 65 percent with only a very little drop in the model accuracy. Also, the comparison of the inference time of the original baseline and the compressed model in an android device shows that the inference time is reduced by around 50 percent because of the reduction in the number of operations in the compressed model architecture.

One-Shot Neural Architecture Search (NAS) is a promising method to significantly reduce search time without any separate training. It can be treated as a Network Compression problem on the architecture parameters from an overparameterized network. However, there are two issues associated with most one-shot NAS methods. First, dependencies between a node and its predecessors and successors are often disregarded which result in improper treatment over zero operations. Second, architecture parameters pruning based on their magnitude is questionable. In this thesis, classic Bayesian learning approach is applied to alleviate these two issues. Unlike other NAS methods, we train the over-parameterized network for only one epoch before update network architecture. Impressively, this enabled us to find the optimal architecture in both proxy and proxyless tasks on CIFAR-10 within only 0.2 GPU days using a single GPU. As a byproduct, our approach can be transferred directly to convolutional neural networks compression by enforcing structural sparsity that is able to achieve extremely sparse networks without accuracy deterioration.