This paper proposes a sparse Bayesian treatment of deep neural networks (DNNs) for system identification. Although DNNs show impressive approximation ability in various fields, several challenges still exist for system identification problems. First, DNNs are known to be too complex that they can easily overfit the training data. Second, the selection of the input regressors for system identification is nontrivial. Third, uncertainty quantification of the model parameters and predictions are necessary. The proposed Bayesian approach offers a principled way to alleviate the above challenges by marginal likelihood/model evidence approximation and structured group sparsity-inducing priors construction. The identification algorithm is derived as an iterative regularised optimisation procedure that can be solved as efficiently as training typical DNNs. Remarkably, an efficient and recursive Hessian calculation method for each layer of DNNs is developed, turning the intractable training/optimisation process into a tractable one. Furthermore, a practical calculation approach based on the Monte-Carlo integration method is derived to quantify the uncertainty of the parameters and predictions. The effectiveness of the proposed Bayesian approach is demonstrated on several linear and nonlinear system identification benchmarks by achieving good and competitive simulation accuracy. The code to reproduce the experimental results is open-sourced and available online.

Hand anthropometry is one of the fundamentals of ergonomic research and product design. Many studies have been conducted to analyze the hand dimensions among different populations, however, the definitions and the numbers of those dimensions were usually selected based on the experience of the researchers and the available equipment. Few studies explored the importance of each hand dimension regarding the 3D shape of the hand. In this paper, we aim to identify the dominant dimensions that influence the hand shape variability while considering the stability of the measurements in practice. A novel four-step research method was proposed where in the first step, based on literature study, we defined 58 landmarks and 53 dimensions for the exploration. In the second step, 80,000 virtual hand models, each had the associated 53 dimensions, were augmented by changing the weights of Principle Components (PCs) of a statistical shape model (SSM). Deep neural networks (DNNs) were used to establish the inverse relationships from the dimensions to the weight of each PC of the hand SSM. Using the structured sparsity learning method, we identified 21 dominant dimensions that represent 90% of the variance of the hand shape. In the third step, two different manual measuring methods were used to evaluate the stability of the measurements in practice. Finally, we selected 16 dominant dimensions with lower measurement variance by synthesizing the findings in Step 2 and 3. It was concluded that the recognized 21 dominant dimensions can be treated as the reference dimensions for anthropometric study and using the selected 16 dominant dimensions with lower measurement variance, ergonomists are able to generate a 3D hand model based on simple measurement tools with an accuracy of 5.9 mm. Though the accuracy is limited, the efforts are minimum, and the results can be used as an indicator in the early stage of research/design.