Implementation Aspects of a Single-Layer LSBDL Model

Abstract

This paper presents the implementation of the single-layer least-squares-based deep learning (LSBDL) model, optimized using the steepest descent method. As a showcase, the work numerically validates LSBDL’s performance in complex non-linear applications, such as surface fitting. LSBDL is proposed as a transparent deep learning solution, uniquely merging the theoretical robustness and quality control capabilities of the least squares (LS) method with the flexibility of deep learning (DL) models. Unlike conventional black-box DL architectures, the LSBDL framework naturally provides statistical quality assessment metrics, including the covariance matrix of estimated parameters and precision of predicted outcomes. This enables seamless model mis-specification and outlier detection using established reliability theory. The key focus of this study is the model’s demonstrated efficiency, accuracy, and performance in complex non-linear applications. In a complex surface fitting application, the implemented LSBDL model achieved a root mean square error (RMSE) of 0.0021, which is significantly lower than the simulated noise level. Furthermore, the estimated LS residuals are consistent with the simulated (and also estimated) standard deviation of σ = 0.01. The implemented model offers an effective, statistically grounded, and numerically efficient solution for handling complex non-linear problems, particularly those involving heterogeneous and correlated observations. All hyperparameters, initialization steps, optimization, and validation procedures are thoroughly discussed. The Matlab and Python code is freely available at: https://github.com/tud-dasaa/lsbdl.v1.