Feasibility of Neural Network Based Virtual Sensor for Vehicle Unsprung Mass Relative Velocity Estimation

With the automotive industry moving towards automated driving, sensing is becoming an increasingly important part of enabling technology. The virtual sensors allow data fusion from various vehicle sensors and provide a prediction for measurement that is hard or too expensive to measure in another way or in the case of demand on continuous detection. In this paper, virtual sensing is discussed for the case of vehicle suspension control, where information about the relative velocity of the unsprung mass for each vehicle corner is required. The corresponding goal can be identified as a regression task with multi-input sequence input. The hypothesis is that the state-ofart method of Bidirectional Long-Short Term Memory (BiLSMT) can solve it. In this paper, a virtual sensor has been proposed and developed by training a neural network model. The simulations have been performed using an experimentally validated full vehicle model in IPG Carmaker. Simulations provided the reference data which was used for Neural Network (NN) training. The extensive dataset covering 26 scenarios has been used to obtain training, validation and testing data. The Bayesian Search was used to select the best neural network structure using root mean square error as a metric. The best network is made of 167 BiLSTM, 256 fully connected hidden units and 4 output units. Error histograms and spectral analysis of the predicted signal compared to the reference signal are presented. The results demonstrate the good applicability of neural network-based virtual sensors for the estimation of vehicle unsprung mass relative velocity.


Introduction
Nowadays, the automotive industry focuses a lot on Automated Driving (AD) as a very promising solution to improve safety and comfort. The main functional components related to AD include perception, decision-making, and vehicle control. New generation vehicles will need more information to accomplish these three tasks and ensure safe driving without human involvement. The required data can be gained using standard and novel sensors, Vehicle to Everything (V2X) communication, and sensor fusion.
Sensors are crucial components needed for AD, as they provide the data required to perceive the environment and vehicle state estimation [1]. The first group include laser imaging detection and ranging (LIDAR), radio detection and ranging (Radar) [2], ultra-has a crucial influence on the ride comfort and handling of the vehicle [17]. The tuning of the passive suspension has limited applicability to improve the vehicle's ride quality, as comfort and handling are conflicting objectives [18,19]. As a solution, semi-active or active suspension controlled by specialised control algorithms should be used.
During the last decades, many strategies for suspension control have been developed. The majority of them use the velocity of sprung mass and velocity of unsprung masses as input parameters. In commercial systems, velocities are evaluated using integrated data from IMU sensor placed on vehicle sprung mass and integrated/differentiated data from accelerometer and displacement sensor placed on vehicle unsprung mass. The combination of accelerometer and displacement sensor is often used for unsprung mass velocity estimation, as there is much noise in the measurement data, and it is not easy to obtain acceptable results using only one sensor. The development of virtual sensors may solve the need for physical sensors on unsprung mass. Only a few works in this field appeared in recent years.
One of the first works was Milanese et al. [20]. The authors investigated the problem of designing suitable linear virtual sensors with a focus on the estimation of the relative vertical position and velocity between chassis and wheel, using the data from the accelerometer placed on the chassis or wheel. For the task, the Direct Virtual Sensor design technique has been proposed. In 2014, Pletschen and Badur [21] presented a new nonlinear suspension state estimation approach based on Kalman Filter theory and Takagi-Sugeno modelling. Wang et al. [22] proposed the Adaptive Kalman Filter for the purpose to obtain accurate state estimation of a vehicle's suspension system under different road conditions. Jeong et al. [23] proposed a strategy for relative suspension velocity estimation. The method consists of mathematical modelling and direct measurements provided by an IMU sensor. Vertical front suspension forces are calculated using governing equations combined with heave, roll and pitch accelerations measured by the sensor. Out of these forces, relative suspension velocity is derived. Vazquez et al. [24] provided suspension state, road profile and transfer load estimation methodology using deflection sensors, accelerometers and gyrometers. The observation scheme used is a linear Kalman filter. Despite the approach's simplicity, its robustness against uncertainty is remarkable.
As it can be seen, the most commonly used techniques for suspension state estimation involve Kalman Filter; we propose a new approach in this investigation. The main contribution of this paper investigates the feasibility of a Neural Network (NN) model-based virtual sensor for unsprung mass relative velocity estimation. NNs are increasingly used for sequence regression tasks, including mechanical state prediction [25], lateral vehicle velocity estimation [26,27]). This leads to the assumption that similar methods can be used for virtual sensor development to estimate unsprung mass relative velocity. Based on [26], the Bidirectional Long-short time memory (BiLSTM) based NN model may provide better performance compared to 2 layers of Long-short time memory (LSTM) and 1-dimensional convolutional NN (1D CNN). Therefore, BiLSTM was selected for the NN-model of a proposed virtual sensor for unsprung mass relative velocity estimation in the presented research.
The proposed virtual sensor is designed for real-time application for vehicle unsprung mass relative velocity estimation using multiple sensor data of selected sample window size. Designed virtual sensors decrease the need for physical sensors on vehicle unsprung masses. The output can be used for semi-active/active suspension control. The proposed NN model-based virtual sensor was validated and tested on simulated data, with error data analysis, including time and frequency domains. In addition, error distribution was analysed using an error histogram.
The further structure of the paper is as follows. In Section 2, we present (i) a vehicle model, (ii) manoeuvres used for dataset creation, and (iii) the BiLSTM-based Deep NN model and its optimisation algorithm. Section 3 provides the results of vehicle model validation. Hyperparameter optimisation, BiLSTM-based DNN structure selection is provided in this section as well. Finally, we analyse virtual sensor validation and testing results using the Root Mean Squared Error (RMSE) of unsprung mass relative velocity. In Section 4, the discussion is performed, and further steps for system development are presented.

Materials and Methods
In this Section, materials and methods used for data acquisition and development, validation and testing of the virtual sensor for vehicle relative velocity of unsprung mass are described. First, the vehicle mathematical model is presented, including software descriptions and scenarios. Second, NN-model used for the virtual sensor is described, including model structure and hyperparameter optimisations method.

Virtual Sensor for vehicle unsprung mass relative velocity estimation
For vehicles equipped with semi-active or active suspensions, the velocities of sprung and unsprung masses are required to implement a control strategy. Sprung mass velocity can be measured using an IMU sensor placed on the sprung mass. The velocity of unsprung mass commonly is evaluated using data from accelerometer and displacement sensors. Our approach for VS creation was supervised learning with selected sensors input and recorded relative unsprung mass velocity data ( Figure 1).
The vehicle model of Sport Utility Vehicle (SUV) was built in the IPG CarMaker simulation platform. The model has been parametrised and validated using field tests data from the proving ground; vehicle parameters are presented in Table 1. It is a modified electric Range Rover Evoque vehicle with on-board electric motors. The tire parametrisation using experimental data was performed to simulate tire dynamics.  Simulation data included 14 parameters as inputs: sprung mass accelerations in 3 directions (x, y, z), angular rates around these three axes, longitudinal vehicle velocity, front wheels' steering angle and overall vehicle steering angle from steering system, angular velocities of the wheels. The vertical velocities of four unsprung masses were used as an output.
Test tracks and manoeuvres were selected with consideration of data collection for NN training and validation. For training, more dynamic data must be collected; therefore, three scenarios were chosen from the standard scenarios in IPG CarMaker ( Figure 2).
DEU Hockenheimring track has an overall length of 2.6 km. There are some straight sections as well as hard cornering. Various speeds up to 120 km/h and driving manners were applied to cover most vehicle dynamics. Secondly, a rural road roundabouts scenario was selected. This scenario contains braking and acceleration manoeuvres. Also, lane changes and driving around the constant radius turns. The overall length driven by the vehicle on this track is 1.76 km, with the maximum speed was 75 km/h. Furthermore, road DEU Heilbronn was used. This curvy road contains inclinations and declinations. Additional user-defined stopping sections were used to induce acceleration and deceleration manoeuvres. The overall track length covered was 5.79 km with an average speed of 55 km/h. The driver model was modified to three different parameter sets featuring defensive, normal, and aggressive driving styles in all mentioned test scenarios. Different longitudinal, lateral acceleration levels and varying cruising speeds define each driving style.  Table 3), and the data was used for NN testing.

BiLSTM-based Deep Neural Network model
To take advantage of sensor data being sequential, bidirectional long-short term memory (BiLSTM)-based recurrent network was used. This type of neural network (NN) is attributed to recurrent neural networks (RNN) and can learn sequential data models and base prediction on past and current signal values. Model structure, hyperparameter selection experiment and results of validation and testing are presented in this section.
The structure of the selected RNN includes six layers, see Figure 3. NN model layers include sequence input layer, BiLSTM layer, one hidden FC layer, Dropout layer, one output FC layer and regression output. The sequential input layer lets in data from all input channels as a sample sequence of length defined by the window size parameter. BiLSTM layer includes extended LSTM units that propagate signals forward and backwards. That may improve model performance. Hidden FC layer and output FC layers process features extracted in the BiLSTM layer. Units count of the Output FC layer is equal to the outputs count of the virtual sensor. The dropout layer is between the hidden FC and the output FC layer. It reduces NN dependence on single features from BiLSTM, as dropout rate of 0.5 is based.
The input of the network is sequential samples of all inputs M for window size W. The input data samples are fed to the network sequentially, and only the output of BiLSTM for the last input sequence sample is forwarded to the next layer. Therefore, calculation duration in the BiLSTM layer depends linearly on window size W. As each BiLSTM unit does the same calculations, matrix and vector calculation is used. As a result, calculation speed depends less on the overall unit count in NN layers such as BiLSMT and FC. All operations can be done in one cycle of the parallel processing unit and especially graphical processing units (GPU). This network model used in the real-time situation would require holding input data samples that cover selected window size W. The model unit counts are limited to 512 and window size to 51. Therefore, model implementation on low-power devices and automotive on-board computers is feasible.
Even a tiny artificial NN has at least some hyperparameters. They are defined before training and are not optimised during the training process. However, hyperparameters have a significant impact on NN performance and need careful selection or optimisation. As hyperparameters are changed before training, NN training and validation operations are performed for each combination. Therefore, training itself is a long operation, and selecting hyperparameter is times longer. In order to formalise the process of hyperparameter selection, unique optimisation methods are used.
There are three main methods for hyperparameter optimisation: grid search, random search, and Bayesian search. Grid search is the simplest but most computationally expensive. It involves iterating through the defined multi-dimensional grid of hyperparameters combinations. If the step size in this grid is small, iterations count can become huge, and this type of optimisation may take a very long time and many computation resources. It is possible that the grid is too sparse and will not provide the best possible solution. Also, significant time will be spent on unpromising combinations. Random search has no defined grid, and parameter combinations are generated randomly; the random process may be faster compared to grid search if the probability distribution is uniform. Random search has shown that it can find good combinations faster compared to grid search. Bayesian optimisation is similar to random search. Instead of randomly selecting pairs, it analyses previous combinations results, builds the Gaussian probability model, and makes a new combination selection to improve the model (see Figure 4). It saves all combinations, always saving the best, also it provides an estimate that may be not tested yet but provides even better performance. Hence, time is reduced and avoids the drawbacks of grid search.

Vehicle Model Validation and Dataset Generation
The vehicle model has been parametrised and validated on an IPG CarMaker-based simulation platform using field tests data from the proving ground [28]. The obstacle avoidance manoeuvre is presented in Figure 4 as a validation example, achieved accuracy RMSE=0.39 m/s 2 . The test is performed to determine vehicle nature at a severe lane-change manoeuvre. Overall track length is 61 m. All sections should be marked with cones of a minimum height of 500 mm. ISO 3888-2:2011 [29] does not specify a minimum or limit velocity level, but throttle application must be stopped in 2 m after entering section 1. After vehicle model validation, the dataset was generated for NN training, validation and testing. All the roads and manoeuvres are described in Section 2, as well as input and output data.

Results of Hyperparameter Optimisation
During hyperparameter optimisation of the selected NN model, the predefined window size was used for data samples and selectable unit counts in BiLSTM and FC layers. The window sizes of 3, 7, 11, 17, 19, 21, 25, 31 and 51 have been used. The bigger the window size, the more features can be extracted from the signals, especially the lower frequency and more complex features. On the other hand, a bigger window results in linearly increasing computation duration to get the final output of the sequence from the model. Therefore, a trade-off between the duration of computation and accuracy should be introduced.
During the Bayesian Search, the unit counts in BiLSTM, and FC layers are being selected from range [1,512]. The models with each selected combination of unit counts are trained and validated on Nvidia Geforce 2080 Ti graphical processing unit (GPU) using MATLAB. For each window size, the mini-batch size is selected to fill the GPU memory as much as possible, as models are pretty small and in cases of small window size do not utilise GPU completely. The total count of training made was around 350. ADAM optimiser used for training with 0.001 learning rate and training was limited to 30 epochs.
RMSE of unsprung mass relative velocity was used as a metric for optimisation. As two parameters were optimised and one metric was used, a 3D mesh can be drawn to show the optimisation process. In Figure 5, hyperparameter optimisation mesh for a window size of 21 is shown.
Training and validation error graphs showed that RMSE reduces mainly in 15 epochs and not much after 25 epochs. There were no overfitting detected in graphs of the training process. The hyperparameter optimisation process that was wrapping the training would reject overfitting networks based on validation RMSE after each training.
In Figure 5, it can be observed that blue dots correspond to tested combinations, black for following possible combination, red star shows feasible model minimum point, and red mesh shows model mean. Under that mesh, there is a 2D graph with isolines of various colours. Blue points represent low values, and yellow ones correspond to high values of the estimated objective, RMSE. This model helps the search algorithm to select the following points in hyperparameter space. However, the 2D graph of observed and estimated objective function value on the y axis and trial number on the x-axis is provided. After a Bayesian search for 30-60 iterations for each window size, only the best network models were selected for each sample window size. The best models are presented with their hyperparameters, RMSE and relative calculation duration in Table 2. The calculation duration is based on sequential calculation (minibatch size of 1) of all validation samples, which are 19,800 minus (window size -1). As duration depends on processor speed, only relative calculation time is presented compared to the smallest sample window as a baseline, which is about 36 seconds for all samples. The calculation time measurement has been performed three times because of temporal dependencies on hardware performance. The results show that RMSE reduces when window sizes from 3 to 19 are used. Therefore, accuracy is being improved, and relative calculation time increases with sample window size growth. The graphs of relative performance and calculation duration are shown in Figure 6. This graph demonstrates that the size of the window of 19 brings the best improvement in performance compared to the increase in calculation duration; longer sample windows bring diminishing returns. In Figure 6, the best RMSE improvement compared to windows size 3 is achieved using window size 19. RMSE improvement in percent and calculation duration delta shows that the most significant positive delta is achieved at a window size of 7 and 19. After evaluating these graphs, the conclusion can be made that the best sample window size is 19. Also, it brings the best compromise between accuracy and computation duration. Therefore, validation and testing results are provided for the window size 19 in the next section.

Virtual sensor validation and testing
In this section, we provide and analyse virtual sensor outputs on validation and testing data in comparison to reference data, which is unsprung mass relative velocities provided as part of simulated vehicle parameters. The developed virtual sensor estimates unsprung mass relative velocity in the vertical direction. The time series and frequency diagrams of the original and predicted vertical velocity of the unsprung mass is presented in Figure 7.
The time-domain graphs of original simulation output and NN model output for one wheel are provided for validation simulation of Hockenheimring with the normal driving scenario in Figure 7.a.
In order to better present an actual difference between actual and estimated velocity, the corresponding absolute error is shown in Figure 7.b. The max. absolute error is about 0.11 m/s while RMSE is 0.0081. The spikes in absolute error correspond with higher frequency changes in unsprung mass relative velocity and acceleration.
Frequency analysis has been performed to understand how the difference is spread over the spectrum of measurable frequencies. The spectrum relative error was calculated as the delta between predicted and reference signal spectrums divided by the spectrum of the reference signal. The results are shown in Figure 7.c. Measurements are made at 100 samples per second; we provide graphs from 0 to 15 Hz based on primary and secondary ride quality assessment. The observed relative error is mostly between 0 and 10 Hz. The max. relative error is concentrated around 3 Hz. Additionally, in Figure 8, the error histogram shows how predicted signal error values are distributed. In addition to Figure 7.b, Figure 8 shows that most errors are between -0.01384 and 0.01402 m/s. A small disbalance around zero is related to the asymmetric bin ranges, considering that the form of distribution meets Gaussian distribution. Complete testing procedure involved 3 validation and 20 testing simulations, and RMSE results calculated for each wheel (unsprung mass) relative velocity prediction separately and overall RMSE of simulation. The results are shown in Table 3. Furthermore, the training RMSE was included to validate the training correctness, as RMSE on the training set should be smaller than on the validation and testing sets. In the data we see, there is not much noise in higher frequency (Figure 7 c). Therefore, the assumption is that augmenting input sensor data with simulated sensor and process noise may improve generalisation.  Table 3, first, aggressive driving increases RMSE, which may be related to higher acceleration and jerk in vehicle motion, as unsprung mass movement depends on road profile and drivers behaviour. Second, higher vehicle speeds increase the RMSE of prediction, as higher speed means higher frequencies, which can be explained by the fact that higher speed increased the frequency of change in unsprung mass relative velocity. Higher frequency signals caused by abrupt manoeuvres may include signal frequencies that are beyond Nyquist frequency for 100 Hz sampling rate that may lead to aliasing in the spectrum.

Discussion
This research aimed to develop a virtual sensor for unsprung mass relative velocity prediction based on other vehicle movement characterising sensor data, including IMU and steering related sensors. The hypothesis was that this problem could be solved as a sequential signal regression task. Literature review showed that state-of-art multi-input sequence regression could be implemented using recursive NN, and BiLSTM based models are currently achieving impressive results.
In order to implement an NN-based virtual sensor, training, validation and testing datasets are needed. These datasets were acquired from simulations using IPG Carmaker based on the experimentally validated vehicle model. The simulations covered two tracks and three various driving styles for training, one track and three different artificial driving styles for validation, and five manoeuvres for testing with various parameters and road profiles.
The NN structure has been selected for multi-input sequential regression. The hyperparameter optimisation using Bayesian search was made to select the best parameters. The experiment was done to evaluate the developed models' performance as main metrics RMSE and relative calculation time were selected. Based on the simulations, the NN model with a window size of 19 provides the best performance improvement compared to the computation duration increase.
During the simulation studies, virtual sensor output signals were compared to the reference and analysed in the time and frequency domain. The best NN model demonstrates that the predicted signal is close to reference with an RMSE of 0.0081 and a maximum error of 0.11 m/s. Most errors are concentrated between -0.01384 and 0.01402 m/s, and the error distribution is Gaussian. Frequency domain analysis shows that most of the error is between 0 and 10 Hz, with peak values at 3 Hz.
Analysing the achieved results, it can be concluded that virtual sensors using an NN model such as BiLSTM is a viable option for unsprung mass relative velocity prediction. The final NN consists of 167 BiLSTM, 256 hidden FC units and four output FC units. This virtual sensor running on Nvidia Geforce 2080 Ti GPU, even sequentially (with a minibatch size of 1), can process inputs at 500 Hz frequency, and where the limitation is data bus latency. In future, the proposed virtual sensor will be assessed on dSPACE real-time target machine, and later on embedded systems that are closer in performance to modern vehicles microcontrollers.