Assessing Methods for Handling Missing Data Using an LSTM Deep Learning Model in Traffic Forecasting

Assessing Methods for Handling Missing Data Using an LSTM Deep Learning Model in Traffic Forecasting

Büthker, Wouter (TU Delft Electrical Engineering, Mathematics and Computer Science)

Congeduti, E. (mentor)
Iosifidis, G. (graduation committee)

Delft University of Technology

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2023-06-28

Due to the increasing popularity of various types of sensors in traffic management, it has become significantly easier to collect data on traffic flow. However, the integrity of these data sets is often compromised due to missing values resulting from sensor failures, communication errors, and other malfunctions. This study investigates the effect of missing data on the performance of Long Short-Term Memory (LSTM) models in traffic flow prediction and assesses strategies to handle these missing values. By actively removing values from a complete data set, three strategies to handle these missing values are evaluated: dropping null values, replacing them with zero, and linear interpolation. We show that LSTM models are surprisingly resilient to missing data, with little impact on prediction accuracy for up to 40% of missing data, irrespective of the strategy used. For higher proportions of missing data, dropping null values leads to significant performance degradation, while zero-filling and interpolation maintain predictive accuracy. This paper provides insights into the choice of missing data handling strategies in time-series prediction tasks, demonstrating the potential of LSTM models for traffic forecasting under less-than-ideal data conditions

LSTM
Traffic forecasting
Missing data

http://resolver.tudelft.nl/uuid:bbff479b-5653-4406-b4b9-b66591cb410d

Student theses

bachelor thesis

© 2023 Wouter Büthker