Testing Deep Reinforcement Learning agents for safety and performance failures is critical but computationally expensive, requiring efficient methods to discover failure-inducing scenarios. Indago, a state-of-the-art testing framework, addresses this by using a Multi-Layer Perceptron (MLP) surrogate model to guide a search algorithm towards potential failures. However, the MLP considers only static initial environment configurations. We propose replacing the static MLP with a Long Short-Term Memory (LSTM) network to leverage the temporal data from agent interaction sequences during training. We evaluated our LSTM-based surrogate against the original MLP within the Indago framework on the HighwayEnv autonomous driving benchmark. Although the best LSTM exhibited lower predictive precision on a held-out test set (11.5% vs. 24%), its integration into the search-based testing pipeline led to a higher failure rate and input diversity. The LSTM guided search discovered more failures on average (34.6% vs. 30%) and explored the input space more thoroughly, achieving 99% cluster coverage and a tenfold increase in input entropy compared to the MLP. Interestingly, training on very short sequences of only two timesteps produced the most effective models, a finding that challenges the initial hypothesis that longer temporal contexts are beneficial for improved performance.