We study the data-driven finite-horizon linear quadratic regularization (LQR) problem reformulated as a semidefinite program (SDP). Our contribution is to propose two novel accelerated first-order methods for solving the resulting SDP. Our methods enjoy adaptive stepsize and adaptive smoothing parameters that speed up convergence and in turn, enhance scalability. Finally, we compare our accelerated first-order methods and show their benefits via numerical simulations on a benchmark LQR example.