In the automotive industry automation is popular and every year car OEMs advance their technology to be able to drive autonomously. Longitudinal control of the vehicles is an important part of the complete autonomous driving system. The difficulty of this control problem lies with changing longitudinal dynamics and the lack of full-state system information. This complicates controller design when using classic model-based approaches such as Optimal Control (OC). Currently the controllers are still manually tuned by control engineers in the
vehicle. This is time consuming and expensive, therefore other methods for controller design such as learning are explored. Reinforcement Learning (RL) is one of those methods. To examine the potential benefits of learning a controller, this work will make a comparison between RL and OC. For RL, an actor-critic structure using deterministic policy gradient is applied. Due to partially observable system dynamics OC is used as an optimal output feedback controller. The comparison complies speed control of an autonomous vehicle. The RL agent will learn a controller by training on a nonlinear high fidelity vehicle model. In this work it was demonstrated that RL can reach the same performance as OC when all environmental settings are comparable. When environmental settings deviate, it was is found that RL outperforms OC. To verify the simulated results all controllers were confirmed in an experimental real-life setting.In conclusion, this proved a promising benefit of learning with respect to classical controller computation, when dealing with partially available system information.