To increase performance of air-to-air missile guidance, a novel guidance law is developed using reinforcement learning methods. This guidance law is based on behavior obtained from optimal control methods and subsequently aims to approximate its performance. The study compares the developed guidance law to a traditional guidance law and optimal solutions. It is established that the novel guidance law outperforms the implemented traditional guidance law in terms of range and time of flight to target. Generated trajectories mimic behavior of optimal control and a large fraction of optimal performance is achieved in terms of range (~90%) and time of flight (~95%).