Considerable overlap exists between emotion and Reinforcement Learning (RL). Emotion influences action selection while RL selects actions based on their anticipated result. Emotions also provide feedback on a situation, reflecting if the situation is desirable or not. The same type of feedback is given in RL based on the results of a state change. Finally, emotion drives adaptations in behaviour while RL continuously updates its policy based on newly gained experience. Because of this overlap, we theorized that a mapping exists from elements of RL to emotions, such that the occurrence and development of these RL emotions matches that in humans. Theory on emotions shows that complex emotions develop later and habituation can be observed in joy and fear. Further supported by theory on mapping situations to emotions, we mapped joy, hope/fear and confirmation emotions. We showed mathematically and in simulations that the development and occurrence of RL emotions with the mapping we created matches expectations based on emotion theory.