Nowadays video plays an important role in the coaching of athletes across many different sports. To make more use of the advantages videos can provide for coaching, the Dutch Sailing team is shifting from manually recording short videos towards continuously recording training sessions. This new recording approach provides opportunities and creates challenges at the same time. In this thesis we present a pipeline to address the problems with the stability of the recording and the first steps towards a Visual Analysis Framework, which leverages the available video data. New information is extracted from the video recordings by detecting and tracking the boat and sailors. Moreover, we semi-automatically highlight interesting intervals in time of a recorded training session. These are the first steps towards an extensive Visual Analysis Framework which has the potential to make the analysis of the videos easier and provide the coaches with tools to improve the analysis of the performance of the sailors.