Recent research has shown that a Learning from Demonstration (LfD) approach is useful for teaching robots flexible skills efficiently, and it opens the possibility for non-expert users to program these skills. When learning from demonstration data, learning frameworks should learn representations that are flexible and can generalize to unseen situations. Within the context of multi-reference frame skill learning, this work proposes a framework to learn such a representation without using task-specific heuristics or pre-segmentation of the demonstrations. Local policies are first learned by fitting the local dynamics with respect to each frame using Gaussian Processes (GP). A classifier that determines the relevance of each frame for every time step is then trained in a self-supervised manner. The uncertainty quantification capability of Gaussian Processes is exploited to improve the performance of the local policies and the self-supervised learning process of the classifier. The framework is validated through multi-frame tasks in simulation as well as on a robotic manipulator with a pick-and-place re-shelving task. Its performance is also compared to that of the Task-Parameterised Gaussian Mixture Model (TPGMM) with simulated and robotic data. In this comparison, the proposed model performs better according to metrics that quantify deviation from the goal at each reference frame and according to similarity measures between demonstrations and their corresponding reproductions.