Real-world environments require robots to continuously acquire new skills while retain-ing previously learned abilities, all without the need for clearly defined task boundaries. Storing all past data to prevent forgetting is impractical due to storage and privacy con-cerns. To address this, we propose a method that efficiently restores a robot’s proficiency in previously learned tasks over its lifespan. Using an Episodic Memory M, our approach enables experience replay during training and retrieval during testing for local fine-tuning, allowing rapid adaptation to previously encountered problems without explicit task iden-tifiers. Additionally, we introduce a selective weighting mechanism that emphasizes the most challenging segments of retrieved demonstrations, focusing local adaptation where it is most needed. This framework offers a scalable solution for lifelong learning in dy-namic, task-unaware environments, combining retrieval-based local adaptation with selec-tive weighting to enhance robot performance in open-ended scenarios.