Safe deployment of neural networks to classify time series in safety-critical applications relies on the ability of the classifier to detect data that does not originate from the same distribution as the training data. The aim of this paper is to propose a framework for detecting whether time series data is sampled from a different distribution than the training data, known as the problem of out-of-distribution (OOD) detection. We propose a novel distance-based OOD method for time series data using a hierarchical clustering method together with dynamic time-warping to measure the difference between a new data instance and the training set. The method is evaluated in the context of mechanical ventilation, a safety critical application, using both simulated and clinical datasets. Results of the mechanical ventilation use case demonstrate that the proposed approach effectively detects out-of-distribution data and improves classification performance in diverse settings.