Background and purpose: One of the main challenges in external beam radiotherapy treatment of locally advanced cervical cancer patients is dealing with bladder and rectum filling. Organ filling causes inter­fraction motion of the uterus, requiring large treatment planning volumes, or a plan library. Current assessment of tumor position is mainly done by visual inspection of a Cone Beam Computed Tomography (CBCT) scan. Eventually, this can lead to inter-­ and intra-­observer variability when choosing the best treatment plan from the plan library based on bladder filling. The incoming introduction of auto­contouring tools to obtain automatically-­generated (AG) contours of the bladder and the rectum on CBCT scans, allows the easier identification of these organs at risk and consequently, faster localization of the tumor region. However, to rely on these AG contours in the decision of plan selection, it is necessary to know if they have been reliably segmented. The goal of this project is to develop a strategy based on quantitative image features, to evaluate the quality of the AG contours to know if they are suitable for plan selection assessment. Materials & Methods: 140 LACC patients from Erasmus MC were included. For each patient, bladder and rectum contours were obtained from each of the CBCT scans done throughout the treatment (five fractions (CBCT scans) per patient). These contours were automatically­generated using a deep learning­-based auto­segmentation algorithm. Gold­-standard contours were manually delineated in some CBCT scans, but the rest of the automatically­generated contours did not have the corresponding ground-­truth contour, hence they were labeled with a score between 1 (bad quality) and 5 (good quality). For consistency, gold­-standard contours were included in the dataset with the class label 5. The contours were relabeled to have a binary classification problem, and those with label 3 were removed. Each contour volume was divided into three subregions: core region, inner and outer shell. This contour data was used for a comparison study between two supervised machine learning (ML) methodologies: Random forest (RF) networks and Logistic Regression (LR). For both strategies, feature extraction and selection were implemented. In RF methodology, a prior step of dimensionality reduction using principal component analysis (PCA) was performed. In LR, univariate feature selection followed by a multivariate logistic regression analysis was done. Before implementing the classifiers, the dataset was split into a training set and a test set. The ML models were trained using the training set, and they were tested on new unseen data. Predictions on the test data were obtained and used for evaluation of the model's performance using evaluation metrics: accuracy, sensitivity, specificity, confusion matrix, ROC curve, and AUC. Results: The RF classifier performed on the bladder test data with an AUC value of 0.87, while for the LR model, the value obtained was 0.77. The trained RF model identified the accurate and inaccurate bladder contours with a sensitivity of 94% and a specificity of 54%. The trained LR model resulted in a sensitivity of 91% and a specificity of 42%. In the case of the rectum, the RF classifier performance is indicated with the AUC value of 0.89, while the LR model obtained a value of 0.84. In the case of sensitivity and specificity, the RF model got 96% and 38%, and the LR classifier 95% and 38%, respectively. Conclusion: Random forest classifiers give the best results in terms of performance and classification skills for the OARs considered, especially for the bladder. It has been demonstrated that quantitative image features, paired with the corresponding contour class label, can be used for deriving statistical relationships from the data. This allows the identification of contouring errors and classifying the contours based on their quality. With the increasing automation of different steps in the radiotherapy treatment workflows, the automatic contour QA tool developed would be a key step in the process to ensure a faster, more feasible, and consistent plan selection. The tool could act as a support tool for radiotherapy technicians when choosing the plan from the plan library that best fits the daily anatomy of the patient.