When researchers are left with important questions and no historical data is available, such as during the spread of a new virus, then Cooke’s Classical Model (CM) of Structured Expert Judgment (SEJ) is one of the methods that can be used to make predictions. The model aggregates expert assessments into a single prediction, which we call a Decision Maker (DM). This bachelor thesis investigates the robustness and discrepancy of Cooke’s Classical Model using a dataset of 49 different studies. Five types of DMs are analyzed: Equal Weight (EWDM), Global Weight (GWDM), Global Weight Optimized (GWDM opt), Item Weight (IWDM), and Item Weight Optimized (IWDM opt). Robustness is assessed by analyzing how calibration scores of DMs change when individual experts or calibration questions are removed. Furthermore, we look at the Robustness by Distribution Ratio (RDR). Discrepancy is analyzed by comparing the information score obtained from the uniform background measure and the information scores obtained using other DMs as background measures.

The analysis shows that the more experts there are in a study, the more robust the DMs become. For some DMs, robustness also improves with more calibration questions, while for others, no clear trend is observed. Overall, the IWDM, IWDM opt, and GWDM opt are found to be more robust, while the EWDM and the GWDM are the least discrepant. The thesis concludes with recommendations for choosing the best-performing, most robust, or least discrepant DM depending on the number of experts and calibration questions available in a study.