Is every expert equal?
An analysis of the differences in performance in structured expert judgement
S.J. Harkema (TU Delft - Electrical Engineering, Mathematics and Computer Science)
G. F. Nane – Mentor (TU Delft - Applied Probability)
RM Cooke – Graduation committee member (TU Delft - Applied Probability)
J.G. Spandaw – Coach (TU Delft - Analysis)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
In this thesis the differences in performance scores of experts in the Classical Model for structured expert judgement are analyzed. The underlying assumption in the Classical Model is that variance in performances of experts in a panel is at least partly resultant of the expert's ability to quantify uncertainty. This assumption is tested against the so called Random Expert Hypothesis, that states that these differences are solely resultant of random fluctuations. At the five percent significance level it is concluded that the variation in the combined score of experts cannot exclusively be explained by random fluctuations. When the assumption is tested individually for three different subject fields, health, policy and science, the Random Expert Hypothesis cannot be rejected for both health and policy related studies. Lastly it is shown that the variation in performances between the best and worst expert in a panel strongly correlates with the performance of the best expert against random panels. This indicates that the aggregation of experts according to the scoring rule in the Classical Model may primarily work to diminish the influence of low performing experts.