Foodborne pathogens represent a significant public health burden. Quantifying the relative importance of various potential sources of foodborne infection is challenging due to data scarcity and uncertainty in empirical studies. Structured expert judgment (SEJ) provides a valuable methodological alternative to gain insights into source attribution of foodborne pathogens. We conducted a SEJ study to attribute human cases of 26 foodborne pathogens in the Netherlands to seven major transmission pathways, 20 food groups, and two animal groups, in a typical post-COVID19 year, using Cooke's classical model. The elicitation process involved snowball recruitment, expertise self-assessment, and a workshop where experts answered calibration questions to capture their uncertainty as input for the model. Subsequently, experts completed the target questions to obtain attributable proportions at the ‘kitchen door’ level. Results indicated that transmission was predominantly (>50%) foodborne for Staphylococcus aureus , Listeria monocytogenes , Yersinia spp., Bacillus cereus , Clostridium perfringens , certain non-typhoidal Salmonella serotypes, Campylobacter spp., hepatitis E virus and Toxoplasma gondii , whereas person-to-person transmission was the primary pathway for astrovirus, rotavirus, norovirus, and sapovirus. Brucella spp. and typhoidal Salmonella were attributed primarily (>85%) to international travel. All other pathogens showed attributions of <50% to any individual pathway. Substantial differences were observed when dividing foodborne transmission into food groups. Key contributors included food handlers and vermin, various meats (e.g., pork, beef, chicken), and shellfish. These SEJ-derived estimates complement existing data by providing pathogen-specific insights in the Dutch context.

This study evaluates five scoring rules, or measures of statistical accuracy, for assessing uncertainty estimates from expert judgment studies and model forecasts. These rules — the Continuously Ranked Probability Score ((Formula presented.)), Kolmogorov-Smirnov ((Formula presented.)), Cramer-von Mises ((Formula presented.)), Anderson Darling ((Formula presented.)), and chi-square test — were applied to 6864 expert uncertainty estimates from 49 Classical Model (CM) studies. We compared their sensitivity to various biases and their ability to serve as performance-based weight for expert estimates. Additionally, the piecewise uniform and Metalog distribution were evaluated for their representation of expert estimates because four of the five rules require interpolating the experts' estimates. Simulating biased estimates reveals varying sensitivity of the considered test statistics to these biases. Expert weights derived using one measure of statistical accuracy were evaluated with other measures to assess their performance. The main conclusions are (1) (Formula presented.) overlooks important biases, while chi-square and (Formula presented.) behave similarly, as do (Formula presented.) and (Formula presented.). (2) All measures except (Formula presented.) agree that performance weighting is superior to equal weighting with respect to statistical accuracy. (3) Neither distributions can effectively predict the position of a removed quantile estimate. These insights show the behavior of different scoring rules for combining uncertainty estimates from expert or models, and extent the knowledge for best-practices.

The problem of failure rate estimation is considered in the light of data acquired from the ATV reporting system. A number of different models for preventive maintenance are discussed which make the failure rate identifiable, as are more general bounding methods which do not require identifiability. These techniques are illustrated on ATV data which is shown in graphs of subsurvival functions. These graphs show, amongst other things, the effect of different maintenance policies on identical systems.

Averaging quantiles as a way of combining experts' judgments is studied both mathematically and empirically. Quantile averaging is equivalent to taking the harmonic mean of densities evaluated at quantile points. A variance shrinkage law is established between equal and harmonic weighting. Data from 49 post-2006 studies are extended to include harmonic weighting in addition to equal and performance-based weighting. It emerges that harmonic weighting has the highest average information and degraded statistical accuracy. The hypothesis that the quantile average is statistically accurate would be rejected at the 5% level in 28 studies and at the 0.1% level in 15 studies. For performance weighting, these numbers are 3 and 1, for equal weighting 2 and 1.

The ice sheets covering Antarctica and Greenland present the greatest uncertainty in, and largest potential contribution to, future sea level rise. The uncertainty arises from a paucity of suitable observations covering the full range of ice sheet behaviors, incomplete understanding of the influences of diverse processes, and limitations in defining key boundary conditions for the numerical models. To investigate the impact of these uncertainties on ice sheet projections we undertook a structured expert judgement study. Here, we interrogate the findings of that study to identify the dominant drivers of uncertainty in projections and their relative importance as a function of ice sheet and time. We find that for the 21st century, Greenland surface melting, in particular the role of surface albedo effects, and West Antarctic ice dynamics, specifically the role of ice shelf buttressing, dominate the uncertainty. The importance of these effects holds under both a high-end 5°C global warming scenario and another that limits global warming to 2°C. During the 22nd century the dominant drivers of uncertainty shift. Under the 5°C scenario, East Antarctic ice dynamics dominate the uncertainty in projections, driven by the possible role of ice flow instabilities. These dynamic effects only become dominant, however, for a temperature scenario above the Paris Agreement 2°C target and beyond 2100. Our findings identify key processes and factors that need to be addressed in future modeling and observational studies in order to reduce uncertainties in ice sheet projections.

Epidemiologic cohort studies have consistently demonstrated that long-term exposure to ambient fine particles (PM_2.5) is associated with mortality. Nevertheless, extrapolating results to understudied locations may involve considerable uncertainty. To explore this issue, this review discusses the evidence for (i) the associated risk of mortality, (ii) the shape of the concentration-response function, (iii) a causal interpretation, and (iv) how the source mix/composition of PM_2.5and population characteristics may alter the effect. The accumulated evidence suggests the following: (i) In the United States, the change in all-cause mortality risk per μg/m³is about 0.8%. (ii) The concentration-response function appears nonlinear. (iii) Causation is overwhelmingly supported. (iv) Fossil fuel combustion-related sources are likely more toxic than others, and age, race, and income may modify the effect. To illustrate the use of our findings in support of a risk assessment in an understudied setting, we consider Kuwait. However, given the complexity of this relationship and the heterogeneity in reported effects, it is unreasonable to think that, in such circumstances, point estimates can be meaningful. Consequently, quantitative probabilistic estimates, which cannot be derived objectively, become essential. Formally elicited expert judgment can provide such estimates, and this review provides the evidence to support an elicitation.

Coronavirus disease 2019 (COVID-19) forecasts from over 100 models are readily available. However, little published information exists regarding the performance of their uncertainty estimates (i.e. probabilistic performance). To evaluate their probabilistic performance, we employ the classical model (CM), an established method typically used to validate expert opinion. In this analysis, we assess both the predictive and probabilistic performance of COVID-19 forecasting models during 2021. We also compare the performance of aggregated forecasts (i.e. ensembles) based on equal and CM performance-based weights to an established ensemble from the Centers for Disease Control and Prevention (CDC). Our analysis of forecasts of COVID-19 mortality from 22 individual models and three ensembles across 49 states indicates that - (i) good predictive performance does not imply good probabilistic performance, and vice versa; (ii) models often provide tight but inaccurate uncertainty estimates; (iii) most models perform worse than a naive baseline model; (iv) both the CDC and CM performance-weighted ensembles perform well; but (v) while the CDC ensemble was more informative, the CM ensemble was more statistically accurate across states. This study presents a worthwhile method for appropriately assessing the performance of probabilistic forecasts and can potentially improve both public health decision-making and COVID-19 modelling.

Drawing on risk methods from volcano crises, we developed a rapid COVID-19 infection model for the partial return of pupils to primary schools in England in June and July 2020, and a full return in September 2020. The model handles uncertainties in key parameters, using a stochastic re-sampling technique, allowing us to evaluate infection levels as a function of COVID-19 prevalence and projected pupil and staff headcounts. Assuming average national adult prevalence, for the first scenario (as at 1 June 2020) we found that between 178 and 924 [90% CI] schools would have at least one infected individual, out of 16 769 primary schools in total. For the second return (July), our estimate ranged between 336 (2%) and 1873 (11%) infected schools. For a full return in September 2020, our projected range was 661 (4%) to 3310 (20%) infected schools, assuming the same prevalence as for 5 June. If national prevalence fell to one-quarter of that, the projected September range would decrease to between 381 (2%) and 900 (5%) schools but would increase to between 2131 (13%) and 9743 (58%) schools if prevalence increased to 4× June level. When regional variations in prevalence and school size distribution were included in the model, a slight decrease in the projected number of infected schools was indicated, but uncertainty on estimates increased markedly. The latter model variant indicated that 82% of infected schools would be in areas where prevalence exceeded the national average and the probability of multiple infected persons in a school would be higher in such areas. Post hoc, our model projections for 1 September 2020 were seen to have been realistic and reasonable (in terms of related uncertainties) when data on schools' infections were released by official agencies following the start of the 2020/2021 academic year.

Regular vines (R-vines) copulas build high dimensional joint densities from arbitrary one-dimensional margins and (conditional) bivariate copula densities. Vine densities enable the computation of all conditional distributions, though the calculations can be numerically intensive. Saturated continuous nonparametric Bayes nets (CNPBN) are regular vines. Computing regression functions from the vine copula density is termed vine regression. The epicycles of regression–including/excluding covariates, interactions, higher order terms, multicollinearity, model fit, transformations, heteroscedasticity, bias–are dispelled. One simply computes the regressions from the vine copula density. Only the question of finding an adequate vine copula remains. Vine regression is applied to a data set from the National Longitudinal Study of Youth relating breastfeeding to IQ. The expected effects of breastfeeding on IQ depend on IQ, on the baseline level of breastfeeding, on the duration of additional breastfeeding and on the values of other covariates. A child given two weeks breastfeeding can expect to increase his/her IQ by 1.5–2 IQ points by adding 10 weeks of breastfeeding, depending on values of other covariates. A child given two years breastfeeding can expect to gain from 0.48–0.65 IQ points from 10 additional weeks. Adding 10 weeks breastfeeding to each of the 3,179 children in this data set has a net present value $50,700,000 according to the Bayes net, compared to $29,000,000 according to the linear regression.

Post-2006 expert judgment data has been extended to 530 experts assessing 580 calibration variables from their fields. New analysis shows that point predictions as medians of combined expert distributions outperform combined medians, and medians of performance weighted combinations outperform medians of equal weighted combinations. Relative to the equal weight combination of medians, using the medians of performance weighted combinations yields a 65% improvement. Using the medians of equally weighted combinations yields a 46% improvement. The Random Expert Hypothesis underlying all performance-blind combination schemes, namely that differences in expert performance reflect random stressors and not persistent properties of the experts, is tested by randomly scrambling expert panels. Generating distributions for a full set of performance metrics, the hypotheses that the original panels’ performance measures are drawn from distributions produced by random scrambling are rejected at significance levels ranging from E−6 to E−12. Random stressors cannot produce the variations in performance seen in the original panels. In- and out-of-sample validation results are updated.

Despite considerable advances in process understanding, numerical modeling, and the observational record of ice sheet contributions to global mean sea-level rise (SLR) since the Fifth Assessment Report (AR5) of the Intergovernmental Panel on Climate Change, severe limitations remain in the predictive capability of ice sheet models. As a consequence, the potential contributions of ice sheets remain the largest source of uncertainty in projecting future SLR. Here, we report the findings of a structured expert judgement study, using unique techniques for modeling correlations between inter- and intra-ice sheet processes and their tail dependences. We find that since the AR5, expert uncertainty has grown, in particular because of uncertain ice dynamic effects. For a +2 °C temperature scenario consistent with the Paris Agreement, we obtain a median estimate of a 26 cm SLR contribution by 2100, with a 95th percentile value of 81 cm. For a +5 °C temperature scenario more consistent with unchecked emissions growth, the corresponding values are 51 and 178 cm, respectively. Inclusion of thermal expansion and glacier contributions results in a global total SLR estimate that exceeds 2 m at the 95th percentile. Our findings support the use of scenarios of 21st century global total SLR exceeding 2 m for planning purposes. Beyond 2100, uncertainty and projected SLR increase rapidly. The 95th percentile ice sheet contribution by 2200, for the +5 °C scenario, is 7.5 m as a result of instabilities coming into play in both West and East Antarctica. Introducing process correlations and tail dependences increases estimates by roughly 15%.

The sharp rise in Oklahoma seismicity since 2009 is due to wastewater injection. The role of injection depth is an open, complex issue, yet critical for hazard assessment and regulation. We developed an advanced Bayesian network to model joint conditional dependencies between spatial, operational, and seismicity parameters. We found that injection depth relative to crystalline basement most strongly correlates with seismic moment release. The joint effects of depth and volume are critical, as injection rate becomes more influential near the basement interface. Restricting injection depths to 200 to 500 meters above basement could reduce annual seismic moment release by a factor of 1.4 to 2.8. Our approach enables identification of subregions where targeted regulation May mitigate effects of induced earthquakes, aiding operators and regulators in wastewater disposal regions.

The Classical Model (CM) is a performance-based approach for mathematically aggregating judgements from multiple experts, when reasoning about target questions under uncertainty. Individual expert performance is assessed against a set of seed questions, items from their field, for which the analyst knows or will know the true values, but the experts do not; the experts are, however, expected to provide accurate and informative distributional judgements that capture these values reliably. Performance is measured according to metrics for each expert’s statistical accuracy and informativeness, and the two metrics are convolved to determine a weight for each expert, with which to modulate their contribution when pooling them together for a final combined assessment of the desired target values. This chapter provides mathematical and practical details of the CM, including describing the method for measuring expert performance and discussing approaches for devising good seed questions.

Validation is the hallmark of science. For expert judgment to contribute to science-based uncertainty quantification, it must become amenable to empirical validation. Using data in which experts quantify uncertainty on variables from their fields whose true values are known post hoc, this chapter explains how validation is performed in the Classical Model for structured expert judgment and reviews results for different combination methods.

Probabilistic thinking can often be unintuitive. This is the case even for simple problems, let alone the more complex ones arising in climate modelling, where disparate information sources need to be combined. The physical models, the natural variability of systems, the measurement errors and their dependence upon the observational period length should be modelled together in order to understand the intricacies of the underlying processes. We use Bayesian networks (BNs) to connect all the above-mentioned pieces in a climate trend uncertainty quantification framework. Inference in such models allows us to observe some seemingly nonsensical outcomes. We argue that they must be pondered rather than discarded until we understand how they arise. We would like to stress that the main focus of this paper is the use of BNs in complex probabilistic settings rather than the application itself.

Background Recently the World Health Organization, Foodborne Disease Burden Epidemiology Reference Group (FERG) estimated that 31 foodborne diseases (FBDs) resulted in over 600 million illnesses and 420,000 deaths worldwide in 2010. Knowing the relative role importance of different foods as exposure routes for key hazards is critical to preventing illness. This study reports the findings of a structured expert elicitation providing globally comparable food source attribution estimates for 11 major FBDs in each of 14 world subregions. Methods and findings We used Cooke’s Classical Model to elicit and aggregate judgments of 73 international experts. Judgments were elicited from each expert individually and aggregated using both equal and performance weights. Performance weighted results are reported as they increased the informativeness of estimates, while retaining accuracy. We report measures of central tendency and uncertainty bounds on food source attribution estimate. For some pathogens we see relatively consistent food source attribution estimates across subregions of the world; for others there is substantial regional variation. For example, for non-typhoidal salmonellosis, pork was of minor importance compared to eggs and poultry meat in the American and African subregions, whereas in the European and Western Pacific subregions the importance of these three food sources were quite similar. Our regional results broadly agree with estimates from earlier European and North American food source attribution research. As in prior food source attribution research, we find relatively wide uncertainty bounds around our median estimates. Conclusions We present the first worldwide estimates of the proportion of specific foodborne diseases attributable to specific food exposure routes. While we find substantial uncertainty around central tendency estimates, we believe these estimates provide the best currently available basis on which to link FBDs and specific foods in many parts of the world, providing guidance for policy actions to control FBDs.