Drawing on risk methods from volcano crises, we developed a rapid COVID-19 infection model for the partial return of pupils to primary schools in England in June and July 2020, and a full return in September 2020. The model handles uncertainties in key parameters, using a stochastic re-sampling technique, allowing us to evaluate infection levels as a function of COVID-19 prevalence and projected pupil and staff headcounts. Assuming average national adult prevalence, for the first scenario (as at 1 June 2020) we found that between 178 and 924 [90% CI] schools would have at least one infected individual, out of 16 769 primary schools in total. For the second return (July), our estimate ranged between 336 (2%) and 1873 (11%) infected schools. For a full return in September 2020, our projected range was 661 (4%) to 3310 (20%) infected schools, assuming the same prevalence as for 5 June. If national prevalence fell to one-quarter of that, the projected September range would decrease to between 381 (2%) and 900 (5%) schools but would increase to between 2131 (13%) and 9743 (58%) schools if prevalence increased to 4× June level. When regional variations in prevalence and school size distribution were included in the model, a slight decrease in the projected number of infected schools was indicated, but uncertainty on estimates increased markedly. The latter model variant indicated that 82% of infected schools would be in areas where prevalence exceeded the national average and the probability of multiple infected persons in a school would be higher in such areas. Post hoc, our model projections for 1 September 2020 were seen to have been realistic and reasonable (in terms of related uncertainties) when data on schools' infections were released by official agencies following the start of the 2020/2021 academic year. @en

Regular vines (R-vines) copulas build high dimensional joint densities from arbitrary one-dimensional margins and (conditional) bivariate copula densities. Vine densities enable the computation of all conditional distributions, though the calculations can be numerically intensive. Saturated continuous nonparametric Bayes nets (CNPBN) are regular vines. Computing regression functions from the vine copula density is termed vine regression. The epicycles of regression–including/excluding covariates, interactions, higher order terms, multicollinearity, model fit, transformations, heteroscedasticity, bias–are dispelled. One simply computes the regressions from the vine copula density. Only the question of finding an adequate vine copula remains. Vine regression is applied to a data set from the National Longitudinal Study of Youth relating breastfeeding to IQ. The expected effects of breastfeeding on IQ depend on IQ, on the baseline level of breastfeeding, on the duration of additional breastfeeding and on the values of other covariates. A child given two weeks breastfeeding can expect to increase his/her IQ by 1.5–2 IQ points by adding 10 weeks of breastfeeding, depending on values of other covariates. A child given two years breastfeeding can expect to gain from 0.48–0.65 IQ points from 10 additional weeks. Adding 10 weeks breastfeeding to each of the 3,179 children in this data set has a net present value $50,700,000 according to the Bayes net, compared to $29,000,000 according to the linear regression.@en

Post-2006 expert judgment data has been extended to 530 experts assessing 580 calibration variables from their fields. New analysis shows that point predictions as medians of combined expert distributions outperform combined medians, and medians of performance weighted combinations outperform medians of equal weighted combinations. Relative to the equal weight combination of medians, using the medians of performance weighted combinations yields a 65% improvement. Using the medians of equally weighted combinations yields a 46% improvement. The Random Expert Hypothesis underlying all performance-blind combination schemes, namely that differences in expert performance reflect random stressors and not persistent properties of the experts, is tested by randomly scrambling expert panels. Generating distributions for a full set of performance metrics, the hypotheses that the original panels’ performance measures are drawn from distributions produced by random scrambling are rejected at significance levels ranging from E−6 to E−12. Random stressors cannot produce the variations in performance seen in the original panels. In- and out-of-sample validation results are updated.@en

Epidemiologic cohort studies have consistently demonstrated that long-term exposure to ambient fine particles (PM2.5) is associated with mortality. Nevertheless, extrapolating results to understudied locations may involve considerable uncertainty. To explore this issue, this review discusses the evidence for (i) the associated risk of mortality, (ii) the shape of the concentration-response function, (iii) a causal interpretation, and (iv) how the source mix/composition of PM2.5and population characteristics may alter the effect. The accumulated evidence suggests the following: (i) In the United States, the change in all-cause mortality risk per μg/m3is about 0.8%. (ii) The concentration-response function appears nonlinear. (iii) Causation is overwhelmingly supported. (iv) Fossil fuel combustion-related sources are likely more toxic than others, and age, race, and income may modify the effect. To illustrate the use of our findings in support of a risk assessment in an understudied setting, we consider Kuwait. However, given the complexity of this relationship and the heterogeneity in reported effects, it is unreasonable to think that, in such circumstances, point estimates can be meaningful. Consequently, quantitative probabilistic estimates, which cannot be derived objectively, become essential. Formally elicited expert judgment can provide such estimates, and this review provides the evidence to support an elicitation.@en

The ice sheets covering Antarctica and Greenland present the greatest uncertainty in, and largest potential contribution to, future sea level rise. The uncertainty arises from a paucity of suitable observations covering the full range of ice sheet behaviors, incomplete understanding of the influences of diverse processes, and limitations in defining key boundary conditions for the numerical models. To investigate the impact of these uncertainties on ice sheet projections we undertook a structured expert judgement study. Here, we interrogate the findings of that study to identify the dominant drivers of uncertainty in projections and their relative importance as a function of ice sheet and time. We find that for the 21st century, Greenland surface melting, in particular the role of surface albedo effects, and West Antarctic ice dynamics, specifically the role of ice shelf buttressing, dominate the uncertainty. The importance of these effects holds under both a high-end 5°C global warming scenario and another that limits global warming to 2°C. During the 22nd century the dominant drivers of uncertainty shift. Under the 5°C scenario, East Antarctic ice dynamics dominate the uncertainty in projections, driven by the possible role of ice flow instabilities. These dynamic effects only become dominant, however, for a temperature scenario above the Paris Agreement 2°C target and beyond 2100. Our findings identify key processes and factors that need to be addressed in future modeling and observational studies in order to reduce uncertainties in ice sheet projections.@en

Background Recently the World Health Organization, Foodborne Disease Burden Epidemiology Reference Group (FERG) estimated that 31 foodborne diseases (FBDs) resulted in over 600 million illnesses and 420,000 deaths worldwide in 2010. Knowing the relative role importance of different foods as exposure routes for key hazards is critical to preventing illness. This study reports the findings of a structured expert elicitation providing globally comparable food source attribution estimates for 11 major FBDs in each of 14 world subregions. Methods and findings We used Cooke’s Classical Model to elicit and aggregate judgments of 73 international experts. Judgments were elicited from each expert individually and aggregated using both equal and performance weights. Performance weighted results are reported as they increased the informativeness of estimates, while retaining accuracy. We report measures of central tendency and uncertainty bounds on food source attribution estimate. For some pathogens we see relatively consistent food source attribution estimates across subregions of the world; for others there is substantial regional variation. For example, for non-typhoidal salmonellosis, pork was of minor importance compared to eggs and poultry meat in the American and African subregions, whereas in the European and Western Pacific subregions the importance of these three food sources were quite similar. Our regional results broadly agree with estimates from earlier European and North American food source attribution research. As in prior food source attribution research, we find relatively wide uncertainty bounds around our median estimates. Conclusions We present the first worldwide estimates of the proportion of specific foodborne diseases attributable to specific food exposure routes. While we find substantial uncertainty around central tendency estimates, we believe these estimates provide the best currently available basis on which to link FBDs and specific foods in many parts of the world, providing guidance for policy actions to control FBDs.@en

The increase of multidrug resistance and resistance to last-line antibiotics is a major global public health threat. Although surveillance programs provide useful current and historical information on the scale of the problem, the future emergence and spread of antibiotic resistance is uncertain, and quantifying this uncertainty is crucial for guiding decisions about investment in antibiotics and resistance control strategies. Mathematical and statistical models capable of projecting future rates are challenged by the paucity of data and the complexity of the emergence and spread of resistance, but experts have relevant knowledge. We use the Classical Model of structured expert judgment to elicit projections with uncertainty bounds of resistance rates through 2026 for nine pathogen-antibiotic pairs in four European countries and empirically validate the assessments against data on a set of calibration questions. The performance-weighted combination of experts in France, Spain, and the United Kingdom projected that resistance for five pairs on the World Health Organization’s priority pathogens list (E. coli and K. pneumoniae resistant to third-generation cephalosporins and carbapenems and MRSA) would remain below 50% in 2026. In Italy, although upper bounds of 90% credible ranges exceed 50% resistance for some pairs, the medians suggest Italy will sustain or improve its current rates. We compare these expert projections to statistical forecasts based on historical data from the European Antimicrobial Resistance Surveillance Network (EARS-Net). Results from the statistical models differ from each other and from the judgmental forecasts in many cases. The judgmental forecasts include information from the experts about the impact of current and future shifts in infection control, antibiotic usage, and other factors that cannot be easily captured in statistical forecasts, demonstrating the potential of structured expert judgment as a tool for better understanding the uncertainty about future antibiotic resistance.@en

Despite considerable advances in process understanding, numerical modeling, and the observational record of ice sheet contributions to global mean sea-level rise (SLR) since the Fifth Assessment Report (AR5) of the Intergovernmental Panel on Climate Change, severe limitations remain in the predictive capability of ice sheet models. As a consequence, the potential contributions of ice sheets remain the largest source of uncertainty in projecting future SLR. Here, we report the findings of a structured expert judgement study, using unique techniques for modeling correlations between inter- and intra-ice sheet processes and their tail dependences. We find that since the AR5, expert uncertainty has grown, in particular because of uncertain ice dynamic effects. For a +2 °C temperature scenario consistent with the Paris Agreement, we obtain a median estimate of a 26 cm SLR contribution by 2100, with a 95th percentile value of 81 cm. For a +5 °C temperature scenario more consistent with unchecked emissions growth, the corresponding values are 51 and 178 cm, respectively. Inclusion of thermal expansion and glacier contributions results in a global total SLR estimate that exceeds 2 m at the 95th percentile. Our findings support the use of scenarios of 21st century global total SLR exceeding 2 m for planning purposes. Beyond 2100, uncertainty and projected SLR increase rapidly. The 95th percentile ice sheet contribution by 2200, for the +5 °C scenario, is 7.5 m as a result of instabilities coming into play in both West and East Antarctica. Introducing process correlations and tail dependences increases estimates by roughly 15%.@en

We clarify key aspects of the evaluation, by de Vries and van de Wal (2015), of our expert elicitation paper on the contributions of ice sheet melting to sea level rise due to future global temperature rise scenarios (Bamber and Aspinall 2013), and extend the conversation with further analysis of their proposed approach for combining expert uncertainty judgments.@en

Averaging quantiles as a way of combining experts' judgments is studied both mathematically and empirically. Quantile averaging is equivalent to taking the harmonic mean of densities evaluated at quantile points. A variance shrinkage law is established between equal and harmonic weighting. Data from 49 post-2006 studies are extended to include harmonic weighting in addition to equal and performance-based weighting. It emerges that harmonic weighting has the highest average information and degraded statistical accuracy. The hypothesis that the quantile average is statistically accurate would be rejected at the 5% level in 28 studies and at the 0.1% level in 15 studies. For performance weighting, these numbers are 3 and 1, for equal weighting 2 and 1.@en

We update the 2008 TU Delft structured expert judgment database with data from 33 professionally contracted Classical Model studies conducted between 2006 and March 2015 to evaluate its performance relative to other expert aggregation models. We briefly review alternative mathematical aggregation schemes, including harmonic weighting, before focusing on linear pooling of expert judgments with equal weights and performance-based weights. Performance weighting outperforms equal weighting in all but 1 of the 33 studies in-sample. True out-of-sample validation is rarely possible for Classical Model studies, and cross validation techniques that split calibration questions into a training and test set are used instead. Performance weighting incurs an “out-of-sample penalty” and its statistical accuracy out-of-sample is lower than that of equal weighting. However, as a function of training set size, the statistical accuracy of performance-based combinations reaches 75% of the equal weight value when the training set includes 80% of calibration variables. At this point the training set is sufficiently powerful to resolve differences in individual expert performance. The information of performance-based combinations is double that of equal weighting when the training set is at least 50% of the set of calibration variables. Previous out-of-sample validation work used a Total Out-of-Sample Validity Index based on all splits of the calibration questions into training and test subsets, which is expensive to compute and includes small training sets of dubious value. As an alternative, we propose an Out-of-Sample Validity Index based on averaging the product of statistical accuracy and information over all training sets sized at 80% of the calibration set. Performance weighting outperforms equal weighting on this Out-of-Sample Validity Index in 26 of the 33 post-2006 studies; the probability of 26 or more successes on 33 trials if there were no difference between performance weighting and equal weighting is 0.001.@en

Probabilistic thinking can often be unintuitive. This is the case even for simple problems, let alone the more complex ones arising in climate modelling, where disparate information sources need to be combined. The physical models, the natural variability of systems, the measurement errors and their dependence upon the observational period length should be modelled together in order to understand the intricacies of the underlying processes. We use Bayesian networks (BNs) to connect all the above-mentioned pieces in a climate trend uncertainty quantification framework. Inference in such models allows us to observe some seemingly nonsensical outcomes. We argue that they must be pondered rather than discarded until we understand how they arise. We would like to stress that the main focus of this paper is the use of BNs in complex probabilistic settings rather than the application itself.@en

Coronavirus disease 2019 (COVID-19) forecasts from over 100 models are readily available. However, little published information exists regarding the performance of their uncertainty estimates (i.e. probabilistic performance). To evaluate their probabilistic performance, we employ the classical model (CM), an established method typically used to validate expert opinion. In this analysis, we assess both the predictive and probabilistic performance of COVID-19 forecasting models during 2021. We also compare the performance of aggregated forecasts (i.e. ensembles) based on equal and CM performance-based weights to an established ensemble from the Centers for Disease Control and Prevention (CDC). Our analysis of forecasts of COVID-19 mortality from 22 individual models and three ensembles across 49 states indicates that - (i) good predictive performance does not imply good probabilistic performance, and vice versa; (ii) models often provide tight but inaccurate uncertainty estimates; (iii) most models perform worse than a naive baseline model; (iv) both the CDC and CM performance-weighted ensembles perform well; but (v) while the CDC ensemble was more informative, the CM ensemble was more statistically accurate across states. This study presents a worthwhile method for appropriately assessing the performance of probabilistic forecasts and can potentially improve both public health decision-making and COVID-19 modelling. @en

Stochastic generation, i.e., electrical power production by an uncontrolled primary energy source, is expected to play an important role in future power systems. A new power system structure is created due to the large-scale implementation of this small-scale, distributed, non-dispatchable generation; the `horizontally-operated¿ system. Modeling methodologies that can deal with the operational uncertainty introduced by these power units should be used for analyzing the impact of this generation to the system. In this contribution, the principles for this modeling are presented, based on the decoupling of the single stochastic generator behavior (marginal distribution-stochastic unit capacity) from the concurrent behavior of the stochastic generators (stochastic dependence structure-stochastic system dispatch). Subsequently, the stochastic bounds methodology is applied to model the extreme power contribution of the stochastic generation to the system, based on two new sampling concepts (comonotonicity¿countermonotonicity). The application of this methodology to the power system leads to the definition of clusters of positively correlated stochastic generators and the combination of different clusters based on the sampling concepts. The stochastic decomposition and clustering concepts presented in this contribution provide the basis for the application of new uncertainty analysis techniques for the modeling of stochastic generation in power systems. Keywords: Stochastic power generation; Distributed generation; Steady-state analysis; Uncertainty analysis; Monte-Carlo simulation; Risk management@en

Stochastic generation, i.e., electrical power production by an uncontrolled primary energy source, is expected to play an important role in future power systems. A new power system structure is created due to the large-scale implementation of this small-scale, distributed, non-dispatchable generation; the `horizontally-operated¿ system. Modeling methodologies that can deal with the operational uncertainty introduced by these power units should be used for analyzing the impact of this generation to the system. In this contribution, the principles for this modeling are presented, based on the decoupling of the single stochastic generator behavior (marginal distribution-stochastic unit capacity) from the concurrent behavior of the stochastic generators (stochastic dependence structure-stochastic system dispatch). Subsequently, the stochastic bounds methodology is applied to model the extreme power contribution of the stochastic generation to the system, based on two new sampling concepts (comonotonicity¿countermonotonicity). The application of this methodology to the power system leads to the definition of clusters of positively correlated stochastic generators and the combination of different clusters based on the sampling concepts. The stochastic decomposition and clustering concepts presented in this contribution provide the basis for the application of new uncertainty analysis techniques for the modeling of stochastic generation in power systems. Keywords: Stochastic power generation; Distributed generation; Steady-state analysis; Uncertainty analysis; Monte-Carlo simulation; Risk management@en

A structured expert judgment study was organized to obtain input data for a microbial risk-assessment model describing the transmission of campylobacter during broiler-chicken processing in the Netherlands. More specially, the expert study was aimed at quantifying the uncertainty on input parameters of this model and focused on the contamination of broiler-chicken carcasses with campylobacter during processing. Following the protocol for structured expert judgment studies, expert assessments were elicited individually through subjective probability distribution functions. The classical model was used to aggregate the individual experts' distributions in order to obtain a single combined distribution per variable. Three different weighting schemes were applied, including equal weighting and performance-based weighting with and without optimalization of the combined distributions. The individual experts' weights were based on their performance on the seed variables. Results of the various weighting schemes are presented in terms of performance, robustness, and combined distributions of the seed variables and some of the query variables. All three weighting schemes had adequate performance, with the optimized combined distributions significantly outperforming both the equal weight and the nonoptimized combined distributions. Hence, this weighting scheme, having adequate robustness, was chosen for further processing of the results.@en

A structured expert judgment study was organized to obtain input data for a microbial risk-assessment model describing the transmission of campylobacter during broiler-chicken processing in the Netherlands. More specially, the expert study was aimed at quantifying the uncertainty on input parameters of this model and focused on the contamination of broiler-chicken carcasses with campylobacter during processing. Following the protocol for structured expert judgment studies, expert assessments were elicited individually through subjective probability distribution functions. The classical model was used to aggregate the individual experts' distributions in order to obtain a single combined distribution per variable. Three different weighting schemes were applied, including equal weighting and performance-based weighting with and without optimalization of the combined distributions. The individual experts' weights were based on their performance on the seed variables. Results of the various weighting schemes are presented in terms of performance, robustness, and combined distributions of the seed variables and some of the query variables. All three weighting schemes had adequate performance, with the optimized combined distributions significantly outperforming both the equal weight and the nonoptimized combined distributions. Hence, this weighting scheme, having adequate robustness, was chosen for further processing of the results.@en

The goal of this study is to show how to quantify the benefits of accelerated learning about key parameters of the climatic system and use this knowledge to improve decision-making on climate policy. The US social cost of carbon (SCC) methodology is used in innovative ways to value new Earth observing systems (EOSs). The study departs from the strict US SCC methodology, and from previous work, in that net benefits are used instead of only damages to calculate the value of information of the enhanced systems. In other respects the US SCC methodology is followed closely. We compute the surfeit expected net benefits of learning the actionable information earlier, with the enhanced system, versus learning later with existing systems. The enhanced systems are designed to give reliable information about climate sensitivity on accelerated timescales relative to existing systems; therefore, the decision context stipulates that a global reduced emissions path would be deployed upon receiving suitable information on the rate of temperature rise with a suitable level of confidence. By placing the enhanced observing system in a decision context, the SCC enables valuing this system as a real option. Policy relevance Uncertainty in key parameters of the climatic system is often cited as a barrier for near-term reductions of carbon emissions. It is a truism among risk managers that uncertainty costs money, and its reduction has economic value. Advancing policy making under uncertainty requires valuing the reduction in uncertainty. Using CLARREO, a new proposed EOS,as an example, this article applies value of information/real option theory to value the reduction of uncertainty in the decadal rate of temperature rise. The US interagency social cost of carbon directive provides the decision context for the valuations. It is shown that the real option value of the uncertainty reduction, relative to existing observing systems, is a very large multiple of the new system's cost.@en

The goal of this study is to show how to quantify the benefits of accelerated learning about key parameters of the climatic system and use this knowledge to improve decision-making on climate policy. The US social cost of carbon (SCC) methodology is used in innovative ways to value new Earth observing systems (EOSs). The study departs from the strict US SCC methodology, and from previous work, in that net benefits are used instead of only damages to calculate the value of information of the enhanced systems. In other respects the US SCC methodology is followed closely. We compute the surfeit expected net benefits of learning the actionable information earlier, with the enhanced system, versus learning later with existing systems. The enhanced systems are designed to give reliable information about climate sensitivity on accelerated timescales relative to existing systems; therefore, the decision context stipulates that a global reduced emissions path would be deployed upon receiving suitable information on the rate of temperature rise with a suitable level of confidence. By placing the enhanced observing system in a decision context, the SCC enables valuing this system as a real option. Policy relevance Uncertainty in key parameters of the climatic system is often cited as a barrier for near-term reductions of carbon emissions. It is a truism among risk managers that uncertainty costs money, and its reduction has economic value. Advancing policy making under uncertainty requires valuing the reduction in uncertainty. Using CLARREO, a new proposed EOS,as an example, this article applies value of information/real option theory to value the reduction of uncertainty in the decadal rate of temperature rise. The US interagency social cost of carbon directive provides the decision context for the valuations. It is shown that the real option value of the uncertainty reduction, relative to existing observing systems, is a very large multiple of the new system's cost.@en