GN
G.F. Nane
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
36 records found
1
Structured Expert Judgment (SEJ) allows decision-makers to account for uncertainty in scenarios where reliable data is unavailable. To combine multiple expert opinions effectively, Cooke’s ClassicalModel scores each expert on their calibration and informativeness. Two calibration scores are available for evaluating discrete binary events, yet both have distinct limitations. Cooke’s chi-squared calibration score relies on an asymptotic distribution and is dependable only for impractically large numbers of calibration questions, whereas the binomial calibration score of Hanea and Nane employs an exact distribution but displays anomalous behaviour whose causes have not been fully established.
This thesis pursues two aims. The first is to characterise the binomial calibration score in detail and to explain its irregularities. The second is to develop a more reliable alternative based on e-values, a modern statistical tool for quantifying evidence against a null hypothesis, offering exact validity without asymptotic
approximations.
The analysis relies on both simulated and empirical data. Simulations were used to isolate the influence of the bin configuration, the information score, and the number of calibration questions, while three datasets, namely SIPS, ACE IDEA, and ACE GJP, were used to test whether the same patterns arise in practice. To address the binomial score’s flaws, two new calibration scores were constructed, denoted CalbinE1 (restricted multiplicative) and CalbinE2 (restricted arithmetic mean), by combining bin-wise e-values and rescaling them via min(1,1/Etotal). Finally, performance-weighted decision makerswere constructed for each calibration score and evaluated through cross-validation over 500 random train/test splits of the empirical datasets.
The binomial calibration score suffers from a masking effect and actively over-penalises almost perfectly calibrated experts, which worsens as sample sizes grow. Altering the bin configuration does not remedy this, as it is shown that these flaws are caused by its fundamental structure: the use of the two-sided mid-p-value and the computation of the score over an aggregate distribution. The proposed e-value calibration scores resolve the masking effect and reward almost perfectly calibrated experts, although they frequently return the maximum value and thereby lose discrimination among well-calibrated experts. When evaluating the aggregated decision makers on the larger ACE GJP dataset, DMχ2 achieved the highest mean under the chi-squared score, whereas DMbinE2 performed best under the binomial score. However, on the sparse SIPS dataset, both e-value decision makers systematically outperformed the established ones across all calibration scores, with DMbinE2 performing best.
It is concluded that the binomial calibration score is structurally unreliable and that the e-value calibration scores could provide a small-sample alternative. The restricted arithmetic mean score CalbinE2 is the most promising, as it alone requires no assumption of asymptotic independence between bins and performs strongly on small datasets. Its principal limitation, a loss of discriminating power induced by the rescaling, identifies the choice of rescaling function as the most fruitful direction for further work. ...
This thesis pursues two aims. The first is to characterise the binomial calibration score in detail and to explain its irregularities. The second is to develop a more reliable alternative based on e-values, a modern statistical tool for quantifying evidence against a null hypothesis, offering exact validity without asymptotic
approximations.
The analysis relies on both simulated and empirical data. Simulations were used to isolate the influence of the bin configuration, the information score, and the number of calibration questions, while three datasets, namely SIPS, ACE IDEA, and ACE GJP, were used to test whether the same patterns arise in practice. To address the binomial score’s flaws, two new calibration scores were constructed, denoted CalbinE1 (restricted multiplicative) and CalbinE2 (restricted arithmetic mean), by combining bin-wise e-values and rescaling them via min(1,1/Etotal). Finally, performance-weighted decision makerswere constructed for each calibration score and evaluated through cross-validation over 500 random train/test splits of the empirical datasets.
The binomial calibration score suffers from a masking effect and actively over-penalises almost perfectly calibrated experts, which worsens as sample sizes grow. Altering the bin configuration does not remedy this, as it is shown that these flaws are caused by its fundamental structure: the use of the two-sided mid-p-value and the computation of the score over an aggregate distribution. The proposed e-value calibration scores resolve the masking effect and reward almost perfectly calibrated experts, although they frequently return the maximum value and thereby lose discrimination among well-calibrated experts. When evaluating the aggregated decision makers on the larger ACE GJP dataset, DMχ2 achieved the highest mean under the chi-squared score, whereas DMbinE2 performed best under the binomial score. However, on the sparse SIPS dataset, both e-value decision makers systematically outperformed the established ones across all calibration scores, with DMbinE2 performing best.
It is concluded that the binomial calibration score is structurally unreliable and that the e-value calibration scores could provide a small-sample alternative. The restricted arithmetic mean score CalbinE2 is the most promising, as it alone requires no assumption of asymptotic independence between bins and performs strongly on small datasets. Its principal limitation, a loss of discriminating power induced by the rescaling, identifies the choice of rescaling function as the most fruitful direction for further work. ...
Structured Expert Judgment (SEJ) allows decision-makers to account for uncertainty in scenarios where reliable data is unavailable. To combine multiple expert opinions effectively, Cooke’s ClassicalModel scores each expert on their calibration and informativeness. Two calibration scores are available for evaluating discrete binary events, yet both have distinct limitations. Cooke’s chi-squared calibration score relies on an asymptotic distribution and is dependable only for impractically large numbers of calibration questions, whereas the binomial calibration score of Hanea and Nane employs an exact distribution but displays anomalous behaviour whose causes have not been fully established.
This thesis pursues two aims. The first is to characterise the binomial calibration score in detail and to explain its irregularities. The second is to develop a more reliable alternative based on e-values, a modern statistical tool for quantifying evidence against a null hypothesis, offering exact validity without asymptotic
approximations.
The analysis relies on both simulated and empirical data. Simulations were used to isolate the influence of the bin configuration, the information score, and the number of calibration questions, while three datasets, namely SIPS, ACE IDEA, and ACE GJP, were used to test whether the same patterns arise in practice. To address the binomial score’s flaws, two new calibration scores were constructed, denoted CalbinE1 (restricted multiplicative) and CalbinE2 (restricted arithmetic mean), by combining bin-wise e-values and rescaling them via min(1,1/Etotal). Finally, performance-weighted decision makerswere constructed for each calibration score and evaluated through cross-validation over 500 random train/test splits of the empirical datasets.
The binomial calibration score suffers from a masking effect and actively over-penalises almost perfectly calibrated experts, which worsens as sample sizes grow. Altering the bin configuration does not remedy this, as it is shown that these flaws are caused by its fundamental structure: the use of the two-sided mid-p-value and the computation of the score over an aggregate distribution. The proposed e-value calibration scores resolve the masking effect and reward almost perfectly calibrated experts, although they frequently return the maximum value and thereby lose discrimination among well-calibrated experts. When evaluating the aggregated decision makers on the larger ACE GJP dataset, DMχ2 achieved the highest mean under the chi-squared score, whereas DMbinE2 performed best under the binomial score. However, on the sparse SIPS dataset, both e-value decision makers systematically outperformed the established ones across all calibration scores, with DMbinE2 performing best.
It is concluded that the binomial calibration score is structurally unreliable and that the e-value calibration scores could provide a small-sample alternative. The restricted arithmetic mean score CalbinE2 is the most promising, as it alone requires no assumption of asymptotic independence between bins and performs strongly on small datasets. Its principal limitation, a loss of discriminating power induced by the rescaling, identifies the choice of rescaling function as the most fruitful direction for further work.
This thesis pursues two aims. The first is to characterise the binomial calibration score in detail and to explain its irregularities. The second is to develop a more reliable alternative based on e-values, a modern statistical tool for quantifying evidence against a null hypothesis, offering exact validity without asymptotic
approximations.
The analysis relies on both simulated and empirical data. Simulations were used to isolate the influence of the bin configuration, the information score, and the number of calibration questions, while three datasets, namely SIPS, ACE IDEA, and ACE GJP, were used to test whether the same patterns arise in practice. To address the binomial score’s flaws, two new calibration scores were constructed, denoted CalbinE1 (restricted multiplicative) and CalbinE2 (restricted arithmetic mean), by combining bin-wise e-values and rescaling them via min(1,1/Etotal). Finally, performance-weighted decision makerswere constructed for each calibration score and evaluated through cross-validation over 500 random train/test splits of the empirical datasets.
The binomial calibration score suffers from a masking effect and actively over-penalises almost perfectly calibrated experts, which worsens as sample sizes grow. Altering the bin configuration does not remedy this, as it is shown that these flaws are caused by its fundamental structure: the use of the two-sided mid-p-value and the computation of the score over an aggregate distribution. The proposed e-value calibration scores resolve the masking effect and reward almost perfectly calibrated experts, although they frequently return the maximum value and thereby lose discrimination among well-calibrated experts. When evaluating the aggregated decision makers on the larger ACE GJP dataset, DMχ2 achieved the highest mean under the chi-squared score, whereas DMbinE2 performed best under the binomial score. However, on the sparse SIPS dataset, both e-value decision makers systematically outperformed the established ones across all calibration scores, with DMbinE2 performing best.
It is concluded that the binomial calibration score is structurally unreliable and that the e-value calibration scores could provide a small-sample alternative. The restricted arithmetic mean score CalbinE2 is the most promising, as it alone requires no assumption of asymptotic independence between bins and performs strongly on small datasets. Its principal limitation, a loss of discriminating power induced by the rescaling, identifies the choice of rescaling function as the most fruitful direction for further work.
Combining clinical registry data collected over several years can increase statistical power, but may also introduce bias when the underlying data generating process changes over time. This thesis investigates which years of the Dutch ColoRectal Audit (DCRA) can be pooled for statistical analysis without introducing practically relevant temporal heterogeneity.
The analysis uses national colorectal cancer surgery data from 2018 to 2024. The data are harmonised using the registry documentation and input from a medical expert. This has resulted in a comprehensive, refined dataset, including 47242 observations and 123 variables.
Temporal stability was assessed at three levels. Marginal changes in individual variables were quantified using effect size measures. Changes in the dependence structure were examined through year specific pairwise correlation matrices and their differences across consecutive years. Finally, a random forest analysis was used as a multivariate analysis to identify variables contributing to differences between years.
Since the effect sizes were classified as small, the corresponding marginal distributions in the univariate setting were sufficiently similar to pool the data of different years of one variable. Combining the analysis of the marginal distributions and the found evidence that the pairwise dependence structure among the selected variables remained stable between consecutive years,
we concluded that these results support pooling the selected variables across the compared consecutive years. The outcomes of the random forest analysis are consistent with the effect size analysis.
As an illustration, the data of individual years and data with the consecutive years pooled were used in exploratory analysis of severe postoperative complications, in which the time between diagnosis and surgery emerged as a potentially relevant variable.
The framework described is applied to DCRA data, but the underlying problem occurs more broadly in evolving medical and administrative registries. The methods used in this thesis can therefore be expanded to other registries as well as to other sectors. ...
The analysis uses national colorectal cancer surgery data from 2018 to 2024. The data are harmonised using the registry documentation and input from a medical expert. This has resulted in a comprehensive, refined dataset, including 47242 observations and 123 variables.
Temporal stability was assessed at three levels. Marginal changes in individual variables were quantified using effect size measures. Changes in the dependence structure were examined through year specific pairwise correlation matrices and their differences across consecutive years. Finally, a random forest analysis was used as a multivariate analysis to identify variables contributing to differences between years.
Since the effect sizes were classified as small, the corresponding marginal distributions in the univariate setting were sufficiently similar to pool the data of different years of one variable. Combining the analysis of the marginal distributions and the found evidence that the pairwise dependence structure among the selected variables remained stable between consecutive years,
we concluded that these results support pooling the selected variables across the compared consecutive years. The outcomes of the random forest analysis are consistent with the effect size analysis.
As an illustration, the data of individual years and data with the consecutive years pooled were used in exploratory analysis of severe postoperative complications, in which the time between diagnosis and surgery emerged as a potentially relevant variable.
The framework described is applied to DCRA data, but the underlying problem occurs more broadly in evolving medical and administrative registries. The methods used in this thesis can therefore be expanded to other registries as well as to other sectors. ...
Combining clinical registry data collected over several years can increase statistical power, but may also introduce bias when the underlying data generating process changes over time. This thesis investigates which years of the Dutch ColoRectal Audit (DCRA) can be pooled for statistical analysis without introducing practically relevant temporal heterogeneity.
The analysis uses national colorectal cancer surgery data from 2018 to 2024. The data are harmonised using the registry documentation and input from a medical expert. This has resulted in a comprehensive, refined dataset, including 47242 observations and 123 variables.
Temporal stability was assessed at three levels. Marginal changes in individual variables were quantified using effect size measures. Changes in the dependence structure were examined through year specific pairwise correlation matrices and their differences across consecutive years. Finally, a random forest analysis was used as a multivariate analysis to identify variables contributing to differences between years.
Since the effect sizes were classified as small, the corresponding marginal distributions in the univariate setting were sufficiently similar to pool the data of different years of one variable. Combining the analysis of the marginal distributions and the found evidence that the pairwise dependence structure among the selected variables remained stable between consecutive years,
we concluded that these results support pooling the selected variables across the compared consecutive years. The outcomes of the random forest analysis are consistent with the effect size analysis.
As an illustration, the data of individual years and data with the consecutive years pooled were used in exploratory analysis of severe postoperative complications, in which the time between diagnosis and surgery emerged as a potentially relevant variable.
The framework described is applied to DCRA data, but the underlying problem occurs more broadly in evolving medical and administrative registries. The methods used in this thesis can therefore be expanded to other registries as well as to other sectors.
The analysis uses national colorectal cancer surgery data from 2018 to 2024. The data are harmonised using the registry documentation and input from a medical expert. This has resulted in a comprehensive, refined dataset, including 47242 observations and 123 variables.
Temporal stability was assessed at three levels. Marginal changes in individual variables were quantified using effect size measures. Changes in the dependence structure were examined through year specific pairwise correlation matrices and their differences across consecutive years. Finally, a random forest analysis was used as a multivariate analysis to identify variables contributing to differences between years.
Since the effect sizes were classified as small, the corresponding marginal distributions in the univariate setting were sufficiently similar to pool the data of different years of one variable. Combining the analysis of the marginal distributions and the found evidence that the pairwise dependence structure among the selected variables remained stable between consecutive years,
we concluded that these results support pooling the selected variables across the compared consecutive years. The outcomes of the random forest analysis are consistent with the effect size analysis.
As an illustration, the data of individual years and data with the consecutive years pooled were used in exploratory analysis of severe postoperative complications, in which the time between diagnosis and surgery emerged as a potentially relevant variable.
The framework described is applied to DCRA data, but the underlying problem occurs more broadly in evolving medical and administrative registries. The methods used in this thesis can therefore be expanded to other registries as well as to other sectors.
Accurate prediction of complications after surgery can positively impact patient treatment. For years, people have been trying to create models to predict complications after surgery. This becomes difficult partly because complications only occur sporadically, causing a class imbalance in the dataset. This class imbalance causes problems when training predictive models.
Patient data for 767 surgeries at Medisch Spectrum Twente between March 2020 and December 2023 was used to compare predictive models for complications after colorectal surgery. We predict two types of complications: any and serious. For the severity of complications, the Clavien-Dindo classification system was used.
This report uses (weighted) logistic regression, random forests, Bayesian networks and TAN classifiers to predict complications after colorectal surgery. The models are evaluated using the AU-ROC, AU-PRC and MCC metrics, taking into account the class imbalance. Additionally, to handle the class imbalance we made use of weighting schemes and sampling methods. Sampling methods affected each method differently, with the most beneficial effects being for the prediction of serious complications. Random forests performed as the best method among the attempted methods, predicting both any complications and serious complications most accurately. ...
Patient data for 767 surgeries at Medisch Spectrum Twente between March 2020 and December 2023 was used to compare predictive models for complications after colorectal surgery. We predict two types of complications: any and serious. For the severity of complications, the Clavien-Dindo classification system was used.
This report uses (weighted) logistic regression, random forests, Bayesian networks and TAN classifiers to predict complications after colorectal surgery. The models are evaluated using the AU-ROC, AU-PRC and MCC metrics, taking into account the class imbalance. Additionally, to handle the class imbalance we made use of weighting schemes and sampling methods. Sampling methods affected each method differently, with the most beneficial effects being for the prediction of serious complications. Random forests performed as the best method among the attempted methods, predicting both any complications and serious complications most accurately. ...
Accurate prediction of complications after surgery can positively impact patient treatment. For years, people have been trying to create models to predict complications after surgery. This becomes difficult partly because complications only occur sporadically, causing a class imbalance in the dataset. This class imbalance causes problems when training predictive models.
Patient data for 767 surgeries at Medisch Spectrum Twente between March 2020 and December 2023 was used to compare predictive models for complications after colorectal surgery. We predict two types of complications: any and serious. For the severity of complications, the Clavien-Dindo classification system was used.
This report uses (weighted) logistic regression, random forests, Bayesian networks and TAN classifiers to predict complications after colorectal surgery. The models are evaluated using the AU-ROC, AU-PRC and MCC metrics, taking into account the class imbalance. Additionally, to handle the class imbalance we made use of weighting schemes and sampling methods. Sampling methods affected each method differently, with the most beneficial effects being for the prediction of serious complications. Random forests performed as the best method among the attempted methods, predicting both any complications and serious complications most accurately.
Patient data for 767 surgeries at Medisch Spectrum Twente between March 2020 and December 2023 was used to compare predictive models for complications after colorectal surgery. We predict two types of complications: any and serious. For the severity of complications, the Clavien-Dindo classification system was used.
This report uses (weighted) logistic regression, random forests, Bayesian networks and TAN classifiers to predict complications after colorectal surgery. The models are evaluated using the AU-ROC, AU-PRC and MCC metrics, taking into account the class imbalance. Additionally, to handle the class imbalance we made use of weighting schemes and sampling methods. Sampling methods affected each method differently, with the most beneficial effects being for the prediction of serious complications. Random forests performed as the best method among the attempted methods, predicting both any complications and serious complications most accurately.
In this thesis, we will be making a quantitative risk model for cloud risks. The Dutch government has held several debates about digital sovereignty and is worried about vendor lock-in. Because of these developments, Auditdienst Rijk is interested in creating a quantitative risk model to help audit the choice of cloud for an information system.
The quantitative risk model proposed in this thesis is a dynamic Bayesian network. We account for various risks, such as a sanction risk, the risk of data being mandated by a foreign government, a data center being flooded, a fire at a data center, hardware issues at a data center, and various cyber attacks.
In order to get the parameters of the model, we consulted various data sources. In the end, the data sources were insufficient for our purpose. So we did a structured expert judgement study for most of the model parameters.
We used our parameters and dynamic Bayesian network in a Monte Carlo simulation to find the probability of failure. We were interested in the probability that all our data centers were simultaneously offline for a consecutive period and the fraction of data lost.
We saw a few trends. The probability of failure increases as the simulation horizon increases. As a consequence, it may be wise to destroy data when it is no longer needed to ensure a decrease in the likelihood that the data will lose its confidentiality. The second trend we saw was that increasing the number of data centers tended to decrease the probability of failure with respect to availability; in other words, the number of consecutive days all data centers were offline decreased. Meanwhile, the probability of failure with respect to confidentiality increased when the number of data centers increased. This is because there is a larger attack surface for risks affecting confidentiality. We also saw that storing a backup in a separate data center was beneficial for the availability of an IT system. ...
The quantitative risk model proposed in this thesis is a dynamic Bayesian network. We account for various risks, such as a sanction risk, the risk of data being mandated by a foreign government, a data center being flooded, a fire at a data center, hardware issues at a data center, and various cyber attacks.
In order to get the parameters of the model, we consulted various data sources. In the end, the data sources were insufficient for our purpose. So we did a structured expert judgement study for most of the model parameters.
We used our parameters and dynamic Bayesian network in a Monte Carlo simulation to find the probability of failure. We were interested in the probability that all our data centers were simultaneously offline for a consecutive period and the fraction of data lost.
We saw a few trends. The probability of failure increases as the simulation horizon increases. As a consequence, it may be wise to destroy data when it is no longer needed to ensure a decrease in the likelihood that the data will lose its confidentiality. The second trend we saw was that increasing the number of data centers tended to decrease the probability of failure with respect to availability; in other words, the number of consecutive days all data centers were offline decreased. Meanwhile, the probability of failure with respect to confidentiality increased when the number of data centers increased. This is because there is a larger attack surface for risks affecting confidentiality. We also saw that storing a backup in a separate data center was beneficial for the availability of an IT system. ...
In this thesis, we will be making a quantitative risk model for cloud risks. The Dutch government has held several debates about digital sovereignty and is worried about vendor lock-in. Because of these developments, Auditdienst Rijk is interested in creating a quantitative risk model to help audit the choice of cloud for an information system.
The quantitative risk model proposed in this thesis is a dynamic Bayesian network. We account for various risks, such as a sanction risk, the risk of data being mandated by a foreign government, a data center being flooded, a fire at a data center, hardware issues at a data center, and various cyber attacks.
In order to get the parameters of the model, we consulted various data sources. In the end, the data sources were insufficient for our purpose. So we did a structured expert judgement study for most of the model parameters.
We used our parameters and dynamic Bayesian network in a Monte Carlo simulation to find the probability of failure. We were interested in the probability that all our data centers were simultaneously offline for a consecutive period and the fraction of data lost.
We saw a few trends. The probability of failure increases as the simulation horizon increases. As a consequence, it may be wise to destroy data when it is no longer needed to ensure a decrease in the likelihood that the data will lose its confidentiality. The second trend we saw was that increasing the number of data centers tended to decrease the probability of failure with respect to availability; in other words, the number of consecutive days all data centers were offline decreased. Meanwhile, the probability of failure with respect to confidentiality increased when the number of data centers increased. This is because there is a larger attack surface for risks affecting confidentiality. We also saw that storing a backup in a separate data center was beneficial for the availability of an IT system.
The quantitative risk model proposed in this thesis is a dynamic Bayesian network. We account for various risks, such as a sanction risk, the risk of data being mandated by a foreign government, a data center being flooded, a fire at a data center, hardware issues at a data center, and various cyber attacks.
In order to get the parameters of the model, we consulted various data sources. In the end, the data sources were insufficient for our purpose. So we did a structured expert judgement study for most of the model parameters.
We used our parameters and dynamic Bayesian network in a Monte Carlo simulation to find the probability of failure. We were interested in the probability that all our data centers were simultaneously offline for a consecutive period and the fraction of data lost.
We saw a few trends. The probability of failure increases as the simulation horizon increases. As a consequence, it may be wise to destroy data when it is no longer needed to ensure a decrease in the likelihood that the data will lose its confidentiality. The second trend we saw was that increasing the number of data centers tended to decrease the probability of failure with respect to availability; in other words, the number of consecutive days all data centers were offline decreased. Meanwhile, the probability of failure with respect to confidentiality increased when the number of data centers increased. This is because there is a larger attack surface for risks affecting confidentiality. We also saw that storing a backup in a separate data center was beneficial for the availability of an IT system.
Copula-Based Regression for Discrete Data
Conditional quantiles and variable selection under discretized covariates
This thesis takes a step towards developing a vine-based regression method tailored to mixed-type (continuous & discrete) data. We visualize the difference between continuous conditional quantile functions of bivariate copulas and their discretized (discretized by binning the continuous variable into bins of equal probability) conditional quantile functions. This showed how discretization into a small number of bins loses dependence, especially in the tails. We show a different binning procedure can improve the preservation of dependence after discretization, motivating the development of methodologies which compute an ‘optimal’ (optimal in the sense of balancing tail dependence with dependence around the median) binning procedure given some data. For application in computing an ‘optimal’ binning procedure, we make an attempt at analytically quantifying the difference between discretized conditional quantiles and continuous conditional quantiles of bivariate copulas, but the analytical derivation is not possible for most settings (with exceptions for bivariate Clayton and bivariate Frank copulas, and the smallest conditioning value of discretized covariate).
From the aforementioned conclusion that discretization loses dependence, we infer a variable
selection measure tailored to mixed-type data should be biased against discretized covariates,
and that this bias should be monotone decreasing with the number of bins of the discretized
covariate. The bias for/against discretized covariates of various variable selection measures is
investigated, in a scenario with a single covariate and a scenario with two covariates. With a single covariate, Pearson’s/polyserial correlation and Kendall’s tau/tau-b were found unsuitable as variable selection measures for mixed-type data, due to a lack of bias against discretized covariates. Conditional log-likelihood and check-loss at the quantile level 0.05 seem nearly identically biased in the bivariate setting, although this should be said with the caveat that all simulation scenarios are homoskedastic. Finally, we show in a three-dimensional setting that correlation between covariates does not seem to affect the predictive performance when both covariates are continuous, but correlation between covariates has significant negative effects on the predictive performance when one of the covariates is discretized. This difference between the effect of discretization in two dimensions and three dimensions should be kept in mind when developing variable selection procedures for mixed-type data. ...
From the aforementioned conclusion that discretization loses dependence, we infer a variable
selection measure tailored to mixed-type data should be biased against discretized covariates,
and that this bias should be monotone decreasing with the number of bins of the discretized
covariate. The bias for/against discretized covariates of various variable selection measures is
investigated, in a scenario with a single covariate and a scenario with two covariates. With a single covariate, Pearson’s/polyserial correlation and Kendall’s tau/tau-b were found unsuitable as variable selection measures for mixed-type data, due to a lack of bias against discretized covariates. Conditional log-likelihood and check-loss at the quantile level 0.05 seem nearly identically biased in the bivariate setting, although this should be said with the caveat that all simulation scenarios are homoskedastic. Finally, we show in a three-dimensional setting that correlation between covariates does not seem to affect the predictive performance when both covariates are continuous, but correlation between covariates has significant negative effects on the predictive performance when one of the covariates is discretized. This difference between the effect of discretization in two dimensions and three dimensions should be kept in mind when developing variable selection procedures for mixed-type data. ...
This thesis takes a step towards developing a vine-based regression method tailored to mixed-type (continuous & discrete) data. We visualize the difference between continuous conditional quantile functions of bivariate copulas and their discretized (discretized by binning the continuous variable into bins of equal probability) conditional quantile functions. This showed how discretization into a small number of bins loses dependence, especially in the tails. We show a different binning procedure can improve the preservation of dependence after discretization, motivating the development of methodologies which compute an ‘optimal’ (optimal in the sense of balancing tail dependence with dependence around the median) binning procedure given some data. For application in computing an ‘optimal’ binning procedure, we make an attempt at analytically quantifying the difference between discretized conditional quantiles and continuous conditional quantiles of bivariate copulas, but the analytical derivation is not possible for most settings (with exceptions for bivariate Clayton and bivariate Frank copulas, and the smallest conditioning value of discretized covariate).
From the aforementioned conclusion that discretization loses dependence, we infer a variable
selection measure tailored to mixed-type data should be biased against discretized covariates,
and that this bias should be monotone decreasing with the number of bins of the discretized
covariate. The bias for/against discretized covariates of various variable selection measures is
investigated, in a scenario with a single covariate and a scenario with two covariates. With a single covariate, Pearson’s/polyserial correlation and Kendall’s tau/tau-b were found unsuitable as variable selection measures for mixed-type data, due to a lack of bias against discretized covariates. Conditional log-likelihood and check-loss at the quantile level 0.05 seem nearly identically biased in the bivariate setting, although this should be said with the caveat that all simulation scenarios are homoskedastic. Finally, we show in a three-dimensional setting that correlation between covariates does not seem to affect the predictive performance when both covariates are continuous, but correlation between covariates has significant negative effects on the predictive performance when one of the covariates is discretized. This difference between the effect of discretization in two dimensions and three dimensions should be kept in mind when developing variable selection procedures for mixed-type data.
From the aforementioned conclusion that discretization loses dependence, we infer a variable
selection measure tailored to mixed-type data should be biased against discretized covariates,
and that this bias should be monotone decreasing with the number of bins of the discretized
covariate. The bias for/against discretized covariates of various variable selection measures is
investigated, in a scenario with a single covariate and a scenario with two covariates. With a single covariate, Pearson’s/polyserial correlation and Kendall’s tau/tau-b were found unsuitable as variable selection measures for mixed-type data, due to a lack of bias against discretized covariates. Conditional log-likelihood and check-loss at the quantile level 0.05 seem nearly identically biased in the bivariate setting, although this should be said with the caveat that all simulation scenarios are homoskedastic. Finally, we show in a three-dimensional setting that correlation between covariates does not seem to affect the predictive performance when both covariates are continuous, but correlation between covariates has significant negative effects on the predictive performance when one of the covariates is discretized. This difference between the effect of discretization in two dimensions and three dimensions should be kept in mind when developing variable selection procedures for mixed-type data.
Decoupled Copula Models
Expert Belief Aggregation through Question-Invariant Spaces
This thesis presents the decoupled copula model, a novel theoretical framework for aggregating expert judgments in structured expert judgment (SEJ) studies. The model's key innovation lies in transforming expert assessments into a "decoupled space" where systematic biases can be identified and corrected while capturing potential inter-expert dependencies. Unlike existing Bayesian SEJ methods, which are limited to linear error metrics, our framework accommodates flexible dependency measures with rigorous theoretical criteria and practical tests for their evaluation. While previous Bayesian approaches acknowledged the possibility of bias correction, they lacked practical procedures for implementing these corrections using historical test data. Our framework addresses these limitations by enabling flexible metric choices for measuring biases and dependencies. Captured biases include under- and overconfidence as well as consistent over- and underestimation. Additionally, we introduce novel calibration criteria that have been proven necessary for perfect aggregation methods, along with interpretable calibration metrics that measure discrepancies from these criteria.
Empirical evaluation on 47 real-world SEJ studies demonstrates superior calibration properties while maintaining competitive predictive performance compared to established methods like the Classical Model. The empirical analysis reveals that inter-expert dependency modeling provides limited benefits, suggesting that systematic bias correction, rather than dependency modeling, drives improvements in aggregation performance in practical applications of our model. ...
Empirical evaluation on 47 real-world SEJ studies demonstrates superior calibration properties while maintaining competitive predictive performance compared to established methods like the Classical Model. The empirical analysis reveals that inter-expert dependency modeling provides limited benefits, suggesting that systematic bias correction, rather than dependency modeling, drives improvements in aggregation performance in practical applications of our model. ...
This thesis presents the decoupled copula model, a novel theoretical framework for aggregating expert judgments in structured expert judgment (SEJ) studies. The model's key innovation lies in transforming expert assessments into a "decoupled space" where systematic biases can be identified and corrected while capturing potential inter-expert dependencies. Unlike existing Bayesian SEJ methods, which are limited to linear error metrics, our framework accommodates flexible dependency measures with rigorous theoretical criteria and practical tests for their evaluation. While previous Bayesian approaches acknowledged the possibility of bias correction, they lacked practical procedures for implementing these corrections using historical test data. Our framework addresses these limitations by enabling flexible metric choices for measuring biases and dependencies. Captured biases include under- and overconfidence as well as consistent over- and underestimation. Additionally, we introduce novel calibration criteria that have been proven necessary for perfect aggregation methods, along with interpretable calibration metrics that measure discrepancies from these criteria.
Empirical evaluation on 47 real-world SEJ studies demonstrates superior calibration properties while maintaining competitive predictive performance compared to established methods like the Classical Model. The empirical analysis reveals that inter-expert dependency modeling provides limited benefits, suggesting that systematic bias correction, rather than dependency modeling, drives improvements in aggregation performance in practical applications of our model.
Empirical evaluation on 47 real-world SEJ studies demonstrates superior calibration properties while maintaining competitive predictive performance compared to established methods like the Classical Model. The empirical analysis reveals that inter-expert dependency modeling provides limited benefits, suggesting that systematic bias correction, rather than dependency modeling, drives improvements in aggregation performance in practical applications of our model.
Green hydrogen is increasingly recognised as a key enabler of the energy transition, offering a carbon-free alternative for hard-to-abate sectors. However, its large-scale deployment remains economically challenging due to high capital expenditures (CAPEX) and production costs. The main cost driver is electricity procurement, given the reliance on affordable renewable energy and the challenges posed by its intermittent and uncertain nature. This underscores the importance of smart power sourcing and energy management strategies to reduce power-related expenses.
This thesis investigates the operational and economic performance of a hybrid renewable energy system designed for green hydrogen production. The system integrates an electrolyser with electricity procured via a wind Power Purchase Agreement (PPA), battery energy storage, and participation in the day-ahead electricity market. Hydrogen is assumed to be sold under a fixed-price offtake agreement, placing hydrogen production at the centre of operational decision-making. The primary objective is to determine the optimal dispatch strategy to maximise the long-term profitability of the hydrogen production plant, and to evaluate whether the addition of battery storage and market-based electricity sourcing enhances economic performance under different system configurations and external conditions.
To determine profit-maximising operational strategies, a mathematical optimisation framework is developed, comparing three solution methods with varying assumptions about foresight and uncertainty. The first method uses a deterministic formulation with perfect foresight, expressed as a mixed-integer linear programming (MILP) problem. This is extended into a rolling horizon approach with limited foresight, and a two-stage stochastic optimisation method that captures forecast uncertainty through correlated scenarios of wind generation and electricity prices. These methods are used to analyse how electricity sourcing strategies, system configuration and uncertainty treatment influence operational and economic indicators such as electrolyser load factor, system profitability, and the levelised cost of hydrogen (LCOH).
The results highlight the potential role of battery storage in reducing electricity costs and unlocking additional revenue by strategically charging and discharging in response to market price fluctuations. However, the associated cost savings are not sufficient to outweigh the high investment costs, making battery storage an unprofitable option under the assumed conditions. Another key finding is that allowing for supplementary electricity sourcing from the day-ahead market, in addition to the PPA, enables cost savings by taking advantage of low or even negative market prices. Nonetheless, to comply with green hydrogen certification requirements, the system must maintain a sufficiently large PPA to ensure a predominantly renewable electricity supply.
Regarding modeling approaches, the deterministic perfect foresight model achieves the highest system profitability, as it is able to optimise operations over the entire planning horizon with full knowledge of future conditions. In addition, it has the lowest computational burden, making it an attractive option for long-term planning and design studies. Although the rolling horizon and stochastic models provide a more realistic representation of uncertainty, the difference in profitability outcomes remains modest, with a maximum deviation of approximately 15 percent. For long-term profitability assessment and system design studies, the deterministic model proves to be sufficiently accurate while offering the advantage of lower computational effort.
By comparing different system configurations and optimisation approaches, this thesis provides insights into the trade-offs between model accuracy, economic performance, and operational robustness in integrated hydrogen systems, offering practical guidance for the design and operation of future green hydrogen facilities. ...
This thesis investigates the operational and economic performance of a hybrid renewable energy system designed for green hydrogen production. The system integrates an electrolyser with electricity procured via a wind Power Purchase Agreement (PPA), battery energy storage, and participation in the day-ahead electricity market. Hydrogen is assumed to be sold under a fixed-price offtake agreement, placing hydrogen production at the centre of operational decision-making. The primary objective is to determine the optimal dispatch strategy to maximise the long-term profitability of the hydrogen production plant, and to evaluate whether the addition of battery storage and market-based electricity sourcing enhances economic performance under different system configurations and external conditions.
To determine profit-maximising operational strategies, a mathematical optimisation framework is developed, comparing three solution methods with varying assumptions about foresight and uncertainty. The first method uses a deterministic formulation with perfect foresight, expressed as a mixed-integer linear programming (MILP) problem. This is extended into a rolling horizon approach with limited foresight, and a two-stage stochastic optimisation method that captures forecast uncertainty through correlated scenarios of wind generation and electricity prices. These methods are used to analyse how electricity sourcing strategies, system configuration and uncertainty treatment influence operational and economic indicators such as electrolyser load factor, system profitability, and the levelised cost of hydrogen (LCOH).
The results highlight the potential role of battery storage in reducing electricity costs and unlocking additional revenue by strategically charging and discharging in response to market price fluctuations. However, the associated cost savings are not sufficient to outweigh the high investment costs, making battery storage an unprofitable option under the assumed conditions. Another key finding is that allowing for supplementary electricity sourcing from the day-ahead market, in addition to the PPA, enables cost savings by taking advantage of low or even negative market prices. Nonetheless, to comply with green hydrogen certification requirements, the system must maintain a sufficiently large PPA to ensure a predominantly renewable electricity supply.
Regarding modeling approaches, the deterministic perfect foresight model achieves the highest system profitability, as it is able to optimise operations over the entire planning horizon with full knowledge of future conditions. In addition, it has the lowest computational burden, making it an attractive option for long-term planning and design studies. Although the rolling horizon and stochastic models provide a more realistic representation of uncertainty, the difference in profitability outcomes remains modest, with a maximum deviation of approximately 15 percent. For long-term profitability assessment and system design studies, the deterministic model proves to be sufficiently accurate while offering the advantage of lower computational effort.
By comparing different system configurations and optimisation approaches, this thesis provides insights into the trade-offs between model accuracy, economic performance, and operational robustness in integrated hydrogen systems, offering practical guidance for the design and operation of future green hydrogen facilities. ...
Green hydrogen is increasingly recognised as a key enabler of the energy transition, offering a carbon-free alternative for hard-to-abate sectors. However, its large-scale deployment remains economically challenging due to high capital expenditures (CAPEX) and production costs. The main cost driver is electricity procurement, given the reliance on affordable renewable energy and the challenges posed by its intermittent and uncertain nature. This underscores the importance of smart power sourcing and energy management strategies to reduce power-related expenses.
This thesis investigates the operational and economic performance of a hybrid renewable energy system designed for green hydrogen production. The system integrates an electrolyser with electricity procured via a wind Power Purchase Agreement (PPA), battery energy storage, and participation in the day-ahead electricity market. Hydrogen is assumed to be sold under a fixed-price offtake agreement, placing hydrogen production at the centre of operational decision-making. The primary objective is to determine the optimal dispatch strategy to maximise the long-term profitability of the hydrogen production plant, and to evaluate whether the addition of battery storage and market-based electricity sourcing enhances economic performance under different system configurations and external conditions.
To determine profit-maximising operational strategies, a mathematical optimisation framework is developed, comparing three solution methods with varying assumptions about foresight and uncertainty. The first method uses a deterministic formulation with perfect foresight, expressed as a mixed-integer linear programming (MILP) problem. This is extended into a rolling horizon approach with limited foresight, and a two-stage stochastic optimisation method that captures forecast uncertainty through correlated scenarios of wind generation and electricity prices. These methods are used to analyse how electricity sourcing strategies, system configuration and uncertainty treatment influence operational and economic indicators such as electrolyser load factor, system profitability, and the levelised cost of hydrogen (LCOH).
The results highlight the potential role of battery storage in reducing electricity costs and unlocking additional revenue by strategically charging and discharging in response to market price fluctuations. However, the associated cost savings are not sufficient to outweigh the high investment costs, making battery storage an unprofitable option under the assumed conditions. Another key finding is that allowing for supplementary electricity sourcing from the day-ahead market, in addition to the PPA, enables cost savings by taking advantage of low or even negative market prices. Nonetheless, to comply with green hydrogen certification requirements, the system must maintain a sufficiently large PPA to ensure a predominantly renewable electricity supply.
Regarding modeling approaches, the deterministic perfect foresight model achieves the highest system profitability, as it is able to optimise operations over the entire planning horizon with full knowledge of future conditions. In addition, it has the lowest computational burden, making it an attractive option for long-term planning and design studies. Although the rolling horizon and stochastic models provide a more realistic representation of uncertainty, the difference in profitability outcomes remains modest, with a maximum deviation of approximately 15 percent. For long-term profitability assessment and system design studies, the deterministic model proves to be sufficiently accurate while offering the advantage of lower computational effort.
By comparing different system configurations and optimisation approaches, this thesis provides insights into the trade-offs between model accuracy, economic performance, and operational robustness in integrated hydrogen systems, offering practical guidance for the design and operation of future green hydrogen facilities.
This thesis investigates the operational and economic performance of a hybrid renewable energy system designed for green hydrogen production. The system integrates an electrolyser with electricity procured via a wind Power Purchase Agreement (PPA), battery energy storage, and participation in the day-ahead electricity market. Hydrogen is assumed to be sold under a fixed-price offtake agreement, placing hydrogen production at the centre of operational decision-making. The primary objective is to determine the optimal dispatch strategy to maximise the long-term profitability of the hydrogen production plant, and to evaluate whether the addition of battery storage and market-based electricity sourcing enhances economic performance under different system configurations and external conditions.
To determine profit-maximising operational strategies, a mathematical optimisation framework is developed, comparing three solution methods with varying assumptions about foresight and uncertainty. The first method uses a deterministic formulation with perfect foresight, expressed as a mixed-integer linear programming (MILP) problem. This is extended into a rolling horizon approach with limited foresight, and a two-stage stochastic optimisation method that captures forecast uncertainty through correlated scenarios of wind generation and electricity prices. These methods are used to analyse how electricity sourcing strategies, system configuration and uncertainty treatment influence operational and economic indicators such as electrolyser load factor, system profitability, and the levelised cost of hydrogen (LCOH).
The results highlight the potential role of battery storage in reducing electricity costs and unlocking additional revenue by strategically charging and discharging in response to market price fluctuations. However, the associated cost savings are not sufficient to outweigh the high investment costs, making battery storage an unprofitable option under the assumed conditions. Another key finding is that allowing for supplementary electricity sourcing from the day-ahead market, in addition to the PPA, enables cost savings by taking advantage of low or even negative market prices. Nonetheless, to comply with green hydrogen certification requirements, the system must maintain a sufficiently large PPA to ensure a predominantly renewable electricity supply.
Regarding modeling approaches, the deterministic perfect foresight model achieves the highest system profitability, as it is able to optimise operations over the entire planning horizon with full knowledge of future conditions. In addition, it has the lowest computational burden, making it an attractive option for long-term planning and design studies. Although the rolling horizon and stochastic models provide a more realistic representation of uncertainty, the difference in profitability outcomes remains modest, with a maximum deviation of approximately 15 percent. For long-term profitability assessment and system design studies, the deterministic model proves to be sufficiently accurate while offering the advantage of lower computational effort.
By comparing different system configurations and optimisation approaches, this thesis provides insights into the trade-offs between model accuracy, economic performance, and operational robustness in integrated hydrogen systems, offering practical guidance for the design and operation of future green hydrogen facilities.
When researchers are left with important questions and no historical data is available, such as during the spread of a new virus, then Cooke’s Classical Model (CM) of Structured Expert Judgment (SEJ) is one of the methods that can be used to make predictions. The model aggregates expert assessments into a single prediction, which we call a Decision Maker (DM). This bachelor thesis investigates the robustness and discrepancy of Cooke’s Classical Model using a dataset of 49 different studies. Five types of DMs are analyzed: Equal Weight (EWDM), Global Weight (GWDM), Global Weight Optimized (GWDM opt), Item Weight (IWDM), and Item Weight Optimized (IWDM opt). Robustness is assessed by analyzing how calibration scores of DMs change when individual experts or calibration questions are removed. Furthermore, we look at the Robustness by Distribution Ratio (RDR). Discrepancy is analyzed by comparing the information score obtained from the uniform background measure and the information scores obtained using other DMs as background measures.
The analysis shows that the more experts there are in a study, the more robust the DMs become. For some DMs, robustness also improves with more calibration questions, while for others, no clear trend is observed. Overall, the IWDM, IWDM opt, and GWDM opt are found to be more robust, while the EWDM and the GWDM are the least discrepant. The thesis concludes with recommendations for choosing the best-performing, most robust, or least discrepant DM depending on the number of experts and calibration questions available in a study. ...
The analysis shows that the more experts there are in a study, the more robust the DMs become. For some DMs, robustness also improves with more calibration questions, while for others, no clear trend is observed. Overall, the IWDM, IWDM opt, and GWDM opt are found to be more robust, while the EWDM and the GWDM are the least discrepant. The thesis concludes with recommendations for choosing the best-performing, most robust, or least discrepant DM depending on the number of experts and calibration questions available in a study. ...
When researchers are left with important questions and no historical data is available, such as during the spread of a new virus, then Cooke’s Classical Model (CM) of Structured Expert Judgment (SEJ) is one of the methods that can be used to make predictions. The model aggregates expert assessments into a single prediction, which we call a Decision Maker (DM). This bachelor thesis investigates the robustness and discrepancy of Cooke’s Classical Model using a dataset of 49 different studies. Five types of DMs are analyzed: Equal Weight (EWDM), Global Weight (GWDM), Global Weight Optimized (GWDM opt), Item Weight (IWDM), and Item Weight Optimized (IWDM opt). Robustness is assessed by analyzing how calibration scores of DMs change when individual experts or calibration questions are removed. Furthermore, we look at the Robustness by Distribution Ratio (RDR). Discrepancy is analyzed by comparing the information score obtained from the uniform background measure and the information scores obtained using other DMs as background measures.
The analysis shows that the more experts there are in a study, the more robust the DMs become. For some DMs, robustness also improves with more calibration questions, while for others, no clear trend is observed. Overall, the IWDM, IWDM opt, and GWDM opt are found to be more robust, while the EWDM and the GWDM are the least discrepant. The thesis concludes with recommendations for choosing the best-performing, most robust, or least discrepant DM depending on the number of experts and calibration questions available in a study.
The analysis shows that the more experts there are in a study, the more robust the DMs become. For some DMs, robustness also improves with more calibration questions, while for others, no clear trend is observed. Overall, the IWDM, IWDM opt, and GWDM opt are found to be more robust, while the EWDM and the GWDM are the least discrepant. The thesis concludes with recommendations for choosing the best-performing, most robust, or least discrepant DM depending on the number of experts and calibration questions available in a study.
In some areas of science, like risk analysis or public health, we often don’t have enough data to base decisions on. In such cases, we rely on experts. We create panels of experts and ask them to evaluate quintiles of interest. But, how many experts do we actually need? And when are there too many experts? These are the main questions of my thesis. To answer these questions I used the Classical Model (CM), which evaluates each ex pert based on two performance scores: the calibration score, which measures the expert’s statistical accuracy, and the information score, which measures how precise the expert’s uncertain assessments are. These scores are used to assign weights when combining expert opinions, and several weighting approaches are examined and compared in this thesis. Using data from the National Institute for Public Health and the Environment (RIVM), this study analyzes how the number of experts in a panel affects the performance of the different Decision Makers. Special attention is given to the role of experts who con sistently underestimate outcomes, and whether their uncertain assessments affect their individual performance and the performance of the aggregated distribution. This re search contributes to understanding how to construct effective expert panels for crucial decisions.
...
In some areas of science, like risk analysis or public health, we often don’t have enough data to base decisions on. In such cases, we rely on experts. We create panels of experts and ask them to evaluate quintiles of interest. But, how many experts do we actually need? And when are there too many experts? These are the main questions of my thesis. To answer these questions I used the Classical Model (CM), which evaluates each ex pert based on two performance scores: the calibration score, which measures the expert’s statistical accuracy, and the information score, which measures how precise the expert’s uncertain assessments are. These scores are used to assign weights when combining expert opinions, and several weighting approaches are examined and compared in this thesis. Using data from the National Institute for Public Health and the Environment (RIVM), this study analyzes how the number of experts in a panel affects the performance of the different Decision Makers. Special attention is given to the role of experts who con sistently underestimate outcomes, and whether their uncertain assessments affect their individual performance and the performance of the aggregated distribution. This re search contributes to understanding how to construct effective expert panels for crucial decisions.
Financial Feasibility for Airport Development
A probabilistic approach to airfield design and cost estimation
Master thesis
(2025)
-
L.M. Scholtens, O. Morales Napoles, P.H.A.J.M. van Gelder, G.F. Nane, Arnaud Bots
Large infrastructure development projects, such as airports, are significantly affected by the uncertainties in the geographic and socio-economic environment. Especially in the early stages of the project, little information about the final specifics of a project is available. As a result, accurately predicting the cost of such a project, as well as its financial feasibility, becomes a complex matter.
This thesis aims to develop a methodology framework, which can support the financial decision-making process in the early stages of airport development projects through an interactive tool. The methodology aims to use Structured Expert Judgment (SEJ) and Bayesian Network (BN) characteristics to provide insight into uncertainties surrounding airport development and its corresponding costs.
Two types of uncertainty are found to directly contribute to the financial risk associated with investment in infrastructure development, based on literature. One type pertains to the technical requirements for airfield pavements. The other type of uncertainty relates to price differences due to fluctuations and inflation.
The airfield pavements (the runway, taxiway, and apron) were determined to be the critical elements in cost estimation. Their dimensions and square meter prices were covered in a SEJ study to obtain probability distributions for these variables. The probability distributions were then combined with rank correlation coefficients calculated from a database with reference airports to implement in a Non-Parametric BN (NPBN).
The result of the research is a Graphical User Interface (GUI): DAiCE. This GUI enables the conditionalisation of the NPBN, predicting pavement dimensions and associated costs through simulation. The tool allows for input of project requirements and analysis of the project’s financial outlook.
DAiCE is proven to produce statistically significant results for airfield design issues. The simulated design values and cost estimates obtained from the tool are in line with those retrieved from reference projects. Though the tool is still bound to some limitations, such the included structures and exploitation models, it is concluded that the model of the airfield design landscape and its implementation in DAiCE can be used to support claims regarding the financial feasibility of an airport development project.
This thesis formulates several recommendations for future research, which are specified for SEJ, the construction and application of BNs, and further development of DAiCE. For SEJ, this includes investigation into different aggregation methods for SEJ, diversification of the expert pool, and expansion of the elicited topics. In addition, it is recommended to further develop the used software for constructing BNs, verification of model parameters, and exploring applicability of BNs in other research fields. Finally, expansion of the developed model, and additional functionalities for DAiCE are proposed. ...
This thesis aims to develop a methodology framework, which can support the financial decision-making process in the early stages of airport development projects through an interactive tool. The methodology aims to use Structured Expert Judgment (SEJ) and Bayesian Network (BN) characteristics to provide insight into uncertainties surrounding airport development and its corresponding costs.
Two types of uncertainty are found to directly contribute to the financial risk associated with investment in infrastructure development, based on literature. One type pertains to the technical requirements for airfield pavements. The other type of uncertainty relates to price differences due to fluctuations and inflation.
The airfield pavements (the runway, taxiway, and apron) were determined to be the critical elements in cost estimation. Their dimensions and square meter prices were covered in a SEJ study to obtain probability distributions for these variables. The probability distributions were then combined with rank correlation coefficients calculated from a database with reference airports to implement in a Non-Parametric BN (NPBN).
The result of the research is a Graphical User Interface (GUI): DAiCE. This GUI enables the conditionalisation of the NPBN, predicting pavement dimensions and associated costs through simulation. The tool allows for input of project requirements and analysis of the project’s financial outlook.
DAiCE is proven to produce statistically significant results for airfield design issues. The simulated design values and cost estimates obtained from the tool are in line with those retrieved from reference projects. Though the tool is still bound to some limitations, such the included structures and exploitation models, it is concluded that the model of the airfield design landscape and its implementation in DAiCE can be used to support claims regarding the financial feasibility of an airport development project.
This thesis formulates several recommendations for future research, which are specified for SEJ, the construction and application of BNs, and further development of DAiCE. For SEJ, this includes investigation into different aggregation methods for SEJ, diversification of the expert pool, and expansion of the elicited topics. In addition, it is recommended to further develop the used software for constructing BNs, verification of model parameters, and exploring applicability of BNs in other research fields. Finally, expansion of the developed model, and additional functionalities for DAiCE are proposed. ...
Large infrastructure development projects, such as airports, are significantly affected by the uncertainties in the geographic and socio-economic environment. Especially in the early stages of the project, little information about the final specifics of a project is available. As a result, accurately predicting the cost of such a project, as well as its financial feasibility, becomes a complex matter.
This thesis aims to develop a methodology framework, which can support the financial decision-making process in the early stages of airport development projects through an interactive tool. The methodology aims to use Structured Expert Judgment (SEJ) and Bayesian Network (BN) characteristics to provide insight into uncertainties surrounding airport development and its corresponding costs.
Two types of uncertainty are found to directly contribute to the financial risk associated with investment in infrastructure development, based on literature. One type pertains to the technical requirements for airfield pavements. The other type of uncertainty relates to price differences due to fluctuations and inflation.
The airfield pavements (the runway, taxiway, and apron) were determined to be the critical elements in cost estimation. Their dimensions and square meter prices were covered in a SEJ study to obtain probability distributions for these variables. The probability distributions were then combined with rank correlation coefficients calculated from a database with reference airports to implement in a Non-Parametric BN (NPBN).
The result of the research is a Graphical User Interface (GUI): DAiCE. This GUI enables the conditionalisation of the NPBN, predicting pavement dimensions and associated costs through simulation. The tool allows for input of project requirements and analysis of the project’s financial outlook.
DAiCE is proven to produce statistically significant results for airfield design issues. The simulated design values and cost estimates obtained from the tool are in line with those retrieved from reference projects. Though the tool is still bound to some limitations, such the included structures and exploitation models, it is concluded that the model of the airfield design landscape and its implementation in DAiCE can be used to support claims regarding the financial feasibility of an airport development project.
This thesis formulates several recommendations for future research, which are specified for SEJ, the construction and application of BNs, and further development of DAiCE. For SEJ, this includes investigation into different aggregation methods for SEJ, diversification of the expert pool, and expansion of the elicited topics. In addition, it is recommended to further develop the used software for constructing BNs, verification of model parameters, and exploring applicability of BNs in other research fields. Finally, expansion of the developed model, and additional functionalities for DAiCE are proposed.
This thesis aims to develop a methodology framework, which can support the financial decision-making process in the early stages of airport development projects through an interactive tool. The methodology aims to use Structured Expert Judgment (SEJ) and Bayesian Network (BN) characteristics to provide insight into uncertainties surrounding airport development and its corresponding costs.
Two types of uncertainty are found to directly contribute to the financial risk associated with investment in infrastructure development, based on literature. One type pertains to the technical requirements for airfield pavements. The other type of uncertainty relates to price differences due to fluctuations and inflation.
The airfield pavements (the runway, taxiway, and apron) were determined to be the critical elements in cost estimation. Their dimensions and square meter prices were covered in a SEJ study to obtain probability distributions for these variables. The probability distributions were then combined with rank correlation coefficients calculated from a database with reference airports to implement in a Non-Parametric BN (NPBN).
The result of the research is a Graphical User Interface (GUI): DAiCE. This GUI enables the conditionalisation of the NPBN, predicting pavement dimensions and associated costs through simulation. The tool allows for input of project requirements and analysis of the project’s financial outlook.
DAiCE is proven to produce statistically significant results for airfield design issues. The simulated design values and cost estimates obtained from the tool are in line with those retrieved from reference projects. Though the tool is still bound to some limitations, such the included structures and exploitation models, it is concluded that the model of the airfield design landscape and its implementation in DAiCE can be used to support claims regarding the financial feasibility of an airport development project.
This thesis formulates several recommendations for future research, which are specified for SEJ, the construction and application of BNs, and further development of DAiCE. For SEJ, this includes investigation into different aggregation methods for SEJ, diversification of the expert pool, and expansion of the elicited topics. In addition, it is recommended to further develop the used software for constructing BNs, verification of model parameters, and exploring applicability of BNs in other research fields. Finally, expansion of the developed model, and additional functionalities for DAiCE are proposed.
LeanSolver: Solving theorems through Large Language Models and Search
Improving Theorem Proving with Proof Assistants and Sequential Monte Carlo in Large Language Models
We consider a subset of simple proving exercises that are part of the Lean 4 tutorials. The exercises consist of a statement, and the task will consist of creating a proof term of the desired type through the use of tactics.
Large Language Models on their own are known to be able to produce syntactically correct pieces of text but are unable to come up with semantically correct pieces of text once very similar solutions are lacking in the dataset it was trained on.
This work reviews the Sequential Monte Carlo with Expectation-Maximization (SMX) algorithm for general reinforcement learning problems, and applies it to theorem proving using LLMs.
This work shows that the SMX algorithm is applicable to tasks where formal verification can be performed on the output, allowing the calculated reward to steer the reasoning process.
We furthermore formally prove a theorem that will be important in showing that the resampling step in SMX is necessary to mitigate sample impoverishment.
The theoretical part of this thesis builds on the theory of the SMX paper by verifying the derivation of the E-step, starting from the Evidence Lower Bound (ELBO). This derivation was missing from the original SMX paper. ...
Large Language Models on their own are known to be able to produce syntactically correct pieces of text but are unable to come up with semantically correct pieces of text once very similar solutions are lacking in the dataset it was trained on.
This work reviews the Sequential Monte Carlo with Expectation-Maximization (SMX) algorithm for general reinforcement learning problems, and applies it to theorem proving using LLMs.
This work shows that the SMX algorithm is applicable to tasks where formal verification can be performed on the output, allowing the calculated reward to steer the reasoning process.
We furthermore formally prove a theorem that will be important in showing that the resampling step in SMX is necessary to mitigate sample impoverishment.
The theoretical part of this thesis builds on the theory of the SMX paper by verifying the derivation of the E-step, starting from the Evidence Lower Bound (ELBO). This derivation was missing from the original SMX paper. ...
We consider a subset of simple proving exercises that are part of the Lean 4 tutorials. The exercises consist of a statement, and the task will consist of creating a proof term of the desired type through the use of tactics.
Large Language Models on their own are known to be able to produce syntactically correct pieces of text but are unable to come up with semantically correct pieces of text once very similar solutions are lacking in the dataset it was trained on.
This work reviews the Sequential Monte Carlo with Expectation-Maximization (SMX) algorithm for general reinforcement learning problems, and applies it to theorem proving using LLMs.
This work shows that the SMX algorithm is applicable to tasks where formal verification can be performed on the output, allowing the calculated reward to steer the reasoning process.
We furthermore formally prove a theorem that will be important in showing that the resampling step in SMX is necessary to mitigate sample impoverishment.
The theoretical part of this thesis builds on the theory of the SMX paper by verifying the derivation of the E-step, starting from the Evidence Lower Bound (ELBO). This derivation was missing from the original SMX paper.
Large Language Models on their own are known to be able to produce syntactically correct pieces of text but are unable to come up with semantically correct pieces of text once very similar solutions are lacking in the dataset it was trained on.
This work reviews the Sequential Monte Carlo with Expectation-Maximization (SMX) algorithm for general reinforcement learning problems, and applies it to theorem proving using LLMs.
This work shows that the SMX algorithm is applicable to tasks where formal verification can be performed on the output, allowing the calculated reward to steer the reasoning process.
We furthermore formally prove a theorem that will be important in showing that the resampling step in SMX is necessary to mitigate sample impoverishment.
The theoretical part of this thesis builds on the theory of the SMX paper by verifying the derivation of the E-step, starting from the Evidence Lower Bound (ELBO). This derivation was missing from the original SMX paper.
Bayesian Networks (BNs) are popular models that represent complex relationships between variables, which can be quantified by Conditional Probability Tables (CPTs) in the discrete case. If data are not sufficient, experts can be involved to assess the probabilities in the CPTs through Structured Expert Judgment (SEJ), which is often a burdensome task. To lighten the elicitation burden, several methods have been developed previously to construct CPTs using a limited number of input parameters, such as the Ranked Nodes Method (RNM), InterBeta and Functional Interpolation. These methods are first analyzed theoretically, where limitations and potential improvements are determined, which were used as inspiration to develop extensions to the methods. The methods and newly developed extensions, including "ExtraBeta" and "AutoRNM", were applied to reconstruct fully elicited CPTs. Finally, simulation studies are performed to find best practices for InterBeta. InterBeta with parent weights is determined as the best-performingmethod, and the AutoRNM and ExtraBeta extensions are worth exploring further.
...
Bayesian Networks (BNs) are popular models that represent complex relationships between variables, which can be quantified by Conditional Probability Tables (CPTs) in the discrete case. If data are not sufficient, experts can be involved to assess the probabilities in the CPTs through Structured Expert Judgment (SEJ), which is often a burdensome task. To lighten the elicitation burden, several methods have been developed previously to construct CPTs using a limited number of input parameters, such as the Ranked Nodes Method (RNM), InterBeta and Functional Interpolation. These methods are first analyzed theoretically, where limitations and potential improvements are determined, which were used as inspiration to develop extensions to the methods. The methods and newly developed extensions, including "ExtraBeta" and "AutoRNM", were applied to reconstruct fully elicited CPTs. Finally, simulation studies are performed to find best practices for InterBeta. InterBeta with parent weights is determined as the best-performingmethod, and the AutoRNM and ExtraBeta extensions are worth exploring further.
Patient level predictions in bowel surgery
Comparing variable selections for modelling on real surgery data
We have studied the predictability of the complications after bowel surgery with the dataset provided by the MST.We see that the correlated data and rare event outcomes lead to the modest performance of the models employed. We have done an extensive preprocessing where we went from a total of 375 variables to 51 explanatory variables. We applied 3 variable selection methods to predict severe complications and any complications in 3 different scenarios (before surgery, right after surgery, and after primary stay). We will see that interoperative variables add valuable information (comparing Scenario 1 with Scenario 2). However, the interoperative variables become less important for predicting the complications post-primary stay (in Scenario 3). Generally, the most important variables are general health scores (WHO score and ASA class) except when predicting complications right after surgery then interoperative variables, for example blood loss, seem to contribute more to complications than variables indicating preoperative health. We will see that post-operative variables are selected for modelling complications after the primary hospital stay, but that these models do not perform adequately since not many complications are present after primary hospital stay. We compared a literature based selection, one resulting from performing marginal testing and an AIC forward variable selection. We see that in the literature many variables are considered important. Sadly we did not extract information about the timing or severity of the complications from the literature. The AIC forward seems prone to select correlated variables which often led to the resulting model to overfit. The marginal importance was more restrictive than the literature model. We have seen that the (unweighted) logistic regression severely underestimates the outcomes and that the weighted logistic regression will lead to a more informative model but with more misclassifications. We included the Threat score (TS) and the Matthews correlation coefficient (MCC) in our validation and confirmed that these are valid measures for scoring the performace of models with a rare outcome.
...
We have studied the predictability of the complications after bowel surgery with the dataset provided by the MST.We see that the correlated data and rare event outcomes lead to the modest performance of the models employed. We have done an extensive preprocessing where we went from a total of 375 variables to 51 explanatory variables. We applied 3 variable selection methods to predict severe complications and any complications in 3 different scenarios (before surgery, right after surgery, and after primary stay). We will see that interoperative variables add valuable information (comparing Scenario 1 with Scenario 2). However, the interoperative variables become less important for predicting the complications post-primary stay (in Scenario 3). Generally, the most important variables are general health scores (WHO score and ASA class) except when predicting complications right after surgery then interoperative variables, for example blood loss, seem to contribute more to complications than variables indicating preoperative health. We will see that post-operative variables are selected for modelling complications after the primary hospital stay, but that these models do not perform adequately since not many complications are present after primary hospital stay. We compared a literature based selection, one resulting from performing marginal testing and an AIC forward variable selection. We see that in the literature many variables are considered important. Sadly we did not extract information about the timing or severity of the complications from the literature. The AIC forward seems prone to select correlated variables which often led to the resulting model to overfit. The marginal importance was more restrictive than the literature model. We have seen that the (unweighted) logistic regression severely underestimates the outcomes and that the weighted logistic regression will lead to a more informative model but with more misclassifications. We included the Threat score (TS) and the Matthews correlation coefficient (MCC) in our validation and confirmed that these are valid measures for scoring the performace of models with a rare outcome.
At crime scenes, various methods can be used to determine how much time has passed since the act happened, for example a body left at the scene, camera footage or eye witnesses. In this thesis, however, a different method shall be used to determine the time of the crime. The time will be determined using blood evidence.
To be exact, the goal of this thesis is to analyse the aging of bloodstains through colour analysis, and constructing a Bayesian network (BN) which can accurately make predictions on the time of deposition of a bloodstain.
The data used for the construction of the BNs was obtained from images provided by the Leiden Institute of Physics (LION). To obtain the data, the bloodstains in the images first need to be isolated from the background. Afterwards, the bloodstains are split into two parts: the inner part of the bloodstain and the complete bloodstain. Both the isolation and the splitting are done using a method called masking. Afterwards they can be converted to RGB (red, green, blue) values and using these RGB values, the following data can be collected for each colour channel of both the inner part of the bloodstain and the whole bloodstain: mean, min, max, variance, and the 5%, 20%, 50%, 80% and 95% quantiles.
Using various subsets of the data, BNs can be constructed. The structure of the BN is determined using a structure-learning algorithm called hill-climbing . To determine the validity of a given structure k-fold cross-validation can be performed using a given loss function. In this thesis, k=5 has been used, and the Mean Squared Error (MSE) has been taken as the loss function. Upon comparison of the MSE, it seems that the best model is given by the red values of a bloodstain. However, even the best performing model found in this thesis still has a considerably poor performance as the BN for the red variables has an MSE of 15149.81.
...
To be exact, the goal of this thesis is to analyse the aging of bloodstains through colour analysis, and constructing a Bayesian network (BN) which can accurately make predictions on the time of deposition of a bloodstain.
The data used for the construction of the BNs was obtained from images provided by the Leiden Institute of Physics (LION). To obtain the data, the bloodstains in the images first need to be isolated from the background. Afterwards, the bloodstains are split into two parts: the inner part of the bloodstain and the complete bloodstain. Both the isolation and the splitting are done using a method called masking. Afterwards they can be converted to RGB (red, green, blue) values and using these RGB values, the following data can be collected for each colour channel of both the inner part of the bloodstain and the whole bloodstain: mean, min, max, variance, and the 5%, 20%, 50%, 80% and 95% quantiles.
Using various subsets of the data, BNs can be constructed. The structure of the BN is determined using a structure-learning algorithm called hill-climbing . To determine the validity of a given structure k-fold cross-validation can be performed using a given loss function. In this thesis, k=5 has been used, and the Mean Squared Error (MSE) has been taken as the loss function. Upon comparison of the MSE, it seems that the best model is given by the red values of a bloodstain. However, even the best performing model found in this thesis still has a considerably poor performance as the BN for the red variables has an MSE of 15149.81.
...
At crime scenes, various methods can be used to determine how much time has passed since the act happened, for example a body left at the scene, camera footage or eye witnesses. In this thesis, however, a different method shall be used to determine the time of the crime. The time will be determined using blood evidence.
To be exact, the goal of this thesis is to analyse the aging of bloodstains through colour analysis, and constructing a Bayesian network (BN) which can accurately make predictions on the time of deposition of a bloodstain.
The data used for the construction of the BNs was obtained from images provided by the Leiden Institute of Physics (LION). To obtain the data, the bloodstains in the images first need to be isolated from the background. Afterwards, the bloodstains are split into two parts: the inner part of the bloodstain and the complete bloodstain. Both the isolation and the splitting are done using a method called masking. Afterwards they can be converted to RGB (red, green, blue) values and using these RGB values, the following data can be collected for each colour channel of both the inner part of the bloodstain and the whole bloodstain: mean, min, max, variance, and the 5%, 20%, 50%, 80% and 95% quantiles.
Using various subsets of the data, BNs can be constructed. The structure of the BN is determined using a structure-learning algorithm called hill-climbing . To determine the validity of a given structure k-fold cross-validation can be performed using a given loss function. In this thesis, k=5 has been used, and the Mean Squared Error (MSE) has been taken as the loss function. Upon comparison of the MSE, it seems that the best model is given by the red values of a bloodstain. However, even the best performing model found in this thesis still has a considerably poor performance as the BN for the red variables has an MSE of 15149.81.
To be exact, the goal of this thesis is to analyse the aging of bloodstains through colour analysis, and constructing a Bayesian network (BN) which can accurately make predictions on the time of deposition of a bloodstain.
The data used for the construction of the BNs was obtained from images provided by the Leiden Institute of Physics (LION). To obtain the data, the bloodstains in the images first need to be isolated from the background. Afterwards, the bloodstains are split into two parts: the inner part of the bloodstain and the complete bloodstain. Both the isolation and the splitting are done using a method called masking. Afterwards they can be converted to RGB (red, green, blue) values and using these RGB values, the following data can be collected for each colour channel of both the inner part of the bloodstain and the whole bloodstain: mean, min, max, variance, and the 5%, 20%, 50%, 80% and 95% quantiles.
Using various subsets of the data, BNs can be constructed. The structure of the BN is determined using a structure-learning algorithm called hill-climbing . To determine the validity of a given structure k-fold cross-validation can be performed using a given loss function. In this thesis, k=5 has been used, and the Mean Squared Error (MSE) has been taken as the loss function. Upon comparison of the MSE, it seems that the best model is given by the red values of a bloodstain. However, even the best performing model found in this thesis still has a considerably poor performance as the BN for the red variables has an MSE of 15149.81.
During this bachelorproject, more research is done to investigate the properties of the Hanea’s and Nane’s calibration score and to look at slight adjustment of Hanea’s and Nane’s calibration score. Both simulated data and real data will be used to investigate the behaviour of these calibration scores.
...
During this bachelorproject, more research is done to investigate the properties of the Hanea’s and Nane’s calibration score and to look at slight adjustment of Hanea’s and Nane’s calibration score. Both simulated data and real data will be used to investigate the behaviour of these calibration scores.
Out of Home advertising is traditional outdoor advertising. Over the last decade the digital Out of Home advertising possibilities have grown substantially. These possibilities include hour-based advertisement schedules, rapidly implemented changes to advertisements and obtaining location data from consumers being exposed to advertisements. Investigating these possibilities further for improving customer engagement and optimising target group reach is of great importance in marketing and marketing science. This thesis will focus on exposure data obtained from vehicle locations in central London. Using this data we want to model the frequency with which individuals see advertisements and research the overlap in exposures of individuals to multiple media vehicles where advertisements are displayed. We focus on modelling exposures of individuals to two vehicles, comparing different types of bivariate distributions. In this thesis we present the comparison of two models: an adapted version of the Sarmanov bivariate distribution and copulas. We refer to these models as the Danaher and Copula model respectively.
The fitting and simulations of the models are based on bivariate data of two specified vehicles, to be able to comment on the overlap between these vehicles. To investigate the performance of the models all combinations of vehicles are modelled after which the results are combined. Both models simulate small frequencies of exposures of individuals adequate, but for more extreme values both models fail to represent the observed data. Especially the simulation of overlap is done poorly by the models. Several approaches have been tried to improve the models' performance, without much success. The analysis shows that the data at hand requires more sophisticated methods to handle joint exposure and more research needs to be done.
In addition the variable distance is added to the research problem, which intuitively has great influence on the overlap between media vehicles. This is done by using 3-dimensional copulas. For this research we modify the data: instead of first fitting and simulating a copula model on the data of two vehicles and then combining the results of all vehicle combinations, we first combine the data from all the combinations of vehicles and repeat the fitting procedure thereafter. Several options for the modelling of these copulas are presented. Similar analysis shows, again, that the nature of the data requires more complex models to handle the overlap while also accounting for the vehicle distance. Finally, using the modified data we look back at the 2-dimensional copula model and perceive an excellent resemblance of the data for the Student-t copula. ...
The fitting and simulations of the models are based on bivariate data of two specified vehicles, to be able to comment on the overlap between these vehicles. To investigate the performance of the models all combinations of vehicles are modelled after which the results are combined. Both models simulate small frequencies of exposures of individuals adequate, but for more extreme values both models fail to represent the observed data. Especially the simulation of overlap is done poorly by the models. Several approaches have been tried to improve the models' performance, without much success. The analysis shows that the data at hand requires more sophisticated methods to handle joint exposure and more research needs to be done.
In addition the variable distance is added to the research problem, which intuitively has great influence on the overlap between media vehicles. This is done by using 3-dimensional copulas. For this research we modify the data: instead of first fitting and simulating a copula model on the data of two vehicles and then combining the results of all vehicle combinations, we first combine the data from all the combinations of vehicles and repeat the fitting procedure thereafter. Several options for the modelling of these copulas are presented. Similar analysis shows, again, that the nature of the data requires more complex models to handle the overlap while also accounting for the vehicle distance. Finally, using the modified data we look back at the 2-dimensional copula model and perceive an excellent resemblance of the data for the Student-t copula. ...
Out of Home advertising is traditional outdoor advertising. Over the last decade the digital Out of Home advertising possibilities have grown substantially. These possibilities include hour-based advertisement schedules, rapidly implemented changes to advertisements and obtaining location data from consumers being exposed to advertisements. Investigating these possibilities further for improving customer engagement and optimising target group reach is of great importance in marketing and marketing science. This thesis will focus on exposure data obtained from vehicle locations in central London. Using this data we want to model the frequency with which individuals see advertisements and research the overlap in exposures of individuals to multiple media vehicles where advertisements are displayed. We focus on modelling exposures of individuals to two vehicles, comparing different types of bivariate distributions. In this thesis we present the comparison of two models: an adapted version of the Sarmanov bivariate distribution and copulas. We refer to these models as the Danaher and Copula model respectively.
The fitting and simulations of the models are based on bivariate data of two specified vehicles, to be able to comment on the overlap between these vehicles. To investigate the performance of the models all combinations of vehicles are modelled after which the results are combined. Both models simulate small frequencies of exposures of individuals adequate, but for more extreme values both models fail to represent the observed data. Especially the simulation of overlap is done poorly by the models. Several approaches have been tried to improve the models' performance, without much success. The analysis shows that the data at hand requires more sophisticated methods to handle joint exposure and more research needs to be done.
In addition the variable distance is added to the research problem, which intuitively has great influence on the overlap between media vehicles. This is done by using 3-dimensional copulas. For this research we modify the data: instead of first fitting and simulating a copula model on the data of two vehicles and then combining the results of all vehicle combinations, we first combine the data from all the combinations of vehicles and repeat the fitting procedure thereafter. Several options for the modelling of these copulas are presented. Similar analysis shows, again, that the nature of the data requires more complex models to handle the overlap while also accounting for the vehicle distance. Finally, using the modified data we look back at the 2-dimensional copula model and perceive an excellent resemblance of the data for the Student-t copula.
The fitting and simulations of the models are based on bivariate data of two specified vehicles, to be able to comment on the overlap between these vehicles. To investigate the performance of the models all combinations of vehicles are modelled after which the results are combined. Both models simulate small frequencies of exposures of individuals adequate, but for more extreme values both models fail to represent the observed data. Especially the simulation of overlap is done poorly by the models. Several approaches have been tried to improve the models' performance, without much success. The analysis shows that the data at hand requires more sophisticated methods to handle joint exposure and more research needs to be done.
In addition the variable distance is added to the research problem, which intuitively has great influence on the overlap between media vehicles. This is done by using 3-dimensional copulas. For this research we modify the data: instead of first fitting and simulating a copula model on the data of two vehicles and then combining the results of all vehicle combinations, we first combine the data from all the combinations of vehicles and repeat the fitting procedure thereafter. Several options for the modelling of these copulas are presented. Similar analysis shows, again, that the nature of the data requires more complex models to handle the overlap while also accounting for the vehicle distance. Finally, using the modified data we look back at the 2-dimensional copula model and perceive an excellent resemblance of the data for the Student-t copula.
Sequential decision making using Bayesian networks and reinforcement learning
To determine the hydrocarbon prospect exploration strategy
In the process of drilling wells to produce hydrocarbons, an exploration strategy is used to determine which wells should be drilled and in which order. This strategy is vital, as a suboptimal drilling sequence will lead to more expenses and fewer gains.
Furthermore, the wells considered in most exploration strategies are geologically dependent. Thus, a realistic model of these dependencies will be beneficial and contribute to a more reliable optimal drilling strategy.
Previous research has shown that modelling similarities between the geological properties of prospect wells in the same region and updating the drilling strategy dynamically after more information is available can add much value. However, the currently developed models are not realistic enough to use in practice. Previous models separated the process into two parts, first modelling the geological success with deterministic rewards and then applying dynamic programming to obtain the sequential drilling strategy.
This thesis proposes a new model with the addition of flexible uncertainty and dependency models between the geological characteristics in the hydrocarbon volume calculation of a reservoir. For this purpose, non-parametric Bayesian networks in combination with copulae are used. This model and the input format of the required data align with how data is gathered in the oil and gas industry. The dependencies are modelled based on simple expert assessments.
After the new model for the geological properties of prospect wells and their dependencies is constructed, a decision policy that hinges on the observations of previously drilled wells is constructed. Previously used dynamic programming becomes infeasible with continuous variables and with a large number of wells. Therefore, the reinforcement learning algorithm Q-learning is used, which is better equipped to handle these more complex models.
The analysis shows that Q-learning achieves similar results to dynamic programming. Afterwards, the new model is applied to a case based on real oil and gas industry data. The results show that this case does not lend itself to an improvement when the drilling strategy is sequentially updated as the decisions compared to the original drilling strategy did not change. This occurs because the prospects all have high positive expected rewards and should be drilled, regardless of the other wells’ outcomes and dependencies.
Another case with synthetic data is explored with more favourable characteristics that, in theory, should lead to the result of the policy from the new model and Qlearning outperforming the current industry policy. However, this was not shown yet within the time constraints of this thesis, as the Q-learner should be run for a more extended period and tuned according to the case.
While the model is created for dynamic sequential decision making in the application of drilling wells, this method can easily be adapted to fit any other application where decisions should be updated sequentially, and the
dependencies can be modelled through a Bayesian network. In conclusion, the key contributions of this thesis are the creation of a new model for the dependencies between geological properties of a hydrocarbon reservoir and the use of the Q-learning reinforcement learning algorithm to compute the sequentially updated drilling strategy as an alternative to dynamic programming.
...
Furthermore, the wells considered in most exploration strategies are geologically dependent. Thus, a realistic model of these dependencies will be beneficial and contribute to a more reliable optimal drilling strategy.
Previous research has shown that modelling similarities between the geological properties of prospect wells in the same region and updating the drilling strategy dynamically after more information is available can add much value. However, the currently developed models are not realistic enough to use in practice. Previous models separated the process into two parts, first modelling the geological success with deterministic rewards and then applying dynamic programming to obtain the sequential drilling strategy.
This thesis proposes a new model with the addition of flexible uncertainty and dependency models between the geological characteristics in the hydrocarbon volume calculation of a reservoir. For this purpose, non-parametric Bayesian networks in combination with copulae are used. This model and the input format of the required data align with how data is gathered in the oil and gas industry. The dependencies are modelled based on simple expert assessments.
After the new model for the geological properties of prospect wells and their dependencies is constructed, a decision policy that hinges on the observations of previously drilled wells is constructed. Previously used dynamic programming becomes infeasible with continuous variables and with a large number of wells. Therefore, the reinforcement learning algorithm Q-learning is used, which is better equipped to handle these more complex models.
The analysis shows that Q-learning achieves similar results to dynamic programming. Afterwards, the new model is applied to a case based on real oil and gas industry data. The results show that this case does not lend itself to an improvement when the drilling strategy is sequentially updated as the decisions compared to the original drilling strategy did not change. This occurs because the prospects all have high positive expected rewards and should be drilled, regardless of the other wells’ outcomes and dependencies.
Another case with synthetic data is explored with more favourable characteristics that, in theory, should lead to the result of the policy from the new model and Qlearning outperforming the current industry policy. However, this was not shown yet within the time constraints of this thesis, as the Q-learner should be run for a more extended period and tuned according to the case.
While the model is created for dynamic sequential decision making in the application of drilling wells, this method can easily be adapted to fit any other application where decisions should be updated sequentially, and the
dependencies can be modelled through a Bayesian network. In conclusion, the key contributions of this thesis are the creation of a new model for the dependencies between geological properties of a hydrocarbon reservoir and the use of the Q-learning reinforcement learning algorithm to compute the sequentially updated drilling strategy as an alternative to dynamic programming.
...
In the process of drilling wells to produce hydrocarbons, an exploration strategy is used to determine which wells should be drilled and in which order. This strategy is vital, as a suboptimal drilling sequence will lead to more expenses and fewer gains.
Furthermore, the wells considered in most exploration strategies are geologically dependent. Thus, a realistic model of these dependencies will be beneficial and contribute to a more reliable optimal drilling strategy.
Previous research has shown that modelling similarities between the geological properties of prospect wells in the same region and updating the drilling strategy dynamically after more information is available can add much value. However, the currently developed models are not realistic enough to use in practice. Previous models separated the process into two parts, first modelling the geological success with deterministic rewards and then applying dynamic programming to obtain the sequential drilling strategy.
This thesis proposes a new model with the addition of flexible uncertainty and dependency models between the geological characteristics in the hydrocarbon volume calculation of a reservoir. For this purpose, non-parametric Bayesian networks in combination with copulae are used. This model and the input format of the required data align with how data is gathered in the oil and gas industry. The dependencies are modelled based on simple expert assessments.
After the new model for the geological properties of prospect wells and their dependencies is constructed, a decision policy that hinges on the observations of previously drilled wells is constructed. Previously used dynamic programming becomes infeasible with continuous variables and with a large number of wells. Therefore, the reinforcement learning algorithm Q-learning is used, which is better equipped to handle these more complex models.
The analysis shows that Q-learning achieves similar results to dynamic programming. Afterwards, the new model is applied to a case based on real oil and gas industry data. The results show that this case does not lend itself to an improvement when the drilling strategy is sequentially updated as the decisions compared to the original drilling strategy did not change. This occurs because the prospects all have high positive expected rewards and should be drilled, regardless of the other wells’ outcomes and dependencies.
Another case with synthetic data is explored with more favourable characteristics that, in theory, should lead to the result of the policy from the new model and Qlearning outperforming the current industry policy. However, this was not shown yet within the time constraints of this thesis, as the Q-learner should be run for a more extended period and tuned according to the case.
While the model is created for dynamic sequential decision making in the application of drilling wells, this method can easily be adapted to fit any other application where decisions should be updated sequentially, and the
dependencies can be modelled through a Bayesian network. In conclusion, the key contributions of this thesis are the creation of a new model for the dependencies between geological properties of a hydrocarbon reservoir and the use of the Q-learning reinforcement learning algorithm to compute the sequentially updated drilling strategy as an alternative to dynamic programming.
Furthermore, the wells considered in most exploration strategies are geologically dependent. Thus, a realistic model of these dependencies will be beneficial and contribute to a more reliable optimal drilling strategy.
Previous research has shown that modelling similarities between the geological properties of prospect wells in the same region and updating the drilling strategy dynamically after more information is available can add much value. However, the currently developed models are not realistic enough to use in practice. Previous models separated the process into two parts, first modelling the geological success with deterministic rewards and then applying dynamic programming to obtain the sequential drilling strategy.
This thesis proposes a new model with the addition of flexible uncertainty and dependency models between the geological characteristics in the hydrocarbon volume calculation of a reservoir. For this purpose, non-parametric Bayesian networks in combination with copulae are used. This model and the input format of the required data align with how data is gathered in the oil and gas industry. The dependencies are modelled based on simple expert assessments.
After the new model for the geological properties of prospect wells and their dependencies is constructed, a decision policy that hinges on the observations of previously drilled wells is constructed. Previously used dynamic programming becomes infeasible with continuous variables and with a large number of wells. Therefore, the reinforcement learning algorithm Q-learning is used, which is better equipped to handle these more complex models.
The analysis shows that Q-learning achieves similar results to dynamic programming. Afterwards, the new model is applied to a case based on real oil and gas industry data. The results show that this case does not lend itself to an improvement when the drilling strategy is sequentially updated as the decisions compared to the original drilling strategy did not change. This occurs because the prospects all have high positive expected rewards and should be drilled, regardless of the other wells’ outcomes and dependencies.
Another case with synthetic data is explored with more favourable characteristics that, in theory, should lead to the result of the policy from the new model and Qlearning outperforming the current industry policy. However, this was not shown yet within the time constraints of this thesis, as the Q-learner should be run for a more extended period and tuned according to the case.
While the model is created for dynamic sequential decision making in the application of drilling wells, this method can easily be adapted to fit any other application where decisions should be updated sequentially, and the
dependencies can be modelled through a Bayesian network. In conclusion, the key contributions of this thesis are the creation of a new model for the dependencies between geological properties of a hydrocarbon reservoir and the use of the Q-learning reinforcement learning algorithm to compute the sequentially updated drilling strategy as an alternative to dynamic programming.
In this thesis, we explore a proposal from Stedin that advocates heating the city of Stad aan `t Haringvliet with hydrogen rather than natural gas. Switching from natural gas to hydrogen presents plenty of unknown factors which must be properly evaluated for an accurate and realistic risk picture, including the danger of running out of hydrogen. The objective of this study is to examine the security of a 100% hydrogen supply in city buildings. For this MSc thesis, the security of supply is defined as a reliable and uninterrupted supply of hydrogen for heating purposes. We explore the security of supply by evaluating the risk to it.
To achieve this goal, we construct a hydrogen supply system using Stedin's criteria. The Classical Model for Structured Expert Judgement and the Bayesian Network are the two mathematical techniques we use to evaluate the risks. The lack of acceptable data makes the task of quantifying the risk to supply much more difficult in this regard. Expert opinion remains the sole trustworthy source of information in these conditions for quantifying uncertainty. However, expert opinion should be validated using objective performance measures. As a result, we adopt the Classical Model for Structured Expert Judgement. The resulting uncertainty distributions of the Structured Expert Judgement research are then integrated into a Bayesian Network that models the uncertainty within the hydrogen supply system. Bayesian Networks are an effective tool for visualizing a domain's probabilistic model, examining all random variable interactions, and inferring probability for scenarios based on available data.\\
The findings of Structured Expert Judgment reveal that there will be a forecast of 114 minutes of not enough hydrogen in the buildings of Stad aan 't Haringvliet in the first year of realizing Stedin's pilot, given the event that there will be a lack of hydrogen in Stad aan `t Haringvliet. Moreover, the security of supply is expected to not be achieved with an estimated probability of 0.06839 for the first year of realizing this pilot. Furthermore, failure rates for each component of the hydrogen supply system have also been quantified using the Classical Model, and one or more mitigations have been proposed for each component. Quantifying the Bayesian Network with the distributions for each component of the hydrogen supply system that resulted from the expert judgment study, yields distinct results than experts' aggregated distributions on the estimated probability and the duration of not meeting the energy demand. The results show that the best-estimated probability for not having enough hydrogen in the buildings equals 0.2067 and when we consider the event of having a lack of hydrogen in the buildings, the best estimated time for how long this will last equals 3.135 minutes in the first year of realizing Stedin's pilot.
...
To achieve this goal, we construct a hydrogen supply system using Stedin's criteria. The Classical Model for Structured Expert Judgement and the Bayesian Network are the two mathematical techniques we use to evaluate the risks. The lack of acceptable data makes the task of quantifying the risk to supply much more difficult in this regard. Expert opinion remains the sole trustworthy source of information in these conditions for quantifying uncertainty. However, expert opinion should be validated using objective performance measures. As a result, we adopt the Classical Model for Structured Expert Judgement. The resulting uncertainty distributions of the Structured Expert Judgement research are then integrated into a Bayesian Network that models the uncertainty within the hydrogen supply system. Bayesian Networks are an effective tool for visualizing a domain's probabilistic model, examining all random variable interactions, and inferring probability for scenarios based on available data.\\
The findings of Structured Expert Judgment reveal that there will be a forecast of 114 minutes of not enough hydrogen in the buildings of Stad aan 't Haringvliet in the first year of realizing Stedin's pilot, given the event that there will be a lack of hydrogen in Stad aan `t Haringvliet. Moreover, the security of supply is expected to not be achieved with an estimated probability of 0.06839 for the first year of realizing this pilot. Furthermore, failure rates for each component of the hydrogen supply system have also been quantified using the Classical Model, and one or more mitigations have been proposed for each component. Quantifying the Bayesian Network with the distributions for each component of the hydrogen supply system that resulted from the expert judgment study, yields distinct results than experts' aggregated distributions on the estimated probability and the duration of not meeting the energy demand. The results show that the best-estimated probability for not having enough hydrogen in the buildings equals 0.2067 and when we consider the event of having a lack of hydrogen in the buildings, the best estimated time for how long this will last equals 3.135 minutes in the first year of realizing Stedin's pilot.
...
In this thesis, we explore a proposal from Stedin that advocates heating the city of Stad aan `t Haringvliet with hydrogen rather than natural gas. Switching from natural gas to hydrogen presents plenty of unknown factors which must be properly evaluated for an accurate and realistic risk picture, including the danger of running out of hydrogen. The objective of this study is to examine the security of a 100% hydrogen supply in city buildings. For this MSc thesis, the security of supply is defined as a reliable and uninterrupted supply of hydrogen for heating purposes. We explore the security of supply by evaluating the risk to it.
To achieve this goal, we construct a hydrogen supply system using Stedin's criteria. The Classical Model for Structured Expert Judgement and the Bayesian Network are the two mathematical techniques we use to evaluate the risks. The lack of acceptable data makes the task of quantifying the risk to supply much more difficult in this regard. Expert opinion remains the sole trustworthy source of information in these conditions for quantifying uncertainty. However, expert opinion should be validated using objective performance measures. As a result, we adopt the Classical Model for Structured Expert Judgement. The resulting uncertainty distributions of the Structured Expert Judgement research are then integrated into a Bayesian Network that models the uncertainty within the hydrogen supply system. Bayesian Networks are an effective tool for visualizing a domain's probabilistic model, examining all random variable interactions, and inferring probability for scenarios based on available data.\\
The findings of Structured Expert Judgment reveal that there will be a forecast of 114 minutes of not enough hydrogen in the buildings of Stad aan 't Haringvliet in the first year of realizing Stedin's pilot, given the event that there will be a lack of hydrogen in Stad aan `t Haringvliet. Moreover, the security of supply is expected to not be achieved with an estimated probability of 0.06839 for the first year of realizing this pilot. Furthermore, failure rates for each component of the hydrogen supply system have also been quantified using the Classical Model, and one or more mitigations have been proposed for each component. Quantifying the Bayesian Network with the distributions for each component of the hydrogen supply system that resulted from the expert judgment study, yields distinct results than experts' aggregated distributions on the estimated probability and the duration of not meeting the energy demand. The results show that the best-estimated probability for not having enough hydrogen in the buildings equals 0.2067 and when we consider the event of having a lack of hydrogen in the buildings, the best estimated time for how long this will last equals 3.135 minutes in the first year of realizing Stedin's pilot.
To achieve this goal, we construct a hydrogen supply system using Stedin's criteria. The Classical Model for Structured Expert Judgement and the Bayesian Network are the two mathematical techniques we use to evaluate the risks. The lack of acceptable data makes the task of quantifying the risk to supply much more difficult in this regard. Expert opinion remains the sole trustworthy source of information in these conditions for quantifying uncertainty. However, expert opinion should be validated using objective performance measures. As a result, we adopt the Classical Model for Structured Expert Judgement. The resulting uncertainty distributions of the Structured Expert Judgement research are then integrated into a Bayesian Network that models the uncertainty within the hydrogen supply system. Bayesian Networks are an effective tool for visualizing a domain's probabilistic model, examining all random variable interactions, and inferring probability for scenarios based on available data.\\
The findings of Structured Expert Judgment reveal that there will be a forecast of 114 minutes of not enough hydrogen in the buildings of Stad aan 't Haringvliet in the first year of realizing Stedin's pilot, given the event that there will be a lack of hydrogen in Stad aan `t Haringvliet. Moreover, the security of supply is expected to not be achieved with an estimated probability of 0.06839 for the first year of realizing this pilot. Furthermore, failure rates for each component of the hydrogen supply system have also been quantified using the Classical Model, and one or more mitigations have been proposed for each component. Quantifying the Bayesian Network with the distributions for each component of the hydrogen supply system that resulted from the expert judgment study, yields distinct results than experts' aggregated distributions on the estimated probability and the duration of not meeting the energy demand. The results show that the best-estimated probability for not having enough hydrogen in the buildings equals 0.2067 and when we consider the event of having a lack of hydrogen in the buildings, the best estimated time for how long this will last equals 3.135 minutes in the first year of realizing Stedin's pilot.
Dutch electricity spot price forecasting
Two study cases using structured expert judgement
Master thesis
(2022)
-
A.D.S. Bachasingh, G.F. Nane, D. Kurowicka, P. Chen, M. Koenen, J. Duivenvoorden
Electricity is a quite unique commodity. Due to the economically non-storable nature of the commodity that electricity is, the constant balance between consumption and production, weather effects, such as temperature, wind speed, solar intensity etc, and the intensity of everyday and business activities, e.g. holi- days, weekends, on- and off-peak hours etc., the price dynamics of this commodity are quite unique as they are not observed in any other market. This extreme price volatility has forced market participants to hedge volumes as well as price risks. Naturally, electricity price forecast models are of great interest to portfolio managers.
Current data-driven models are combined with expertise of traders due to the considerable uncertainty of these models. We aim to subject trader expertise to a transparent methodology using structured expert judgement. During this study, we conduct two elicitations whose variables of interest concern average day- ahead spot prices for 2025, 2030 and 2035. We distinguish between baseload and peakload price. The first elicitation uses assessments regarding current Dutch day-ahead electricity spot prices to forecast forecast the variables of interest. We forecast the average day-ahead electricity spot price for 2025, 2030 and 2035. The second elicitation uses assessments regarding data on past, current and future developments in the Dutch electricity markets to forecast forecast the variables of interest. The second elicitation results in a decision maker with a higher price forecast than the decision maker of the first elicitation. Uncertainty regarding the forecast is higher for 2025 and 2030 during the second study. Uncertainty is equally high regarding the peakload price forecast for 2035 in both studies, however the range shifts. ...
Current data-driven models are combined with expertise of traders due to the considerable uncertainty of these models. We aim to subject trader expertise to a transparent methodology using structured expert judgement. During this study, we conduct two elicitations whose variables of interest concern average day- ahead spot prices for 2025, 2030 and 2035. We distinguish between baseload and peakload price. The first elicitation uses assessments regarding current Dutch day-ahead electricity spot prices to forecast forecast the variables of interest. We forecast the average day-ahead electricity spot price for 2025, 2030 and 2035. The second elicitation uses assessments regarding data on past, current and future developments in the Dutch electricity markets to forecast forecast the variables of interest. The second elicitation results in a decision maker with a higher price forecast than the decision maker of the first elicitation. Uncertainty regarding the forecast is higher for 2025 and 2030 during the second study. Uncertainty is equally high regarding the peakload price forecast for 2035 in both studies, however the range shifts. ...
Electricity is a quite unique commodity. Due to the economically non-storable nature of the commodity that electricity is, the constant balance between consumption and production, weather effects, such as temperature, wind speed, solar intensity etc, and the intensity of everyday and business activities, e.g. holi- days, weekends, on- and off-peak hours etc., the price dynamics of this commodity are quite unique as they are not observed in any other market. This extreme price volatility has forced market participants to hedge volumes as well as price risks. Naturally, electricity price forecast models are of great interest to portfolio managers.
Current data-driven models are combined with expertise of traders due to the considerable uncertainty of these models. We aim to subject trader expertise to a transparent methodology using structured expert judgement. During this study, we conduct two elicitations whose variables of interest concern average day- ahead spot prices for 2025, 2030 and 2035. We distinguish between baseload and peakload price. The first elicitation uses assessments regarding current Dutch day-ahead electricity spot prices to forecast forecast the variables of interest. We forecast the average day-ahead electricity spot price for 2025, 2030 and 2035. The second elicitation uses assessments regarding data on past, current and future developments in the Dutch electricity markets to forecast forecast the variables of interest. The second elicitation results in a decision maker with a higher price forecast than the decision maker of the first elicitation. Uncertainty regarding the forecast is higher for 2025 and 2030 during the second study. Uncertainty is equally high regarding the peakload price forecast for 2035 in both studies, however the range shifts.
Current data-driven models are combined with expertise of traders due to the considerable uncertainty of these models. We aim to subject trader expertise to a transparent methodology using structured expert judgement. During this study, we conduct two elicitations whose variables of interest concern average day- ahead spot prices for 2025, 2030 and 2035. We distinguish between baseload and peakload price. The first elicitation uses assessments regarding current Dutch day-ahead electricity spot prices to forecast forecast the variables of interest. We forecast the average day-ahead electricity spot price for 2025, 2030 and 2035. The second elicitation uses assessments regarding data on past, current and future developments in the Dutch electricity markets to forecast forecast the variables of interest. The second elicitation results in a decision maker with a higher price forecast than the decision maker of the first elicitation. Uncertainty regarding the forecast is higher for 2025 and 2030 during the second study. Uncertainty is equally high regarding the peakload price forecast for 2035 in both studies, however the range shifts.