Bottom up load forecasting, is a technique where energy consumption forecasts are made on lower spatial levels, after which the resulting forecasts are aggregated to form forecasts of higher spatial levels. With the current move to renewable energy sources and the importance of reducing the strain on an already congested electricity grid, accurately forecasting both the location and time of future energy consumption has become more important than ever. To this end, this paper analyses the impact of applying bottom up load forecasting to different spatial levels on the electricity grid, including appliance, household, community and city spatial levels. Energy consumption data for these spatial levels were gathered from the 15-minute Texas and California datasets from Pecan Street Dataport. The results obtained in this study, suggest that energy consumption volatility at lower spacial levels and forecast difficulty at higher spacial levels, play an important role in the performance of applying the bottom up load forecasting technique to energy consumption forecasting problems.

A key issue in Reinforcement Learning (RL) research is the difficulty of defining rewards. Inverse Reinforcement Learning (IRL) is a technique that addresses this challenge by learning the rewards from expert demonstrations. In a realistic setting, expert demonstrations are collected from humans, and it is important to acknowledge that these demonstrations can deviate from rationality due to systematic biases known as cognitive biases. One group of cognitive biases, known as risk-sensitive cognitive biases, pertains to individuals' attitudes and behaviors towards risk and uncertainty. This paper investigates the extent to which IRL can learn from demonstrations that contain risk-sensitive cognitive biases such as loss aversion and risk aversion. Modelling biases using concepts from Prospect Theory and System 1 and 2 model and using Maximum Entropy IRL algorithm, this paper concludes that IRL can recreate similar solutions to experts but inferring the underlying motivations and the interactions between them is an intricate problem that requires novel approaches.

Inverse Reinforcement Learning (IRL) is a subfield of Reinforcement Learning (RL) that focuses on recovering the reward function using expert demonstrations. In the field of IRL, Adversarial IRL (AIRL) is a promising algorithm that is postulated to recover non-linear rewards in environments with unknown dynamics. This study investigates the potential benefits of applying the Curriculum Learning (CL) strategy to the AIRL algorithm. For our experiments, we use a randomized partially observable Markov decision process in the form of a grid-world-like environment. Using only expert demonstrations obtained with an RL algorithm under the true reward function, we train AIRL in a variety of configurations and identify an effective curriculum. Our results show, that a well-constructed curriculum can enhance the performance of AIRL twofold in both key aspects: the speed of convergence and the efficiency of using expert demonstrations. We thus conclude that CL can be a useful addition to an AIRL-based solution. Full code is available online in the supplementary material https://github.com/mikhail-vlasenko/curriculum-learning-IRL.

Reinforcement Learning (RL) has been used to successfully train agents for many tasks, but generalizing to a different task - or even unseen examples of the same task - remains difficult. In this thesis, Deep Reinforcement Learning (DRL) is combined with Graph Neural Networks (GNNs) and domain knowledge, with the aim of improving the generalization capabilities of RL-agents. In classical DRL setups, Convolutional Neural Networks (CNNs) and Multilayer Perceptrons (MLPs) are often applied as neural network architectures for an agent’s policy and/or value network. In this thesis, however, GNNs are used to represent the policy and value network of an agent, which allows for the application of relational inductive biases that are more domain-specific than those of MLPs and CNNs. Observations received by the agent from a simple navigation task - which requires some relational reasoning - are encoded as graphs, consisting of entities and relations between them, which are based on domain knowledge. These graphs are then used as structured input for the GNN-based architecture of the agent. This approach is inspired by human relational reasoning, which is argued to be an important factor in human generalization capabilities. Several GNN-based architectures are proposed and compared, from which two main architectures are distilled: R-GCN-domain and R-GCN-GAN. In the R-GCN-domain architecture, the graph encoding of observations is based on domain knowledge, whereas in R-GCN-GAN we aim to combine the relational encoding of a CNN with additional, learned relations, allowing for an end-to-end solution that does not require domain knowledge. Sample efficiency and both in- and out-of-distribution generalization performance of our architectures are tested on a new grid world environment called ’Key-Corridors’. We find that adding domain-specific relational inductive biases with the R-GCNdomain architecture significantly improves sample efficiency and out-of-distribution generalization, when compared to MLPs and CNNs. However, we did not succeed in learning these domain-specific relational inductive biases with R-GCN-GAN, which does not manage to significantly outperform a CNN. Overall, the results indicate that applying relational reasoning in RL - through the use of GNNs and domain knowledge - can be an important tool for improving sample efficiency and generalization performance.

The field of deep reinforcement learning has seen major successes recently, achieving superhuman performance in discrete games such as Go and the Atari domain, as well as astounding results in continuous robot locomotion tasks. However, the correct specification of human intentions in a reward function is highly challenging, which is why state-of-the-art methods lack interpretability and may lead to unforeseen societal impacts when deployed in the real world. To tackle this, we propose multi-objective reinforced active learning (MORAL), a novel framework based on inverse reinforcement learning for combining a diverse set of human norms into a single Pareto optimal policy. We show that through the combination of active preference learning and multi-objective decision-making, one can interactively train an agent to trade off a variety of learned norms as well as primary reward functions, thus mitigating negative side effects. Furthermore, we introduce two toy environments called Burning Warehouse and Delivery, which allow for studying the scalability of our approach in both size of the state space and reward complexity. We find that through mixing expert demonstrations and preferences, we can achieve superior efficiency compared to employing a single type of expert feedback and, finally, suggest that unlike previous literature, MORAL is able to learn a deep reward model consisting of multiple expert utility functions.

The field of automated negotiation promises to improve negotiations, thus, a fair outcome and process should also be considered when building these systems. However, issues exist with computational approaches to fairness with which the field of computer science is mainly concerned. To this end, we propose a new approach to fairness based on that of essentially contested concepts to see if argumentation-based negotiation could be used as an extension to the Stacked Alternating Offers Protocol to improve fairness. Looking at fairness as an essentially contested concept shows that discussion between people somehow influenced by the negotiation system is necessary to maintain its fairness. This in turn means that systems that provide accessible context are fairer than systems that would not do so. Thus arguments, if implemented in an accessible manner, add more context to the negotiation, in turn making an SAOP negotiation fairer.

Inverse Reinforcement Learning (IRL) is a machine learning technique used for learning rewards from the behavior of an expert agent. With complex agents, such as humans, the maximized reward may not be easily retrievable. This is because humans are prone to cognitive biases. Cognitive biases are a form of deviation from rationality that affects everyday human decision-making. Time inconsistent decision-making is a type of a temporal cognitive bias where planning of future actions may vary at different points of time. Existing research in this field explores using IRL algorithms in numerous real-life situations. However, few works examine the effects of temporal biases on the recovered reward function. Hence in this research, we propose a methodology to generate synthetic demonstrations that emulate human data with this bias. An existing method, Maximum Entropy IRL (MEIRL) algorithm is used to recover reward functions from expert models containing aforementioned biases and compare them to the performance of unbiased models. The demonstrations are in a form of Markov Decision Process (MDP), implemented in a Grid- World environment. Temporal biases will be implemented within the expert demonstrations as different types of agents that portray a specific behavior. Our findings show that all biases affect reward learning to a considerable extent, with that effect having different magnitudes depending on different comparisons.

Responding to the trend of increasing use of artificial intelligence (AI), we need to ensure applications of AI are designed, implemented, utilised and evaluated in a careful manner. Explainable AI, or XAI, meaning; - given a certain audience, the details and reasons of both technical processes of the algorithm-support system and the reasoning behind the system to make its functioning clear or easy to understand - is one of the ways to responsibly design and implement AI systems. This research looks into AI-supported public decision-making processes in the Netherlands and the role and possible contribution of XAI in such a context. To this end, I conducted a mixed-method qualitative study; interviewing sixteen respondents from three key-actor groups within two Dutch national public sector executive bodies, additionally performing three observations and document-analysis. Differentiating between different phases of an AI system’s implementation life-cycle, the study unveils how the respective actors - managers, data scientists and domain experts/(potential) AI users - encounter various challenges in bringing an AI system from idea to production. The empirical findings show that many AI systems, whilst technically developed, are not deployed or adopted by the wider organisation. The study discerns the challenges hindering the AI implementation process from an organisational, human and technical point of view. Moreover, the study highlights the need to approach XAI from a multi-purpose, multi-actor perspective; both acknowledging that various actors need different kinds of explanations, but also bridging different respective professional worldviews to apprehend one another. XAI is often seen as a one-size-fits-all solution for various implementation challenges, however the study shows that certain challenges need to be addressed at least beyond traditional ways of XAI from a computer science perspective, and perhaps beyond XAI all together. As such, the insights of this thesis contribute to generating a more realistic idea about the opportunities and limitations of XAI, within real-world AI implementation processes in the public sector.

In recent years financial fraud has seen substantial growth due to the advent of electronic financial services opening many doors for fraudsters. Consequently, the industry of fraud detection has seen a significant growth in scale, but moves slowly in comparison to the ever-changing nature of fraudulent behavior. As the monetary losses associated with financial fraud continue to grow, so does the need for efficient automated decision making systems. Simple decision making rules are often still the industry standard and only show decent results in the short-term, as reverse-engineering such rules is an easy task for smart fraudsters. Supervised learning systems as automated fraud detectors have shown promising results across the field, but are plagued by challenges uniquely prevalent in the field. Disproportional class imbalance in fraudulent transactions, as well as fraudsters continually adopting new schemes make training robust and generally applicable machine learning models an arduous task. This work introduces a novel machine learning pipeline, which makes use of carefully selected synthetic samples of this minority class to augment the training dataset of the supervised model. Synthetic samples representing fraudulent transactions are filtered based on a novel technique to quantify their expected performance as an adversarial example, using both data-driven and human-expert-driven techniques. By providing the supervised model with high-quality synthetic adversarial examples, we aim to improve its generalizability to never-seen-before fraudulent behavior and, in turn, improve its robustness to the volatile nature of financial fraud. Our results show that weakly-supervised models trained on our augmented datasets are able to detect 7% more fraudulent transactions compared to a baseline model trained on the standard dataset, at the cost of a 1% increase in false positives. Our calculations further show that applying this system could lead to a decrease of 1/6 in monetary losses incurred by financial fraud.

In recent years, ships are expected to improve energy efficiency and reduce carbon emissions. For naval vessels, it is important to be able to maintain their mission profile. It is therefore required to provide real-time advice to the ship’s crew on the optimal speed and propulsion mode settings that include the actual environmental conditions. This paper proposes a novel machine learning approach to establish the ship’s fuel consumption per mile for the actual environmental conditions and develop fuel consumption curves for the various propulsion configurations. The proposed approach uses a multi-layer perceptron (MLP) model to establish the ocean parameters based on the own ship data, with an accuracy of 4.150 %. Using these ocean parameters combined with the own ship data, the fuel per mile is predicted and fuel consumption curves are established using an MLP-model, with an accuracy of 1.551 %. This thesis shows that the proposed approach makes it possible to help a ship’s crew make well informed decisions to reduce their CO2 emissions in real time while still meeting their mission profile. The proposed approach is especially useful for military operators since there is no need for external data sources. Further research is identified to optimize the proposed approach using a dataset containing a higher variety and number of environmental conditions and propulsion modes to improve and validate the fuel consumption curves for the entire spectrum of speeds, environmental conditions and propulsion modes.

Negotiation Support Systems (NSSs) can provide help based on the preference setting (domain, issue weights, issue ranking, strategies, etc.) of the users of the systems. However, sometimes the users of the systems might make mistakes in the preference setting. With wrong preferences, the NSSs might provide suggestions that conflict with the users' desires. This thesis focuses on designing a mechanism that could detect the mismatches in issue weights of the users of NSSs and provide recommendations on updating the issue weights with a scoring method. This mechanism also has the ability to provide different recommendations based on the users' self-confidence level. A case study is done with six simulation experiments to test the mechanism. The results show that during the negotiation, the users of the NSS (PN) can have a chance to change the issue weights when mismatches occurred and the NSS (PN) can provide better suggestions after the issue weights are updated. The limitations of the mechanism are concluded at the end together with the future thoughts

Electrical load forecasting, namely short-term load forecasting, is essential to power grids’ safe and efficient operations. The need for accurate short-term load forecasting becomes increasingly pressing with increased renewable energy sources, which are stochastic in their power supply. Most forecasting models are focused on the temporal information for predictions, ignoring the spatial information of neighbouring houses. However, as neighbouring houses are often under similar circumstances, such as weather and holiday conditions, predictions could benefit from this information. Moreover, aggregating electric loads could further improve the accuracy of predictions, as the loads become less stochastic when aggregated. This paper looks at Graph WaveNet, which can capture the hidden spatial dependencies of different houses without needing geographic information about the houses. The framework is compared against a baseline on different aggregation levels. The results show that the framework can benefit from aggregating the residential electrical loads and improves over the baseline on all aggregation levels.

The ever-evolving power grid is becoming smarter and smarter. Modern houses come with smart meters and energy conscious consumers will buy additional smart meters to place in their home to help monitor their energy consumption. This new smart technology also opens the door to more accurate power consumption forecasting. In this study we look at utilizing a partial hierarchy, in which one of the appliances in a household is modelled separately from the rest of the house, to help improve household energy consumption forecasting accuracy. This is done in conjunction with Auto Regressive Moving Average (ARMA) based models. Three variants of ARMA based models will be looked at: Auto Regressive Integrated Moving Average (ARIMA), Seasonal Auto Regressive Integrated Moving Average (SARIMA), and Auto Regressive Integrated Moving Average with Exogenous variables (ARIMAX). These methods will then be compared to more baseline approaches such as a persistence method and a seasonal moving average. Our analysis has led us to conclude that the partial hierarchy model offers little to no benefit when applied in the field of household energy consumption forecasting when built upon ARMA based models. ARMA based models in general appeared to be poor performers when it came to household energy consumption forecasting.

Non-Intrusive Load Monitoring (NILM) is a technique used to disaggregate household power consumption data into individual appliance components without the need for dedicated meters for each appliance. This paper focuses on improving the generalizability of NILM algorithms to unseen households using Convolutional Neural Networks (CNNs) and one-shot transfer learning. The research investigates the effectiveness of one-shot transfer learning in fine-tuning a CNN model to accurately detect the ON/OFF state of appliances in households not seen during the training phase of the CNN. The study utilizes the Pecan Street dataset for training and evaluation, which includes detailed energy consumption records from various locations in the United States. The results suggest that one-shot transfer learning could enhance the performance of the NILM algorithm, particularly when multiple data samples are used for fine-tuning. However, the effectiveness of one-shot transfer learning varies strongly depending on the number of samples and the characteristics of the target household.

Non-intrusive load monitoring (NILM) is a well-researched concept that aims to provide insights into individual appliance energy usage without the need for dedicated meters. This paper explores the possibility of applying the NILM concept to disaggregate energy data from a community level to a household level. By doing so, it addresses privacy concerns associated with real-time household data collection through smart meters. The proposed approach leverages sequence-to-point learning, a deep learning method, to perform the disaggregation. A method that showed promising results in the domain of NILM. The results show good performance on the evaluation metrics, but they also show that the model does a poor job of capturing the load signature.

In today's society, the rapid progression of digitization has led to the automation of various facets of human existence. This transformation has been facilitated by the utilization of algorithms, which are instrumental in driving efficient and effective automated processes. These algorithms have also found widespread adoption in the public sector, where they are employed to streamline and optimize various tasks and operations. The integration of algorithms in the public sector has brought about significant advancements in areas such as predictive policing, social welfare allocation, and healthcare. However, the use and development of these automated processes were subjected to concerns from the public about privacy, bias, accountability, and transparency. Since these concerns are mainly coming from citizens, their involvement in the process of developing algorithmic systems can potentially be of help. We explore the potential of participatory AI in marginalized communities as a means of obtaining valuable input from citizens regarding the development of these algorithmic systems employed by the public sector. One Piece of our approach involves hosting discussions in local community centers in marginalized neighborhoods. Our focus is on dilemmas that are relevant to algorithm design and evaluation decisions, and we frame these dilemmas in various ways, including forms that may not directly relate to societal impact, but are understandable for laypeople. Our key findings suggest that involving marginalized citizens can bring valuable perspectives and insights that are otherwise ignored. By incorporating public perspectives into algorithm development, we can promote inclusive decision-making processes and ensure that algorithms align with community values.