S. Grammatico
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
38 records found
1
As the share of inverter based resource (IBR)s in the Dutch power system increases, frequency stabilisation becomes more difficult because power system inertia decreases. This requires fast control methods, which are commonly referred to as fast frequency response (FFR) or fast active power injection (FAPI). Classical control methods, such as proportional droop, offer limited constraint handling. Open questions also remain regarding effective collective frequency regulation and stability guarantees. Supplementary model predictive control (MPC) is introduced to address these limitations.
This thesis designs and evaluates MPC-based FFR/FAPI methods for a projected Dutch power system in 2035 using deloaded wind generation. To enforce active power recovery to the deloaded state, an offset-free MPC structure is used. The study begins with a decentralised MPC baseline. Centralised and distributed MPC designs are developed with a focus on effective collective frequency regulation. Finally, a tube-based robust centralised MPC formulation is developed to improve robustness.
Offset-free MPC is shown to be effective for active power recovery when dealing with deloaded generation. Furthermore, centralised and distributed control provides improved collective frequency regulation and power sharing compared to the decentralised approach. The tube-based robust MPC formulation improves robustness under uncertainty. ...
This thesis designs and evaluates MPC-based FFR/FAPI methods for a projected Dutch power system in 2035 using deloaded wind generation. To enforce active power recovery to the deloaded state, an offset-free MPC structure is used. The study begins with a decentralised MPC baseline. Centralised and distributed MPC designs are developed with a focus on effective collective frequency regulation. Finally, a tube-based robust centralised MPC formulation is developed to improve robustness.
Offset-free MPC is shown to be effective for active power recovery when dealing with deloaded generation. Furthermore, centralised and distributed control provides improved collective frequency regulation and power sharing compared to the decentralised approach. The tube-based robust MPC formulation improves robustness under uncertainty. ...
As the share of inverter based resource (IBR)s in the Dutch power system increases, frequency stabilisation becomes more difficult because power system inertia decreases. This requires fast control methods, which are commonly referred to as fast frequency response (FFR) or fast active power injection (FAPI). Classical control methods, such as proportional droop, offer limited constraint handling. Open questions also remain regarding effective collective frequency regulation and stability guarantees. Supplementary model predictive control (MPC) is introduced to address these limitations.
This thesis designs and evaluates MPC-based FFR/FAPI methods for a projected Dutch power system in 2035 using deloaded wind generation. To enforce active power recovery to the deloaded state, an offset-free MPC structure is used. The study begins with a decentralised MPC baseline. Centralised and distributed MPC designs are developed with a focus on effective collective frequency regulation. Finally, a tube-based robust centralised MPC formulation is developed to improve robustness.
Offset-free MPC is shown to be effective for active power recovery when dealing with deloaded generation. Furthermore, centralised and distributed control provides improved collective frequency regulation and power sharing compared to the decentralised approach. The tube-based robust MPC formulation improves robustness under uncertainty.
This thesis designs and evaluates MPC-based FFR/FAPI methods for a projected Dutch power system in 2035 using deloaded wind generation. To enforce active power recovery to the deloaded state, an offset-free MPC structure is used. The study begins with a decentralised MPC baseline. Centralised and distributed MPC designs are developed with a focus on effective collective frequency regulation. Finally, a tube-based robust centralised MPC formulation is developed to improve robustness.
Offset-free MPC is shown to be effective for active power recovery when dealing with deloaded generation. Furthermore, centralised and distributed control provides improved collective frequency regulation and power sharing compared to the decentralised approach. The tube-based robust MPC formulation improves robustness under uncertainty.
This thesis develops high-performance numerical methods for convex optimization, variational inequalities, and game theory, targeting computational bottlenecks in modern large-scale systems. By leveraging the underlying mathematical structure of these problems, this work bridges the gap between abstract operator theory and real-time control and strategic decision-making applications.
The first core contribution focuses on accelerating first-order methods for smooth and nonsmooth convex optimization. We introduce adaptive step-size rules and coupled smoothing–momentum techniques that achieve optimal convergence rates. These methods are designed to exploit problem structure, ensuring computational efficiency and enabling fast convergence without requiring prior knowledge of global problem parameters.
Extending beyond single-agent optimization, the research adopts the framework of variational inequalities to address complex equilibrium problems. We propose projection-free algorithms and specialized splitting methods for settings in which traditional projection operators are computationally expensive. This unified approach enables efficient computation of equilibria in dynamic games and distributionally robust models, where decision-makers must account for both strategic interactions and data uncertainty.
The practical relevance of these developments is demonstrated through real-world applications and the introduction of an open-source computational toolkit. Collectively, these contributions provide a scalable and robust framework for fast, structure-aware decision-making in complex multi-agent systems. ...
The first core contribution focuses on accelerating first-order methods for smooth and nonsmooth convex optimization. We introduce adaptive step-size rules and coupled smoothing–momentum techniques that achieve optimal convergence rates. These methods are designed to exploit problem structure, ensuring computational efficiency and enabling fast convergence without requiring prior knowledge of global problem parameters.
Extending beyond single-agent optimization, the research adopts the framework of variational inequalities to address complex equilibrium problems. We propose projection-free algorithms and specialized splitting methods for settings in which traditional projection operators are computationally expensive. This unified approach enables efficient computation of equilibria in dynamic games and distributionally robust models, where decision-makers must account for both strategic interactions and data uncertainty.
The practical relevance of these developments is demonstrated through real-world applications and the introduction of an open-source computational toolkit. Collectively, these contributions provide a scalable and robust framework for fast, structure-aware decision-making in complex multi-agent systems. ...
This thesis develops high-performance numerical methods for convex optimization, variational inequalities, and game theory, targeting computational bottlenecks in modern large-scale systems. By leveraging the underlying mathematical structure of these problems, this work bridges the gap between abstract operator theory and real-time control and strategic decision-making applications.
The first core contribution focuses on accelerating first-order methods for smooth and nonsmooth convex optimization. We introduce adaptive step-size rules and coupled smoothing–momentum techniques that achieve optimal convergence rates. These methods are designed to exploit problem structure, ensuring computational efficiency and enabling fast convergence without requiring prior knowledge of global problem parameters.
Extending beyond single-agent optimization, the research adopts the framework of variational inequalities to address complex equilibrium problems. We propose projection-free algorithms and specialized splitting methods for settings in which traditional projection operators are computationally expensive. This unified approach enables efficient computation of equilibria in dynamic games and distributionally robust models, where decision-makers must account for both strategic interactions and data uncertainty.
The practical relevance of these developments is demonstrated through real-world applications and the introduction of an open-source computational toolkit. Collectively, these contributions provide a scalable and robust framework for fast, structure-aware decision-making in complex multi-agent systems.
The first core contribution focuses on accelerating first-order methods for smooth and nonsmooth convex optimization. We introduce adaptive step-size rules and coupled smoothing–momentum techniques that achieve optimal convergence rates. These methods are designed to exploit problem structure, ensuring computational efficiency and enabling fast convergence without requiring prior knowledge of global problem parameters.
Extending beyond single-agent optimization, the research adopts the framework of variational inequalities to address complex equilibrium problems. We propose projection-free algorithms and specialized splitting methods for settings in which traditional projection operators are computationally expensive. This unified approach enables efficient computation of equilibria in dynamic games and distributionally robust models, where decision-makers must account for both strategic interactions and data uncertainty.
The practical relevance of these developments is demonstrated through real-world applications and the introduction of an open-source computational toolkit. Collectively, these contributions provide a scalable and robust framework for fast, structure-aware decision-making in complex multi-agent systems.
Robust Energy grid design
Exploring Scenario optimization and opportunities to apply it to grid expansion optimization
The current power grid is heavily congested, as a result of more itermittent power loads and less controllable sources and drains of power. This issue is exacerbated by the planned introduction of large quantities of solar and wind capacity, introducing more uncertainty into the grid operation problem. Current expansion plans are not able to cope with the expected loads, nor can they keep the delivery-guarantees currently kept while also decarbonizing the grid. This is a problem for the green transition, and more efficient grid expansion can help mitigating this problem.
The proposed solution leverages extra information found using the scenario approach to grid operation, instead of the Monte-Carlo simulations currently used in grid expansion studies. Doing so exploits the extra information the scenario approach yields with regard to reliability, and operational performance. By optimizing robustly against all samples we find more useful improvements that actually push the boundary of operation, where the Monte-Carlo approach might be less likely to find those same improvements given that is overly focuses on samples of little consequence.
To test this hypothesis, a simple graph representation of a power grid is constructed, and a grid expansion program either using the scenario approach or the Monte-Carlo approach is applied to this graph, resulting in two sets of candidate modifications that either program found to be most promising. To validate the results, operational performance using both the scenario approach and the Monte-Carlo approach are computed, the empirical violation probabilty is computed and computation time is compared. ...
The proposed solution leverages extra information found using the scenario approach to grid operation, instead of the Monte-Carlo simulations currently used in grid expansion studies. Doing so exploits the extra information the scenario approach yields with regard to reliability, and operational performance. By optimizing robustly against all samples we find more useful improvements that actually push the boundary of operation, where the Monte-Carlo approach might be less likely to find those same improvements given that is overly focuses on samples of little consequence.
To test this hypothesis, a simple graph representation of a power grid is constructed, and a grid expansion program either using the scenario approach or the Monte-Carlo approach is applied to this graph, resulting in two sets of candidate modifications that either program found to be most promising. To validate the results, operational performance using both the scenario approach and the Monte-Carlo approach are computed, the empirical violation probabilty is computed and computation time is compared. ...
The current power grid is heavily congested, as a result of more itermittent power loads and less controllable sources and drains of power. This issue is exacerbated by the planned introduction of large quantities of solar and wind capacity, introducing more uncertainty into the grid operation problem. Current expansion plans are not able to cope with the expected loads, nor can they keep the delivery-guarantees currently kept while also decarbonizing the grid. This is a problem for the green transition, and more efficient grid expansion can help mitigating this problem.
The proposed solution leverages extra information found using the scenario approach to grid operation, instead of the Monte-Carlo simulations currently used in grid expansion studies. Doing so exploits the extra information the scenario approach yields with regard to reliability, and operational performance. By optimizing robustly against all samples we find more useful improvements that actually push the boundary of operation, where the Monte-Carlo approach might be less likely to find those same improvements given that is overly focuses on samples of little consequence.
To test this hypothesis, a simple graph representation of a power grid is constructed, and a grid expansion program either using the scenario approach or the Monte-Carlo approach is applied to this graph, resulting in two sets of candidate modifications that either program found to be most promising. To validate the results, operational performance using both the scenario approach and the Monte-Carlo approach are computed, the empirical violation probabilty is computed and computation time is compared.
The proposed solution leverages extra information found using the scenario approach to grid operation, instead of the Monte-Carlo simulations currently used in grid expansion studies. Doing so exploits the extra information the scenario approach yields with regard to reliability, and operational performance. By optimizing robustly against all samples we find more useful improvements that actually push the boundary of operation, where the Monte-Carlo approach might be less likely to find those same improvements given that is overly focuses on samples of little consequence.
To test this hypothesis, a simple graph representation of a power grid is constructed, and a grid expansion program either using the scenario approach or the Monte-Carlo approach is applied to this graph, resulting in two sets of candidate modifications that either program found to be most promising. To validate the results, operational performance using both the scenario approach and the Monte-Carlo approach are computed, the empirical violation probabilty is computed and computation time is compared.
Clustering in Nucleolus Estimations for P2P Energy Exchange
An Analysis of Clustering Methods for Nucleolus Estimation
With the increasing electrification of domestic energy usage and the inherent demands on distribution networks, there are growing incentives for consumers to produce and consume energy locally. Cooperative strategies between consumers can alleviate upstream demand through scheduling the charging and discharging of battery systems on the basis of predicted knowledge of consumers' solar panel generation, consumption habits, and electricity prices. These consumers are called prosumers for their ability to consume and produce energy. Smart scheduling keeps generated energy stored for local consumption, and can help time prosumers' consumption of energy from the distribution networks when energy is cheap. One issue lies in the fair allocation of cost savings amongst prosumers. A well-suited solution concept, the nucleolus, becomes problematic to solve for larger numbers of prosumers. Building upon research done on estimating the nucleolus, this thesis investigates how different methods of clustering prosumers perform in nucleolus estimation.
...
With the increasing electrification of domestic energy usage and the inherent demands on distribution networks, there are growing incentives for consumers to produce and consume energy locally. Cooperative strategies between consumers can alleviate upstream demand through scheduling the charging and discharging of battery systems on the basis of predicted knowledge of consumers' solar panel generation, consumption habits, and electricity prices. These consumers are called prosumers for their ability to consume and produce energy. Smart scheduling keeps generated energy stored for local consumption, and can help time prosumers' consumption of energy from the distribution networks when energy is cheap. One issue lies in the fair allocation of cost savings amongst prosumers. A well-suited solution concept, the nucleolus, becomes problematic to solve for larger numbers of prosumers. Building upon research done on estimating the nucleolus, this thesis investigates how different methods of clustering prosumers perform in nucleolus estimation.
Deep Reinforcement Learning for Battery Arbitrage in the Continuous Intraday Market
Scaling to Minute-Level Trading with TD3+BC under Realistic Market Constraints
This thesis investigates the minute-level operation of a 40 MWh battery in the Continuous Intraday Market. The study focuses on the Dutch market within its European cross-border setting and formulates dispatch as a finite-horizon Markov Decision Process that captures both battery physics and order-book depth. A continuous action-to-power mapping guarantees feasibility by enforcing state-ofcharge, efficiency, and market liquidity constraints. Two deep reinforcement learning methods—TD3 and TD3 with behaviour cloning (TD3+BC)—are implemented and benchmarked against a rollingintrinsic (RI) optimiser, which serves as both baseline and source of expert trajectories. Minute-level resolution proves empirically justified: the RI benchmark at one-minute granularity consistently outperforms its fifteen-minute counterpart. In comparative experiments, TD3+BC outperforms plain TD3 and narrows the gap to RI, reaching within about 4% of its profit on average, though not consistently surpassing it. The learned policy exhibits a distinct cycle-efficient trading style with fewer equivalent full cycles and lower throughput than RI but higher value extracted per cycle, which translates over the project lifetime into a stronger business case, yielding an internal rate of return approximately twice that of the RI baseline. Training TD3+BC for five million steps is computationally tractable on a standard laptop (≈11 hours), and inference runs in milliseconds per step, confirming real-time deployability. The overall framework thus demonstrates that deep reinforcement learning can scale to realistic battery sizes and minutelevel trading, yielding stable and interpretable policies. At the same time, the persistent strength of the RI optimiser highlights the value of structural priors, suggesting that hybrid approaches combining reinforcement learning with optimisation principles may offer the most promising path toward robust, market-ready battery trading systems.
...
This thesis investigates the minute-level operation of a 40 MWh battery in the Continuous Intraday Market. The study focuses on the Dutch market within its European cross-border setting and formulates dispatch as a finite-horizon Markov Decision Process that captures both battery physics and order-book depth. A continuous action-to-power mapping guarantees feasibility by enforcing state-ofcharge, efficiency, and market liquidity constraints. Two deep reinforcement learning methods—TD3 and TD3 with behaviour cloning (TD3+BC)—are implemented and benchmarked against a rollingintrinsic (RI) optimiser, which serves as both baseline and source of expert trajectories. Minute-level resolution proves empirically justified: the RI benchmark at one-minute granularity consistently outperforms its fifteen-minute counterpart. In comparative experiments, TD3+BC outperforms plain TD3 and narrows the gap to RI, reaching within about 4% of its profit on average, though not consistently surpassing it. The learned policy exhibits a distinct cycle-efficient trading style with fewer equivalent full cycles and lower throughput than RI but higher value extracted per cycle, which translates over the project lifetime into a stronger business case, yielding an internal rate of return approximately twice that of the RI baseline. Training TD3+BC for five million steps is computationally tractable on a standard laptop (≈11 hours), and inference runs in milliseconds per step, confirming real-time deployability. The overall framework thus demonstrates that deep reinforcement learning can scale to realistic battery sizes and minutelevel trading, yielding stable and interpretable policies. At the same time, the persistent strength of the RI optimiser highlights the value of structural priors, suggesting that hybrid approaches combining reinforcement learning with optimisation principles may offer the most promising path toward robust, market-ready battery trading systems.
The rapid integration of renewable energy sources, such as wind and solar power, into modern electricity systems has introduced challenges in balancing supply and demand, managing grid congestion, and ensuring efficient energy market participation. This thesis develops a framework for optimizing the portfolio management of hybrid power plants (HPPs) under uncertainty. HPPs combine renewable generation, energy storage (batteries), and flexible demand (electrolyzers) to improve grid efficiency and market participation.
This thesis develops a framework for optimizing HPP portfolio management under uncertainty, specifically targeting participation in the day-ahead market, the Dutch aFRR market, and the strategic use of passive imbalance. Two complementary optimization methods were developed and compared. Stochastic programming (SP) manages uncertainties in wind generation, activations, and day-ahead market prices using scenario-based approaches. Adaptive robust optimization (ARO) employs worst-case scenarios defined by budgeted polyhedral uncertainty sets for wind generation and activations, while using scenarios for day-ahead market price uncertainty. The ARO framework leverages duality theory and a column-and-constraint generation algorithm (CCGA) to iteratively refine robust solutions. Both methods were implemented within a shrinking horizon framework, which allowed for iterative decision-making at each time-step and tested the robustness of the first-stage decisions.
A case study using real-world Dutch market data demonstrated that both methods achieved zero violations by leveraging passive imbalance. ARO consistently delivered more robust first-stage decisions and higher revenue by effectively utilizing activation capacity. Additionally, a parameter study revealed the trade-offs between robustness and revenue, offering valuable insights into the flexibility and effectiveness of each method under varying conditions. This work highlights the potential of hybrid power plants to improve operational reliability and profitability in modern electricity systems ...
This thesis develops a framework for optimizing HPP portfolio management under uncertainty, specifically targeting participation in the day-ahead market, the Dutch aFRR market, and the strategic use of passive imbalance. Two complementary optimization methods were developed and compared. Stochastic programming (SP) manages uncertainties in wind generation, activations, and day-ahead market prices using scenario-based approaches. Adaptive robust optimization (ARO) employs worst-case scenarios defined by budgeted polyhedral uncertainty sets for wind generation and activations, while using scenarios for day-ahead market price uncertainty. The ARO framework leverages duality theory and a column-and-constraint generation algorithm (CCGA) to iteratively refine robust solutions. Both methods were implemented within a shrinking horizon framework, which allowed for iterative decision-making at each time-step and tested the robustness of the first-stage decisions.
A case study using real-world Dutch market data demonstrated that both methods achieved zero violations by leveraging passive imbalance. ARO consistently delivered more robust first-stage decisions and higher revenue by effectively utilizing activation capacity. Additionally, a parameter study revealed the trade-offs between robustness and revenue, offering valuable insights into the flexibility and effectiveness of each method under varying conditions. This work highlights the potential of hybrid power plants to improve operational reliability and profitability in modern electricity systems ...
The rapid integration of renewable energy sources, such as wind and solar power, into modern electricity systems has introduced challenges in balancing supply and demand, managing grid congestion, and ensuring efficient energy market participation. This thesis develops a framework for optimizing the portfolio management of hybrid power plants (HPPs) under uncertainty. HPPs combine renewable generation, energy storage (batteries), and flexible demand (electrolyzers) to improve grid efficiency and market participation.
This thesis develops a framework for optimizing HPP portfolio management under uncertainty, specifically targeting participation in the day-ahead market, the Dutch aFRR market, and the strategic use of passive imbalance. Two complementary optimization methods were developed and compared. Stochastic programming (SP) manages uncertainties in wind generation, activations, and day-ahead market prices using scenario-based approaches. Adaptive robust optimization (ARO) employs worst-case scenarios defined by budgeted polyhedral uncertainty sets for wind generation and activations, while using scenarios for day-ahead market price uncertainty. The ARO framework leverages duality theory and a column-and-constraint generation algorithm (CCGA) to iteratively refine robust solutions. Both methods were implemented within a shrinking horizon framework, which allowed for iterative decision-making at each time-step and tested the robustness of the first-stage decisions.
A case study using real-world Dutch market data demonstrated that both methods achieved zero violations by leveraging passive imbalance. ARO consistently delivered more robust first-stage decisions and higher revenue by effectively utilizing activation capacity. Additionally, a parameter study revealed the trade-offs between robustness and revenue, offering valuable insights into the flexibility and effectiveness of each method under varying conditions. This work highlights the potential of hybrid power plants to improve operational reliability and profitability in modern electricity systems
This thesis develops a framework for optimizing HPP portfolio management under uncertainty, specifically targeting participation in the day-ahead market, the Dutch aFRR market, and the strategic use of passive imbalance. Two complementary optimization methods were developed and compared. Stochastic programming (SP) manages uncertainties in wind generation, activations, and day-ahead market prices using scenario-based approaches. Adaptive robust optimization (ARO) employs worst-case scenarios defined by budgeted polyhedral uncertainty sets for wind generation and activations, while using scenarios for day-ahead market price uncertainty. The ARO framework leverages duality theory and a column-and-constraint generation algorithm (CCGA) to iteratively refine robust solutions. Both methods were implemented within a shrinking horizon framework, which allowed for iterative decision-making at each time-step and tested the robustness of the first-stage decisions.
A case study using real-world Dutch market data demonstrated that both methods achieved zero violations by leveraging passive imbalance. ARO consistently delivered more robust first-stage decisions and higher revenue by effectively utilizing activation capacity. Additionally, a parameter study revealed the trade-offs between robustness and revenue, offering valuable insights into the flexibility and effectiveness of each method under varying conditions. This work highlights the potential of hybrid power plants to improve operational reliability and profitability in modern electricity systems
Recent engineering developments have surrounded us with intelligent devices, which are required to autonomously take rational decisions while interacting with the physical world. These systems are increasingly widespread, interacting and interconnected, thus resulting in decision problems that involve multiple rational agents, generally with conflicting objectives and interrelated operating constraints. Currently relevant examples include autonomous driving, traffic routing, clearance of autonomous bidding markets, power consumption and production scheduling on the electricity grid, control of robotic swarms, and autonomous racing. The mathematical framework for formulating these problems is known as a game. Over the past decade, there has been significant progress in the development of algorithms that determine an action which is simultaneously rational (i.e. optimal) for each agent, namely, a generalized Nash equilibrium (GNE). This solution is particularly favorable as it is self-enforcing, in the sense that no decision maker can improve its payoff by unilaterally deviating from it. However, currently available algorithms for the computation of a GNE present numerous shortcomings, which limit their applicability to real-world non-cooperative decision processes and that we address in this thesis.
First of all, GNE problems typically admit multiple solutions, but currently available algorithms only compute an arbitrary, initialization-dependent GNE. In applications where a predictable and well-defined solution is necessary, it becomes important to select a specific GNE (among potentially infinitely many) that optimizes an arbitrary metric or a secondary, cooperative objective. We develop the first optimal GNE selection algorithms. We compare two different algorithm design methods, both developed under the framework of operator theory: the first, i.e. the hybrid steepest descent method (HSDM), entails a gradient descent of the selection function with vanishing step size combined with a GNE seeking algorithm, while the second requires the solution of a sequence of Variational Inequalities (VIs) with a vanishing regularizing term. Both design methods lead to algorithms that are suitable to distributing the computation between a central node and the agents, and we include ad-hoc algorithms for the particular cases of aggregative and cocoercive games.
Secondly, we consider time-varying games, motivated by the need for algorithms that continuously monitor and control physical multiagent systems. In such games, the agents must track an evolving solution with limited computation time between the problem's updates. This scenario is particularly relevant when the agents are affected by disturbances whose time-scale is comparable to the algorithm convergence rate. The challenge lies in finding algorithms that exhibit fast convergence and a robustness property to external disturbances, both typically associated with a linear convergence rate. We derive and study the asymptotic tracking error of a fully-distributed algorithm (i.e., without a central coordinator) for GNE problems with linear equality constraints. For a time-varying GNE selection problem, we find the HSDM with constant stepsize to be linearly convergent to an approximate solution. We find that the approximation error can be controller by an appropriate choice of the stepsize and number of iterations per time step, and we derive a bound to the asymptotic tracking error.
Finally, again driven by the need of applying GNE seeking algorithms to the control of physical systems, we consider dynamic games, where the decision each agent has to take is a time sequence of inputs to a dynamical system. In this case, the coupling between the agents emerges not only through the objectives and constraints, but also through the system dynamics. Ideally, one should compute the GNE solution by predicting the dynamics over an indefinitely long horizon. This is typically computationally intractable, especially when constraints are present. We then approximate the infinite-horizon control sequence by recomputing at each time instant the solutions to a finite-time equilibrium problem, a method typically known as receding-horizon control (or model predictive control, in the single-agent case). We derive a novel characterization of the infinite horizon objective achieved by the Nash equilibrium trajectory, and we show that one can recover the infinite-horizon performance by including this expression in the agents' objectives as an additive terminal cost. With this result, we conclude asymptotic stability of the steady state under a receding-horizon game-theoretic control action. Compared to the literature, we do not assume stability of the uncontrolled plant, nor we introduce auxiliary constraints. Furthermore, we find that the asymptotic stability of the steady state can be obtained with a more generic terminal cost if the game is potential, as we demonstrate on a practical traffic routing application. ...
First of all, GNE problems typically admit multiple solutions, but currently available algorithms only compute an arbitrary, initialization-dependent GNE. In applications where a predictable and well-defined solution is necessary, it becomes important to select a specific GNE (among potentially infinitely many) that optimizes an arbitrary metric or a secondary, cooperative objective. We develop the first optimal GNE selection algorithms. We compare two different algorithm design methods, both developed under the framework of operator theory: the first, i.e. the hybrid steepest descent method (HSDM), entails a gradient descent of the selection function with vanishing step size combined with a GNE seeking algorithm, while the second requires the solution of a sequence of Variational Inequalities (VIs) with a vanishing regularizing term. Both design methods lead to algorithms that are suitable to distributing the computation between a central node and the agents, and we include ad-hoc algorithms for the particular cases of aggregative and cocoercive games.
Secondly, we consider time-varying games, motivated by the need for algorithms that continuously monitor and control physical multiagent systems. In such games, the agents must track an evolving solution with limited computation time between the problem's updates. This scenario is particularly relevant when the agents are affected by disturbances whose time-scale is comparable to the algorithm convergence rate. The challenge lies in finding algorithms that exhibit fast convergence and a robustness property to external disturbances, both typically associated with a linear convergence rate. We derive and study the asymptotic tracking error of a fully-distributed algorithm (i.e., without a central coordinator) for GNE problems with linear equality constraints. For a time-varying GNE selection problem, we find the HSDM with constant stepsize to be linearly convergent to an approximate solution. We find that the approximation error can be controller by an appropriate choice of the stepsize and number of iterations per time step, and we derive a bound to the asymptotic tracking error.
Finally, again driven by the need of applying GNE seeking algorithms to the control of physical systems, we consider dynamic games, where the decision each agent has to take is a time sequence of inputs to a dynamical system. In this case, the coupling between the agents emerges not only through the objectives and constraints, but also through the system dynamics. Ideally, one should compute the GNE solution by predicting the dynamics over an indefinitely long horizon. This is typically computationally intractable, especially when constraints are present. We then approximate the infinite-horizon control sequence by recomputing at each time instant the solutions to a finite-time equilibrium problem, a method typically known as receding-horizon control (or model predictive control, in the single-agent case). We derive a novel characterization of the infinite horizon objective achieved by the Nash equilibrium trajectory, and we show that one can recover the infinite-horizon performance by including this expression in the agents' objectives as an additive terminal cost. With this result, we conclude asymptotic stability of the steady state under a receding-horizon game-theoretic control action. Compared to the literature, we do not assume stability of the uncontrolled plant, nor we introduce auxiliary constraints. Furthermore, we find that the asymptotic stability of the steady state can be obtained with a more generic terminal cost if the game is potential, as we demonstrate on a practical traffic routing application. ...
Recent engineering developments have surrounded us with intelligent devices, which are required to autonomously take rational decisions while interacting with the physical world. These systems are increasingly widespread, interacting and interconnected, thus resulting in decision problems that involve multiple rational agents, generally with conflicting objectives and interrelated operating constraints. Currently relevant examples include autonomous driving, traffic routing, clearance of autonomous bidding markets, power consumption and production scheduling on the electricity grid, control of robotic swarms, and autonomous racing. The mathematical framework for formulating these problems is known as a game. Over the past decade, there has been significant progress in the development of algorithms that determine an action which is simultaneously rational (i.e. optimal) for each agent, namely, a generalized Nash equilibrium (GNE). This solution is particularly favorable as it is self-enforcing, in the sense that no decision maker can improve its payoff by unilaterally deviating from it. However, currently available algorithms for the computation of a GNE present numerous shortcomings, which limit their applicability to real-world non-cooperative decision processes and that we address in this thesis.
First of all, GNE problems typically admit multiple solutions, but currently available algorithms only compute an arbitrary, initialization-dependent GNE. In applications where a predictable and well-defined solution is necessary, it becomes important to select a specific GNE (among potentially infinitely many) that optimizes an arbitrary metric or a secondary, cooperative objective. We develop the first optimal GNE selection algorithms. We compare two different algorithm design methods, both developed under the framework of operator theory: the first, i.e. the hybrid steepest descent method (HSDM), entails a gradient descent of the selection function with vanishing step size combined with a GNE seeking algorithm, while the second requires the solution of a sequence of Variational Inequalities (VIs) with a vanishing regularizing term. Both design methods lead to algorithms that are suitable to distributing the computation between a central node and the agents, and we include ad-hoc algorithms for the particular cases of aggregative and cocoercive games.
Secondly, we consider time-varying games, motivated by the need for algorithms that continuously monitor and control physical multiagent systems. In such games, the agents must track an evolving solution with limited computation time between the problem's updates. This scenario is particularly relevant when the agents are affected by disturbances whose time-scale is comparable to the algorithm convergence rate. The challenge lies in finding algorithms that exhibit fast convergence and a robustness property to external disturbances, both typically associated with a linear convergence rate. We derive and study the asymptotic tracking error of a fully-distributed algorithm (i.e., without a central coordinator) for GNE problems with linear equality constraints. For a time-varying GNE selection problem, we find the HSDM with constant stepsize to be linearly convergent to an approximate solution. We find that the approximation error can be controller by an appropriate choice of the stepsize and number of iterations per time step, and we derive a bound to the asymptotic tracking error.
Finally, again driven by the need of applying GNE seeking algorithms to the control of physical systems, we consider dynamic games, where the decision each agent has to take is a time sequence of inputs to a dynamical system. In this case, the coupling between the agents emerges not only through the objectives and constraints, but also through the system dynamics. Ideally, one should compute the GNE solution by predicting the dynamics over an indefinitely long horizon. This is typically computationally intractable, especially when constraints are present. We then approximate the infinite-horizon control sequence by recomputing at each time instant the solutions to a finite-time equilibrium problem, a method typically known as receding-horizon control (or model predictive control, in the single-agent case). We derive a novel characterization of the infinite horizon objective achieved by the Nash equilibrium trajectory, and we show that one can recover the infinite-horizon performance by including this expression in the agents' objectives as an additive terminal cost. With this result, we conclude asymptotic stability of the steady state under a receding-horizon game-theoretic control action. Compared to the literature, we do not assume stability of the uncontrolled plant, nor we introduce auxiliary constraints. Furthermore, we find that the asymptotic stability of the steady state can be obtained with a more generic terminal cost if the game is potential, as we demonstrate on a practical traffic routing application.
First of all, GNE problems typically admit multiple solutions, but currently available algorithms only compute an arbitrary, initialization-dependent GNE. In applications where a predictable and well-defined solution is necessary, it becomes important to select a specific GNE (among potentially infinitely many) that optimizes an arbitrary metric or a secondary, cooperative objective. We develop the first optimal GNE selection algorithms. We compare two different algorithm design methods, both developed under the framework of operator theory: the first, i.e. the hybrid steepest descent method (HSDM), entails a gradient descent of the selection function with vanishing step size combined with a GNE seeking algorithm, while the second requires the solution of a sequence of Variational Inequalities (VIs) with a vanishing regularizing term. Both design methods lead to algorithms that are suitable to distributing the computation between a central node and the agents, and we include ad-hoc algorithms for the particular cases of aggregative and cocoercive games.
Secondly, we consider time-varying games, motivated by the need for algorithms that continuously monitor and control physical multiagent systems. In such games, the agents must track an evolving solution with limited computation time between the problem's updates. This scenario is particularly relevant when the agents are affected by disturbances whose time-scale is comparable to the algorithm convergence rate. The challenge lies in finding algorithms that exhibit fast convergence and a robustness property to external disturbances, both typically associated with a linear convergence rate. We derive and study the asymptotic tracking error of a fully-distributed algorithm (i.e., without a central coordinator) for GNE problems with linear equality constraints. For a time-varying GNE selection problem, we find the HSDM with constant stepsize to be linearly convergent to an approximate solution. We find that the approximation error can be controller by an appropriate choice of the stepsize and number of iterations per time step, and we derive a bound to the asymptotic tracking error.
Finally, again driven by the need of applying GNE seeking algorithms to the control of physical systems, we consider dynamic games, where the decision each agent has to take is a time sequence of inputs to a dynamical system. In this case, the coupling between the agents emerges not only through the objectives and constraints, but also through the system dynamics. Ideally, one should compute the GNE solution by predicting the dynamics over an indefinitely long horizon. This is typically computationally intractable, especially when constraints are present. We then approximate the infinite-horizon control sequence by recomputing at each time instant the solutions to a finite-time equilibrium problem, a method typically known as receding-horizon control (or model predictive control, in the single-agent case). We derive a novel characterization of the infinite horizon objective achieved by the Nash equilibrium trajectory, and we show that one can recover the infinite-horizon performance by including this expression in the agents' objectives as an additive terminal cost. With this result, we conclude asymptotic stability of the steady state under a receding-horizon game-theoretic control action. Compared to the literature, we do not assume stability of the uncontrolled plant, nor we introduce auxiliary constraints. Furthermore, we find that the asymptotic stability of the steady state can be obtained with a more generic terminal cost if the game is potential, as we demonstrate on a practical traffic routing application.
Energy systems have been continuously evolving with the advancement in technology. The expected result would be a smooth transition towards clean and more sustainable energy systems which work closely with one another. The focus on using the available energy resources optimally and effectively is a need for energy providers as well as the energy consumers. However, there are different challenges in making these optimal decisions because most of the energy providers have their independent networks. The progress in energy sector has promoted the use of integrated energy systems from different geographical locations. Integrated systems include several large subsystems called areas consisting of electricity and natural gas networks. These large systems are connected to one another through connections known as tie-lines and/or tie-pipes and the energy dispatch can be controlled by the area operator. The main intention of an integrated system is that the electricity and natural gas networks are closely linked as opposed to the formerly isolated systems.
The goal of an optimal energy flow problem for an IEGS network consisting of an objective function is to minimize the system's overall operational cost while satisfying the constraints for the electricity network, natural gas network and the coupling constraints. This study attempts to provide insights into some of the available methods of relaxation/approximation for linearizing the nonlinear, non-convex natural gas flow constraints, their performance, efficiency, and reliability of these methods for multi-area networks. In this overview, various formulations and solution methods of the optimization problem have been examined. Their performance has been compared based on certain performance metrics to find the best possible method to solve such complex optimization problems. Since the integrated systems usually are extremely large, it is vital to separate them into smaller subsystems to solve the optimization problem efficiently. There are different methods used for solving these decentralized optimization problems and it is important to study these algorithms for multi-area IEGS.
The investigation begins with an introduction of the research problem to find the best possible optimization method for IEGS. This study emphasizes the significance of relaxation techniques used to relax the non-convexity and studying optimization methods for integrated electricity natural gas systems. The mathematical formulations and optimization strategies utilized to solve the models are explained in detail. The performance of optimization algorithms is assessed based on certain performance metrics. Finally, this thesis concludes by summarizing the primary findings for the best possible optimization method and suggesting future research possibilities in this topic. To determine which approach best fits the OEF problem for IEGS, a thorough numerical comparison of various relaxation techniques, centralized and decentralized scheme of operation must be made. It is imperative to note that these techniques solve the approximated problem, not the original nonlinear, non-convex problem, which would leave some room for errors and hence an area of future improvement.
...
The goal of an optimal energy flow problem for an IEGS network consisting of an objective function is to minimize the system's overall operational cost while satisfying the constraints for the electricity network, natural gas network and the coupling constraints. This study attempts to provide insights into some of the available methods of relaxation/approximation for linearizing the nonlinear, non-convex natural gas flow constraints, their performance, efficiency, and reliability of these methods for multi-area networks. In this overview, various formulations and solution methods of the optimization problem have been examined. Their performance has been compared based on certain performance metrics to find the best possible method to solve such complex optimization problems. Since the integrated systems usually are extremely large, it is vital to separate them into smaller subsystems to solve the optimization problem efficiently. There are different methods used for solving these decentralized optimization problems and it is important to study these algorithms for multi-area IEGS.
The investigation begins with an introduction of the research problem to find the best possible optimization method for IEGS. This study emphasizes the significance of relaxation techniques used to relax the non-convexity and studying optimization methods for integrated electricity natural gas systems. The mathematical formulations and optimization strategies utilized to solve the models are explained in detail. The performance of optimization algorithms is assessed based on certain performance metrics. Finally, this thesis concludes by summarizing the primary findings for the best possible optimization method and suggesting future research possibilities in this topic. To determine which approach best fits the OEF problem for IEGS, a thorough numerical comparison of various relaxation techniques, centralized and decentralized scheme of operation must be made. It is imperative to note that these techniques solve the approximated problem, not the original nonlinear, non-convex problem, which would leave some room for errors and hence an area of future improvement.
...
Energy systems have been continuously evolving with the advancement in technology. The expected result would be a smooth transition towards clean and more sustainable energy systems which work closely with one another. The focus on using the available energy resources optimally and effectively is a need for energy providers as well as the energy consumers. However, there are different challenges in making these optimal decisions because most of the energy providers have their independent networks. The progress in energy sector has promoted the use of integrated energy systems from different geographical locations. Integrated systems include several large subsystems called areas consisting of electricity and natural gas networks. These large systems are connected to one another through connections known as tie-lines and/or tie-pipes and the energy dispatch can be controlled by the area operator. The main intention of an integrated system is that the electricity and natural gas networks are closely linked as opposed to the formerly isolated systems.
The goal of an optimal energy flow problem for an IEGS network consisting of an objective function is to minimize the system's overall operational cost while satisfying the constraints for the electricity network, natural gas network and the coupling constraints. This study attempts to provide insights into some of the available methods of relaxation/approximation for linearizing the nonlinear, non-convex natural gas flow constraints, their performance, efficiency, and reliability of these methods for multi-area networks. In this overview, various formulations and solution methods of the optimization problem have been examined. Their performance has been compared based on certain performance metrics to find the best possible method to solve such complex optimization problems. Since the integrated systems usually are extremely large, it is vital to separate them into smaller subsystems to solve the optimization problem efficiently. There are different methods used for solving these decentralized optimization problems and it is important to study these algorithms for multi-area IEGS.
The investigation begins with an introduction of the research problem to find the best possible optimization method for IEGS. This study emphasizes the significance of relaxation techniques used to relax the non-convexity and studying optimization methods for integrated electricity natural gas systems. The mathematical formulations and optimization strategies utilized to solve the models are explained in detail. The performance of optimization algorithms is assessed based on certain performance metrics. Finally, this thesis concludes by summarizing the primary findings for the best possible optimization method and suggesting future research possibilities in this topic. To determine which approach best fits the OEF problem for IEGS, a thorough numerical comparison of various relaxation techniques, centralized and decentralized scheme of operation must be made. It is imperative to note that these techniques solve the approximated problem, not the original nonlinear, non-convex problem, which would leave some room for errors and hence an area of future improvement.
The goal of an optimal energy flow problem for an IEGS network consisting of an objective function is to minimize the system's overall operational cost while satisfying the constraints for the electricity network, natural gas network and the coupling constraints. This study attempts to provide insights into some of the available methods of relaxation/approximation for linearizing the nonlinear, non-convex natural gas flow constraints, their performance, efficiency, and reliability of these methods for multi-area networks. In this overview, various formulations and solution methods of the optimization problem have been examined. Their performance has been compared based on certain performance metrics to find the best possible method to solve such complex optimization problems. Since the integrated systems usually are extremely large, it is vital to separate them into smaller subsystems to solve the optimization problem efficiently. There are different methods used for solving these decentralized optimization problems and it is important to study these algorithms for multi-area IEGS.
The investigation begins with an introduction of the research problem to find the best possible optimization method for IEGS. This study emphasizes the significance of relaxation techniques used to relax the non-convexity and studying optimization methods for integrated electricity natural gas systems. The mathematical formulations and optimization strategies utilized to solve the models are explained in detail. The performance of optimization algorithms is assessed based on certain performance metrics. Finally, this thesis concludes by summarizing the primary findings for the best possible optimization method and suggesting future research possibilities in this topic. To determine which approach best fits the OEF problem for IEGS, a thorough numerical comparison of various relaxation techniques, centralized and decentralized scheme of operation must be made. It is imperative to note that these techniques solve the approximated problem, not the original nonlinear, non-convex problem, which would leave some room for errors and hence an area of future improvement.
This thesis report aims to answer the following research question: “Is it possible to estimate relative velocities of vehicles surrounding the ego vehicle using a monocular camera with such an accuracy that meaningful conclusions can be made about the current traffic state?” To answer this question, a velocity estimation algorithm is developed in three major parts: object detection, object tracking with detections and velocity estimation using tracked 2D objects. For the detection part, a version of the YOLOv3 (You Only Look Once version 3) single shot detection neural network is used. For object tracking with detections, the Simple Online and Realtime Tracking (SORT) algorithm is used. The last part, velocity estimation using tracked 2D objects, a state-of-the-art method using a neural network is compared to a novel proposed method, using a 2D to 3D map in combination with a kalman filter using a constant velocity model. The results of the detection and tracking parts were good enough to reason that they are used as a base of the velocity estimation algorithm. When comparing the-state-of-the-art velocity estimation algorithm and the novel approach, the errors of the novel approach were significantly higher, and the results of the state-of-the-art methods could not be replicated. This means that the research question of this thesis can be answered with yes, it is possible to estimate relative velocities of surrounding vehicles, however the resulting estimation errors are too high to make meaningful conclusions about the current traffic state.
...
...
This thesis report aims to answer the following research question: “Is it possible to estimate relative velocities of vehicles surrounding the ego vehicle using a monocular camera with such an accuracy that meaningful conclusions can be made about the current traffic state?” To answer this question, a velocity estimation algorithm is developed in three major parts: object detection, object tracking with detections and velocity estimation using tracked 2D objects. For the detection part, a version of the YOLOv3 (You Only Look Once version 3) single shot detection neural network is used. For object tracking with detections, the Simple Online and Realtime Tracking (SORT) algorithm is used. The last part, velocity estimation using tracked 2D objects, a state-of-the-art method using a neural network is compared to a novel proposed method, using a 2D to 3D map in combination with a kalman filter using a constant velocity model. The results of the detection and tracking parts were good enough to reason that they are used as a base of the velocity estimation algorithm. When comparing the-state-of-the-art velocity estimation algorithm and the novel approach, the errors of the novel approach were significantly higher, and the results of the state-of-the-art methods could not be replicated. This means that the research question of this thesis can be answered with yes, it is possible to estimate relative velocities of surrounding vehicles, however the resulting estimation errors are too high to make meaningful conclusions about the current traffic state.
The Optimal Power Flow (OPF) problem, a cornerstone of power system operations, has gained increased attention since its inception by Carpentier in 1962. OPF is fundamentally an optimization challenge aimed at enhancing electric power system operations within the bounds of physical and operational constraints. Over the decades, various methodologies have been explored to address the OPF problem, adapting to evolving grid complexities and the integration of distributed energy resources. These advancements have brought to the fore issues related to system randomness, fluctuation, and the need for rapid control mechanisms. This thesis introduces a comprehensive solution incorporating an online optimization algorithm tailored for real-time OPF applications. This approach, characterized by minimal computation times, integrates a feedback strategy that obviates the necessity for instantaneous power demand information and employs a Shape-constrained Gaussian Process for the estimation of unknown cost functions. The proposed control algorithm demonstrates robust tracking performance and satisfactory computation efficiency, marking a significant improvement towards optimizing future power networks fraught with increasing size and complexity. Moreover, this work delves into the investigation of various system design parameters, offering insights into potential avenues for enhancing system performance. Through a meticulous examination of these parameters, the thesis sheds light on strategies to refine the integrated system’s efficacy, paving the way for more resilient and efficient power networks.
...
The Optimal Power Flow (OPF) problem, a cornerstone of power system operations, has gained increased attention since its inception by Carpentier in 1962. OPF is fundamentally an optimization challenge aimed at enhancing electric power system operations within the bounds of physical and operational constraints. Over the decades, various methodologies have been explored to address the OPF problem, adapting to evolving grid complexities and the integration of distributed energy resources. These advancements have brought to the fore issues related to system randomness, fluctuation, and the need for rapid control mechanisms. This thesis introduces a comprehensive solution incorporating an online optimization algorithm tailored for real-time OPF applications. This approach, characterized by minimal computation times, integrates a feedback strategy that obviates the necessity for instantaneous power demand information and employs a Shape-constrained Gaussian Process for the estimation of unknown cost functions. The proposed control algorithm demonstrates robust tracking performance and satisfactory computation efficiency, marking a significant improvement towards optimizing future power networks fraught with increasing size and complexity. Moreover, this work delves into the investigation of various system design parameters, offering insights into potential avenues for enhancing system performance. Through a meticulous examination of these parameters, the thesis sheds light on strategies to refine the integrated system’s efficacy, paving the way for more resilient and efficient power networks.
Advancing Deep Reinforcement Learning for Real-World Traffic Signal Control
Addressing Sampling Challenges and Multi-Modal Traffic Dynamics
Deep Reinforcement Learning (DRL) is a promising approach to Traffic Signal Control (TSC). However, significant challenges remain in translating this potential into real-world traffic management solutions. This thesis investigates obstacles hindering the application of DRL in real-world TSC, focusing on low sampling frequencies and the complexities of multi-modal traffic scenarios.
We developed a high-frequency sampling Proximal Policy Optimization (PPO) approach for TSC at a four-legged intersection, integrating both vehicle and pedestrian traffic in a multimodal setting. By employing Invalid Action Masking (IAM), we effectively handle signal timing constraints across these modalities. The framework was evaluated through traffic volume sensitivity analyses, assessments of generalization capabilities, disturbance rejection tests, and comparisons of methods for handling invalid actions.
The results indicate that short sampling intervals, such as 1 second, do not improve performance in terms of time-loss, with 4 to 6 seconds identified as the optimal range for PPO in TSC of a four-legged intersection. The findings also demonstrate that IAM can effectively be incorporated without compromising performance. When evaluating the ability to handle sudden spikes in traffic volume, PPO demonstrated superior performance, outperforming baseline methods such as max-pressure and fixed-time strategies in terms of both overshoot and settling time. Also, the results show that PPO can effectively prioritize vehicle and pedestrian modalities, displaying a clear proportional decrease in time-loss for the prioritized modality.
...
We developed a high-frequency sampling Proximal Policy Optimization (PPO) approach for TSC at a four-legged intersection, integrating both vehicle and pedestrian traffic in a multimodal setting. By employing Invalid Action Masking (IAM), we effectively handle signal timing constraints across these modalities. The framework was evaluated through traffic volume sensitivity analyses, assessments of generalization capabilities, disturbance rejection tests, and comparisons of methods for handling invalid actions.
The results indicate that short sampling intervals, such as 1 second, do not improve performance in terms of time-loss, with 4 to 6 seconds identified as the optimal range for PPO in TSC of a four-legged intersection. The findings also demonstrate that IAM can effectively be incorporated without compromising performance. When evaluating the ability to handle sudden spikes in traffic volume, PPO demonstrated superior performance, outperforming baseline methods such as max-pressure and fixed-time strategies in terms of both overshoot and settling time. Also, the results show that PPO can effectively prioritize vehicle and pedestrian modalities, displaying a clear proportional decrease in time-loss for the prioritized modality.
...
Deep Reinforcement Learning (DRL) is a promising approach to Traffic Signal Control (TSC). However, significant challenges remain in translating this potential into real-world traffic management solutions. This thesis investigates obstacles hindering the application of DRL in real-world TSC, focusing on low sampling frequencies and the complexities of multi-modal traffic scenarios.
We developed a high-frequency sampling Proximal Policy Optimization (PPO) approach for TSC at a four-legged intersection, integrating both vehicle and pedestrian traffic in a multimodal setting. By employing Invalid Action Masking (IAM), we effectively handle signal timing constraints across these modalities. The framework was evaluated through traffic volume sensitivity analyses, assessments of generalization capabilities, disturbance rejection tests, and comparisons of methods for handling invalid actions.
The results indicate that short sampling intervals, such as 1 second, do not improve performance in terms of time-loss, with 4 to 6 seconds identified as the optimal range for PPO in TSC of a four-legged intersection. The findings also demonstrate that IAM can effectively be incorporated without compromising performance. When evaluating the ability to handle sudden spikes in traffic volume, PPO demonstrated superior performance, outperforming baseline methods such as max-pressure and fixed-time strategies in terms of both overshoot and settling time. Also, the results show that PPO can effectively prioritize vehicle and pedestrian modalities, displaying a clear proportional decrease in time-loss for the prioritized modality.
We developed a high-frequency sampling Proximal Policy Optimization (PPO) approach for TSC at a four-legged intersection, integrating both vehicle and pedestrian traffic in a multimodal setting. By employing Invalid Action Masking (IAM), we effectively handle signal timing constraints across these modalities. The framework was evaluated through traffic volume sensitivity analyses, assessments of generalization capabilities, disturbance rejection tests, and comparisons of methods for handling invalid actions.
The results indicate that short sampling intervals, such as 1 second, do not improve performance in terms of time-loss, with 4 to 6 seconds identified as the optimal range for PPO in TSC of a four-legged intersection. The findings also demonstrate that IAM can effectively be incorporated without compromising performance. When evaluating the ability to handle sudden spikes in traffic volume, PPO demonstrated superior performance, outperforming baseline methods such as max-pressure and fixed-time strategies in terms of both overshoot and settling time. Also, the results show that PPO can effectively prioritize vehicle and pedestrian modalities, displaying a clear proportional decrease in time-loss for the prioritized modality.
As Autonomous Vehicles (AVs) navigate through dynamic and constantly changing environments, it is crucial that they take into account the impact of their actions on the decisions of others for safe and efficient interaction with humans. In doing so, they need to anticipate how humans will behave in different situations based on their intentions. This work aims to address these challenges by proposing an expressive game-theoretic framework for modeling the interactions between AVs and human drivers as a multi-agent dynamic game, in which each agent seeks to optimize their respective objective. The optimization problem is solved by obtaining the Nash equilibrium, which accounts for the potential non-cooperative behavior of human drivers. To incorporate the notion of intention into the game-theoretic formulation, we introduce for each agent a parameter known as Social Value Orientation (SVO), reflecting the degree to which an agent is willing to prioritize the welfare of others over its own. We then develop efficient methods to solve this nonlinear optimization problem in a receding-horizon fashion given the agents' SVOs. However, cluttered traffic scenarios are typically characterized by uncertainty regarding the intentions of other traffic participants due to noisy sensor data, and multiple equally admissible equilibrium strategies that humans may adapt to achieve their objective. Therefore, an approximate Bayesian inference method is developed to infer the intentions of the surrounding human participants by estimating the likelihood of SVO based on newly received state observations. We then integrate the estimation module into the game-theoretic planning module in a combined framework and evaluate its predictive performance against algorithms that ignore these sources of uncertainty in two simulated traffic scenarios; ramp-merging at a highway and crossing at uncontrolled intersections. Our results show that the proposed inference method exhibits superior performance compared to all other approaches, with the average prediction error approaching zero. This implies that dynamically changing the SVO values, while planning, effectively captures the true intentions of the surrounding agents.
...
As Autonomous Vehicles (AVs) navigate through dynamic and constantly changing environments, it is crucial that they take into account the impact of their actions on the decisions of others for safe and efficient interaction with humans. In doing so, they need to anticipate how humans will behave in different situations based on their intentions. This work aims to address these challenges by proposing an expressive game-theoretic framework for modeling the interactions between AVs and human drivers as a multi-agent dynamic game, in which each agent seeks to optimize their respective objective. The optimization problem is solved by obtaining the Nash equilibrium, which accounts for the potential non-cooperative behavior of human drivers. To incorporate the notion of intention into the game-theoretic formulation, we introduce for each agent a parameter known as Social Value Orientation (SVO), reflecting the degree to which an agent is willing to prioritize the welfare of others over its own. We then develop efficient methods to solve this nonlinear optimization problem in a receding-horizon fashion given the agents' SVOs. However, cluttered traffic scenarios are typically characterized by uncertainty regarding the intentions of other traffic participants due to noisy sensor data, and multiple equally admissible equilibrium strategies that humans may adapt to achieve their objective. Therefore, an approximate Bayesian inference method is developed to infer the intentions of the surrounding human participants by estimating the likelihood of SVO based on newly received state observations. We then integrate the estimation module into the game-theoretic planning module in a combined framework and evaluate its predictive performance against algorithms that ignore these sources of uncertainty in two simulated traffic scenarios; ramp-merging at a highway and crossing at uncontrolled intersections. Our results show that the proposed inference method exhibits superior performance compared to all other approaches, with the average prediction error approaching zero. This implies that dynamically changing the SVO values, while planning, effectively captures the true intentions of the surrounding agents.
Autonomous vehicles (AVs) can solve a lot of problems related to traffic safety, comfort and congestion. While the sensors used by these vehicles are getting cheaper, more accurate, and software is improving, the first completely automated vehicle does not exist yet. When AVs start gradually driving on the roads, there will be a mixed traffic situation consisting of AVs and human driven vehicles. To understand the implications on the traffic performance, this thesis will implement a Model Predictive Controller (MPC) and the behavioural models for AVs into a numerical simulation in SUMO. The MPC can be used to control multiple AVs in larger traffic simulations to analyse the effects of certain penetration rates and traffic densities when changing the AVs driving behaviour. Both the Post-Encroachment Time (PET) and Time To Collision (TTC) will be used for decision making in overtaking and collision avoidance strategies. In this thesis, the MPC which is solving a quadratic programming problem has proven to successfully overtake other vehicles in an on- and off-ramp highway scenario which results in a smaller average travel time for the AVs than for the human drivers. It is shown that the decrease in average travel time for the AVs was not at the cost of the Human Vehicles (HVs). This is done by decreasing the TTC used by the AVs as a headway for longitudinal overtaking behaviour, and the PET used for lateral lane changing behaviour, independently to a minimum of 0.1 seconds to guaranty safety, and measure the average travel times for both HVs and AVs. Multiple simulations using different penetration rates ranging from 0 to 40 percent also show that the decrease in average travel time for the AVs was not at the cost of the HVs. We can conclude that using an MPC integrated together with the behavioural rules of human drivers into numerical simulation can give a good indication of the implications that AVs have on traffic performance.
...
Autonomous vehicles (AVs) can solve a lot of problems related to traffic safety, comfort and congestion. While the sensors used by these vehicles are getting cheaper, more accurate, and software is improving, the first completely automated vehicle does not exist yet. When AVs start gradually driving on the roads, there will be a mixed traffic situation consisting of AVs and human driven vehicles. To understand the implications on the traffic performance, this thesis will implement a Model Predictive Controller (MPC) and the behavioural models for AVs into a numerical simulation in SUMO. The MPC can be used to control multiple AVs in larger traffic simulations to analyse the effects of certain penetration rates and traffic densities when changing the AVs driving behaviour. Both the Post-Encroachment Time (PET) and Time To Collision (TTC) will be used for decision making in overtaking and collision avoidance strategies. In this thesis, the MPC which is solving a quadratic programming problem has proven to successfully overtake other vehicles in an on- and off-ramp highway scenario which results in a smaller average travel time for the AVs than for the human drivers. It is shown that the decrease in average travel time for the AVs was not at the cost of the Human Vehicles (HVs). This is done by decreasing the TTC used by the AVs as a headway for longitudinal overtaking behaviour, and the PET used for lateral lane changing behaviour, independently to a minimum of 0.1 seconds to guaranty safety, and measure the average travel times for both HVs and AVs. Multiple simulations using different penetration rates ranging from 0 to 40 percent also show that the decrease in average travel time for the AVs was not at the cost of the HVs. We can conclude that using an MPC integrated together with the behavioural rules of human drivers into numerical simulation can give a good indication of the implications that AVs have on traffic performance.
Navigation systems in an Autonomous Vehicles (AV) can be divided into two parts: a path planning block which takes in the environmental data and rules to design a collision-free obstacle and a vehicle control and tracking block which generates actuator inputs for the AV to follow the reference path generated by the path planning block. Each task is fulfilled by a different algorithm with its own performance indices. These algorithms are not usually designed to get the best overall vehicle performance but the best performance of their respective blocks. A planned path that the vehicle cannot follow can therefore be generated and can lead to high tracking error and in some cases collision with obstacles. This can be solved by integrating path planning and vehicle control blocks with the dynamics of the AV.
The goal of this graduation project is therefore to develop an integrated planning and vehicle control algorithm for an Autonomous Vehicles (AV). This is done by integrating a novel-Artificial Potential Fields (APF) with Model Predictive Control (MPC) to solve both path planning and vehicle control using a single optimization problem. The addition of the AV and the Obstacle Vehicle (OV) dynamics to the optimization problem as prediction models along with recursive computation can determine accurate inputs to be given to the AV.
Unlike traditional APF-based path planning where the minimum potential path is generated as the result of a gradient descent method applied on the available map data and obstacle information, the APF is added as a cost to the objective function of the MPC based optimization problem to find the minimum potential path. By using a receding horizon approach for solving the final optimization problem, the potential field can be updated at each time step to avoid moving obstacles. The dynamics of the vehicle added to the optimization problem include both lateral and longitudinal dynamics and are linearized at each time step at the current state of the AV.
This however generates a path which does not travel in the centre of the lane and makes risky manoeuvres. Therefore, a Mixed-Integer Model Predictive Control (MIMPC) algorithm with logical constraints is used to generate an optimal lane to travel in. This optimal lane is used to generate a road potential which can guide the vehicle to the centre of the optimal lane. The MIMPC and the APF-MPC algorithms are run successively to generate a collision-free path.
The logical constraints, also called MLD constraints are converted into a set of linear inequalities with the introduction of logical variables. These logical variables are used to represent individual logical constraints based on the states of the system and on a combination of logical and state constraints. A novel-APF inspired by the Yukawa Potential [?] is designed to represent each obstacle. A convex representation of this non-convex obstacle potential is formulated to simplify the optimization problem. The convex representation of the obstacle APF is obtained by approximating it using a region-based APF where the region is defined by the position of the AV around the obstacle. This is further simplified by approximation using a quadratic Taylor-series expansion.
The simulation was performed on MATLAB on a two-lane road with multiple obstacles. The thesis report ends with a discussion on future work to be taken to further enhance the performance of the controller and to make it road-ready. ...
The goal of this graduation project is therefore to develop an integrated planning and vehicle control algorithm for an Autonomous Vehicles (AV). This is done by integrating a novel-Artificial Potential Fields (APF) with Model Predictive Control (MPC) to solve both path planning and vehicle control using a single optimization problem. The addition of the AV and the Obstacle Vehicle (OV) dynamics to the optimization problem as prediction models along with recursive computation can determine accurate inputs to be given to the AV.
Unlike traditional APF-based path planning where the minimum potential path is generated as the result of a gradient descent method applied on the available map data and obstacle information, the APF is added as a cost to the objective function of the MPC based optimization problem to find the minimum potential path. By using a receding horizon approach for solving the final optimization problem, the potential field can be updated at each time step to avoid moving obstacles. The dynamics of the vehicle added to the optimization problem include both lateral and longitudinal dynamics and are linearized at each time step at the current state of the AV.
This however generates a path which does not travel in the centre of the lane and makes risky manoeuvres. Therefore, a Mixed-Integer Model Predictive Control (MIMPC) algorithm with logical constraints is used to generate an optimal lane to travel in. This optimal lane is used to generate a road potential which can guide the vehicle to the centre of the optimal lane. The MIMPC and the APF-MPC algorithms are run successively to generate a collision-free path.
The logical constraints, also called MLD constraints are converted into a set of linear inequalities with the introduction of logical variables. These logical variables are used to represent individual logical constraints based on the states of the system and on a combination of logical and state constraints. A novel-APF inspired by the Yukawa Potential [?] is designed to represent each obstacle. A convex representation of this non-convex obstacle potential is formulated to simplify the optimization problem. The convex representation of the obstacle APF is obtained by approximating it using a region-based APF where the region is defined by the position of the AV around the obstacle. This is further simplified by approximation using a quadratic Taylor-series expansion.
The simulation was performed on MATLAB on a two-lane road with multiple obstacles. The thesis report ends with a discussion on future work to be taken to further enhance the performance of the controller and to make it road-ready. ...
Navigation systems in an Autonomous Vehicles (AV) can be divided into two parts: a path planning block which takes in the environmental data and rules to design a collision-free obstacle and a vehicle control and tracking block which generates actuator inputs for the AV to follow the reference path generated by the path planning block. Each task is fulfilled by a different algorithm with its own performance indices. These algorithms are not usually designed to get the best overall vehicle performance but the best performance of their respective blocks. A planned path that the vehicle cannot follow can therefore be generated and can lead to high tracking error and in some cases collision with obstacles. This can be solved by integrating path planning and vehicle control blocks with the dynamics of the AV.
The goal of this graduation project is therefore to develop an integrated planning and vehicle control algorithm for an Autonomous Vehicles (AV). This is done by integrating a novel-Artificial Potential Fields (APF) with Model Predictive Control (MPC) to solve both path planning and vehicle control using a single optimization problem. The addition of the AV and the Obstacle Vehicle (OV) dynamics to the optimization problem as prediction models along with recursive computation can determine accurate inputs to be given to the AV.
Unlike traditional APF-based path planning where the minimum potential path is generated as the result of a gradient descent method applied on the available map data and obstacle information, the APF is added as a cost to the objective function of the MPC based optimization problem to find the minimum potential path. By using a receding horizon approach for solving the final optimization problem, the potential field can be updated at each time step to avoid moving obstacles. The dynamics of the vehicle added to the optimization problem include both lateral and longitudinal dynamics and are linearized at each time step at the current state of the AV.
This however generates a path which does not travel in the centre of the lane and makes risky manoeuvres. Therefore, a Mixed-Integer Model Predictive Control (MIMPC) algorithm with logical constraints is used to generate an optimal lane to travel in. This optimal lane is used to generate a road potential which can guide the vehicle to the centre of the optimal lane. The MIMPC and the APF-MPC algorithms are run successively to generate a collision-free path.
The logical constraints, also called MLD constraints are converted into a set of linear inequalities with the introduction of logical variables. These logical variables are used to represent individual logical constraints based on the states of the system and on a combination of logical and state constraints. A novel-APF inspired by the Yukawa Potential [?] is designed to represent each obstacle. A convex representation of this non-convex obstacle potential is formulated to simplify the optimization problem. The convex representation of the obstacle APF is obtained by approximating it using a region-based APF where the region is defined by the position of the AV around the obstacle. This is further simplified by approximation using a quadratic Taylor-series expansion.
The simulation was performed on MATLAB on a two-lane road with multiple obstacles. The thesis report ends with a discussion on future work to be taken to further enhance the performance of the controller and to make it road-ready.
The goal of this graduation project is therefore to develop an integrated planning and vehicle control algorithm for an Autonomous Vehicles (AV). This is done by integrating a novel-Artificial Potential Fields (APF) with Model Predictive Control (MPC) to solve both path planning and vehicle control using a single optimization problem. The addition of the AV and the Obstacle Vehicle (OV) dynamics to the optimization problem as prediction models along with recursive computation can determine accurate inputs to be given to the AV.
Unlike traditional APF-based path planning where the minimum potential path is generated as the result of a gradient descent method applied on the available map data and obstacle information, the APF is added as a cost to the objective function of the MPC based optimization problem to find the minimum potential path. By using a receding horizon approach for solving the final optimization problem, the potential field can be updated at each time step to avoid moving obstacles. The dynamics of the vehicle added to the optimization problem include both lateral and longitudinal dynamics and are linearized at each time step at the current state of the AV.
This however generates a path which does not travel in the centre of the lane and makes risky manoeuvres. Therefore, a Mixed-Integer Model Predictive Control (MIMPC) algorithm with logical constraints is used to generate an optimal lane to travel in. This optimal lane is used to generate a road potential which can guide the vehicle to the centre of the optimal lane. The MIMPC and the APF-MPC algorithms are run successively to generate a collision-free path.
The logical constraints, also called MLD constraints are converted into a set of linear inequalities with the introduction of logical variables. These logical variables are used to represent individual logical constraints based on the states of the system and on a combination of logical and state constraints. A novel-APF inspired by the Yukawa Potential [?] is designed to represent each obstacle. A convex representation of this non-convex obstacle potential is formulated to simplify the optimization problem. The convex representation of the obstacle APF is obtained by approximating it using a region-based APF where the region is defined by the position of the AV around the obstacle. This is further simplified by approximation using a quadratic Taylor-series expansion.
The simulation was performed on MATLAB on a two-lane road with multiple obstacles. The thesis report ends with a discussion on future work to be taken to further enhance the performance of the controller and to make it road-ready.
Optimal bidirectional charging control of Electric Vehicles
Minimizing carbon footprint in a realistic simulation environment
Master thesis
(2022)
-
B.A. Swens, S. Grammatico, P. Mohajerin Esfahani, Simon H. Tindemans, H. Keemink
Electricity grids worldwide are experiencing increased peak demands and decreasing simultaneity due to higher shares of Renewable Energy Sources (RES). It is expected that many grids will soon reach their limits. One solution to mitigate these issues is exploiting flexibility in e.g. electric vehicles. In this work, a shrinking horizon model predictive controller is constructed to optimally charge and discharge EVs with respect to the day ahead electricity price or grid carbon intensity. The model takes into account that users with a dual rate electricity plan only want to charge during their off-peak hours. A feature to implement a household PV setup in the optimization is included. The possible consequences in terms of associated carbon emissions, utility costs and user costs are analysed using a simulation based on data from 4279 charging sessions that took place between June 23, 2021 and June 23, 2022. The sessions are split in 2855 weekday sessions (duration between 4 and 24 hours) and 1424 weekend sessions (duration between 4 and 60 hours). It is found that using current circumstances, minimizing the carbon emissions using bidirectional charging results in a higher price (5.5 %) for the utility than using the current state of the art, unidirectional charging minimizing the wholesale electricity cost. Bidirectional charging minimizing the wholesale electricity cost results in higher emissions compared to unidirectional charging (2.8 %), and even compared to uncontrolled charging (0.9 – 3.6 %). The reason for this seems to be a negative correlation between carbon intensity and wholesale price during the times that vehicles are typically connected although this needs further investigation to be confirmed.
...
Electricity grids worldwide are experiencing increased peak demands and decreasing simultaneity due to higher shares of Renewable Energy Sources (RES). It is expected that many grids will soon reach their limits. One solution to mitigate these issues is exploiting flexibility in e.g. electric vehicles. In this work, a shrinking horizon model predictive controller is constructed to optimally charge and discharge EVs with respect to the day ahead electricity price or grid carbon intensity. The model takes into account that users with a dual rate electricity plan only want to charge during their off-peak hours. A feature to implement a household PV setup in the optimization is included. The possible consequences in terms of associated carbon emissions, utility costs and user costs are analysed using a simulation based on data from 4279 charging sessions that took place between June 23, 2021 and June 23, 2022. The sessions are split in 2855 weekday sessions (duration between 4 and 24 hours) and 1424 weekend sessions (duration between 4 and 60 hours). It is found that using current circumstances, minimizing the carbon emissions using bidirectional charging results in a higher price (5.5 %) for the utility than using the current state of the art, unidirectional charging minimizing the wholesale electricity cost. Bidirectional charging minimizing the wholesale electricity cost results in higher emissions compared to unidirectional charging (2.8 %), and even compared to uncontrolled charging (0.9 – 3.6 %). The reason for this seems to be a negative correlation between carbon intensity and wholesale price during the times that vehicles are typically connected although this needs further investigation to be confirmed.
Traffic network management
"Comparing algorithms for network-wide traffic management using Eclipse SUMO: A pragmatic approach versus Model Predictive Control"
Master thesis
(2022)
-
L. Heunks, S. Grammatico, Tijs Van Bakel, S.D. Gonçalves Melo Pequito, A. Dabiri
The need for smart traffic control has grown over the last years. Initiated by an increased amount of traffic. Network-wide traffic control is becoming a more interesting field for traffic control. Mainly because computer power has increased and optimisation techniques improved. Network-wide traffic aims to improve the overall traffic state by looking at the entire problem instead of sub-problems. Besides improving traffic conditions, network-wide traffic control could support road operators in simplifying their work by taking over some tasks and keeping track of the situation.
For this research, we compare a pragmatic, user-friendly and transparent control method versus a Model Predictive Control (MPC) approach. For the MPC controller, the second-order macroscopic METANET model is chosen. The METANET model describes a network as a directed graph. We test both controllers on two small scale freeway traffic networks. The control measures that are implemented are ramp-metering and rerouting. The simulations are done in a SUMO environment. The key performance index (KPI) used for comparison is Total Time Spend (TTS). The resulting optimisation problem is a Mixed Non-linear Integer Problem (MINLP). This problem is solved with a heuristic method by a Genetic Algorithm (GA).
Simulations for the two networks in a demand scenario around critical density are analysed over 20 iterations. The results prove the potential of both algorithms since both improved the TTS significantly. The NM excels in ease of implementation and ease of understanding for non-experts. While the MPC outperforms the NM in TTS reduction, it is harder to configure and understand for non-experts. The MPC is successfully tested on its capability to prevent undesired behaviour from happening by adding penalties to the objective function. For future research, larger networks need to be investigated, with a focus on simplifying the resulting optimisation problem. It is expected that a piecewise affine approximation is a promising method. ...
For this research, we compare a pragmatic, user-friendly and transparent control method versus a Model Predictive Control (MPC) approach. For the MPC controller, the second-order macroscopic METANET model is chosen. The METANET model describes a network as a directed graph. We test both controllers on two small scale freeway traffic networks. The control measures that are implemented are ramp-metering and rerouting. The simulations are done in a SUMO environment. The key performance index (KPI) used for comparison is Total Time Spend (TTS). The resulting optimisation problem is a Mixed Non-linear Integer Problem (MINLP). This problem is solved with a heuristic method by a Genetic Algorithm (GA).
Simulations for the two networks in a demand scenario around critical density are analysed over 20 iterations. The results prove the potential of both algorithms since both improved the TTS significantly. The NM excels in ease of implementation and ease of understanding for non-experts. While the MPC outperforms the NM in TTS reduction, it is harder to configure and understand for non-experts. The MPC is successfully tested on its capability to prevent undesired behaviour from happening by adding penalties to the objective function. For future research, larger networks need to be investigated, with a focus on simplifying the resulting optimisation problem. It is expected that a piecewise affine approximation is a promising method. ...
The need for smart traffic control has grown over the last years. Initiated by an increased amount of traffic. Network-wide traffic control is becoming a more interesting field for traffic control. Mainly because computer power has increased and optimisation techniques improved. Network-wide traffic aims to improve the overall traffic state by looking at the entire problem instead of sub-problems. Besides improving traffic conditions, network-wide traffic control could support road operators in simplifying their work by taking over some tasks and keeping track of the situation.
For this research, we compare a pragmatic, user-friendly and transparent control method versus a Model Predictive Control (MPC) approach. For the MPC controller, the second-order macroscopic METANET model is chosen. The METANET model describes a network as a directed graph. We test both controllers on two small scale freeway traffic networks. The control measures that are implemented are ramp-metering and rerouting. The simulations are done in a SUMO environment. The key performance index (KPI) used for comparison is Total Time Spend (TTS). The resulting optimisation problem is a Mixed Non-linear Integer Problem (MINLP). This problem is solved with a heuristic method by a Genetic Algorithm (GA).
Simulations for the two networks in a demand scenario around critical density are analysed over 20 iterations. The results prove the potential of both algorithms since both improved the TTS significantly. The NM excels in ease of implementation and ease of understanding for non-experts. While the MPC outperforms the NM in TTS reduction, it is harder to configure and understand for non-experts. The MPC is successfully tested on its capability to prevent undesired behaviour from happening by adding penalties to the objective function. For future research, larger networks need to be investigated, with a focus on simplifying the resulting optimisation problem. It is expected that a piecewise affine approximation is a promising method.
For this research, we compare a pragmatic, user-friendly and transparent control method versus a Model Predictive Control (MPC) approach. For the MPC controller, the second-order macroscopic METANET model is chosen. The METANET model describes a network as a directed graph. We test both controllers on two small scale freeway traffic networks. The control measures that are implemented are ramp-metering and rerouting. The simulations are done in a SUMO environment. The key performance index (KPI) used for comparison is Total Time Spend (TTS). The resulting optimisation problem is a Mixed Non-linear Integer Problem (MINLP). This problem is solved with a heuristic method by a Genetic Algorithm (GA).
Simulations for the two networks in a demand scenario around critical density are analysed over 20 iterations. The results prove the potential of both algorithms since both improved the TTS significantly. The NM excels in ease of implementation and ease of understanding for non-experts. While the MPC outperforms the NM in TTS reduction, it is harder to configure and understand for non-experts. The MPC is successfully tested on its capability to prevent undesired behaviour from happening by adding penalties to the objective function. For future research, larger networks need to be investigated, with a focus on simplifying the resulting optimisation problem. It is expected that a piecewise affine approximation is a promising method.
The Quasi-Online Algorithm in a Robot Packing Environment
Implementation of an Improved Bin Packing Algorithm
Enabling Hybrid Distributed Model Predictive Control of Traffic Signals In Urban Arterial Networks
An on-street application in industry
Road users still encounter unnecessary delays due to inefficient traffic control in urban traffic networks. These delays are ever-increasing and have large environmental and economic consequences. In the Netherlands, most intersections are controlled using actuated controllers, which respond directly to the current traffic demand obtained from vehicle detections. This causes the signal timing to be short-sighted and coordination between intersections is limited.
In order to reduce vehicle delays in urban traffic networks, a hybrid Decentralised Model Predictive traffic signal Controller (DeMPC) is developed based on a new mixed logical dynamical model of a signalised intersection. Operation constraints are also formulated for on-street application and vehicle arrivals at the intersection are predicted by a long-short term memory neural network. A distributed algorithm is also developed to coordinate the signals of the decentralized controllers in urban traffic networks.
Simulation results of a real-world intersection in North-Holland show that the mean delay time per vehicle of the developed DeMPC is 24% lower than the greedy control method of Yunex and only 8% above the actuated controller. The DeMPC outperforms both actuated and Yunex’s method by 10% and 26% in terms of the total number of stops, respectively. For future work, the distributed coordination algorithm should be evaluated by simulation and the DeMPC should be evaluated in combination with green light optimised speed advice. ...
In order to reduce vehicle delays in urban traffic networks, a hybrid Decentralised Model Predictive traffic signal Controller (DeMPC) is developed based on a new mixed logical dynamical model of a signalised intersection. Operation constraints are also formulated for on-street application and vehicle arrivals at the intersection are predicted by a long-short term memory neural network. A distributed algorithm is also developed to coordinate the signals of the decentralized controllers in urban traffic networks.
Simulation results of a real-world intersection in North-Holland show that the mean delay time per vehicle of the developed DeMPC is 24% lower than the greedy control method of Yunex and only 8% above the actuated controller. The DeMPC outperforms both actuated and Yunex’s method by 10% and 26% in terms of the total number of stops, respectively. For future work, the distributed coordination algorithm should be evaluated by simulation and the DeMPC should be evaluated in combination with green light optimised speed advice. ...
Road users still encounter unnecessary delays due to inefficient traffic control in urban traffic networks. These delays are ever-increasing and have large environmental and economic consequences. In the Netherlands, most intersections are controlled using actuated controllers, which respond directly to the current traffic demand obtained from vehicle detections. This causes the signal timing to be short-sighted and coordination between intersections is limited.
In order to reduce vehicle delays in urban traffic networks, a hybrid Decentralised Model Predictive traffic signal Controller (DeMPC) is developed based on a new mixed logical dynamical model of a signalised intersection. Operation constraints are also formulated for on-street application and vehicle arrivals at the intersection are predicted by a long-short term memory neural network. A distributed algorithm is also developed to coordinate the signals of the decentralized controllers in urban traffic networks.
Simulation results of a real-world intersection in North-Holland show that the mean delay time per vehicle of the developed DeMPC is 24% lower than the greedy control method of Yunex and only 8% above the actuated controller. The DeMPC outperforms both actuated and Yunex’s method by 10% and 26% in terms of the total number of stops, respectively. For future work, the distributed coordination algorithm should be evaluated by simulation and the DeMPC should be evaluated in combination with green light optimised speed advice.
In order to reduce vehicle delays in urban traffic networks, a hybrid Decentralised Model Predictive traffic signal Controller (DeMPC) is developed based on a new mixed logical dynamical model of a signalised intersection. Operation constraints are also formulated for on-street application and vehicle arrivals at the intersection are predicted by a long-short term memory neural network. A distributed algorithm is also developed to coordinate the signals of the decentralized controllers in urban traffic networks.
Simulation results of a real-world intersection in North-Holland show that the mean delay time per vehicle of the developed DeMPC is 24% lower than the greedy control method of Yunex and only 8% above the actuated controller. The DeMPC outperforms both actuated and Yunex’s method by 10% and 26% in terms of the total number of stops, respectively. For future work, the distributed coordination algorithm should be evaluated by simulation and the DeMPC should be evaluated in combination with green light optimised speed advice.
Multi-Vehicle Scenario-Based Trajectory Optimisation for Automated Driving
An Application to Urban Traffic Situations
Automated driving is where automobiles meet robotics. With the recent advances in intelligence, sensor technology, wireless technology, and computation power, we are inching ever closer to realising full autonomy in a vehicle. We are nowhere near the end of the line, however. Automated vehicles will have to interact with static obstacles like pavements, dividers, and poles and dynamic elements like pedestrians and other vehicles. This opens up a range of sub-topics on privacy, safety, and decision-making. While driving on the road in the presence of other agents(human or autonomous), safety constraints and collision avoidance are of paramount importance. Even with this achieved, we need a good performance in the latency of processing each iteration; we need constraints to be followed and, of course, ensure minimal errors in our control.
Through this thesis, we introduce automated driving, its general pipeline stack, and industry standards for autonomy. We then get ourselves up-to-date with the latest trends and advances in motion planning, decision-making, and control of automated vehicles. Once that is covered, we narrow our focus towards using a scenario-based approach to safe trajectory planning. We delve in-depth into safety constraints guaranteed by this approach and discuss previous results obtained by using this method with pedestrians in an urban setting.
This thesis aims to extend this previously used scenario-based planning method to a multi-vehicle implementation. Two methods of modelling scenario distributions(assumed to be Gaussian) are proposed, implemented, and compared using evaluation metrics like computation times, safety, and time taken to reach the goal. The first method models each obstacle vehicle as a series of obstacles linked together by the vehicle constraints. Thus, each vehicle is represented by multiple collision regions corresponding to each obstacle. The second method models the vehicle as having a single collision region with multiple scenario distributions. This extension's safety/risk guarantee is shown both theoretically and experimentally. Experiments are conducted in simulation on an urban straight road and at a T-Junction with multiple obstacle vehicles, and performances are compared, not only between these two methods but also with each obstacle modelled as a single disc, which is the baseline implementation. Conclusions are then made based on the performance metrics, and further improvements are proposed. It is shown that modelling the vehicle as a series of linked scenarios improves over the baseline method and the multiple obstacle discs implementation in terms of safety and computation times respectively. ...
Through this thesis, we introduce automated driving, its general pipeline stack, and industry standards for autonomy. We then get ourselves up-to-date with the latest trends and advances in motion planning, decision-making, and control of automated vehicles. Once that is covered, we narrow our focus towards using a scenario-based approach to safe trajectory planning. We delve in-depth into safety constraints guaranteed by this approach and discuss previous results obtained by using this method with pedestrians in an urban setting.
This thesis aims to extend this previously used scenario-based planning method to a multi-vehicle implementation. Two methods of modelling scenario distributions(assumed to be Gaussian) are proposed, implemented, and compared using evaluation metrics like computation times, safety, and time taken to reach the goal. The first method models each obstacle vehicle as a series of obstacles linked together by the vehicle constraints. Thus, each vehicle is represented by multiple collision regions corresponding to each obstacle. The second method models the vehicle as having a single collision region with multiple scenario distributions. This extension's safety/risk guarantee is shown both theoretically and experimentally. Experiments are conducted in simulation on an urban straight road and at a T-Junction with multiple obstacle vehicles, and performances are compared, not only between these two methods but also with each obstacle modelled as a single disc, which is the baseline implementation. Conclusions are then made based on the performance metrics, and further improvements are proposed. It is shown that modelling the vehicle as a series of linked scenarios improves over the baseline method and the multiple obstacle discs implementation in terms of safety and computation times respectively. ...
Automated driving is where automobiles meet robotics. With the recent advances in intelligence, sensor technology, wireless technology, and computation power, we are inching ever closer to realising full autonomy in a vehicle. We are nowhere near the end of the line, however. Automated vehicles will have to interact with static obstacles like pavements, dividers, and poles and dynamic elements like pedestrians and other vehicles. This opens up a range of sub-topics on privacy, safety, and decision-making. While driving on the road in the presence of other agents(human or autonomous), safety constraints and collision avoidance are of paramount importance. Even with this achieved, we need a good performance in the latency of processing each iteration; we need constraints to be followed and, of course, ensure minimal errors in our control.
Through this thesis, we introduce automated driving, its general pipeline stack, and industry standards for autonomy. We then get ourselves up-to-date with the latest trends and advances in motion planning, decision-making, and control of automated vehicles. Once that is covered, we narrow our focus towards using a scenario-based approach to safe trajectory planning. We delve in-depth into safety constraints guaranteed by this approach and discuss previous results obtained by using this method with pedestrians in an urban setting.
This thesis aims to extend this previously used scenario-based planning method to a multi-vehicle implementation. Two methods of modelling scenario distributions(assumed to be Gaussian) are proposed, implemented, and compared using evaluation metrics like computation times, safety, and time taken to reach the goal. The first method models each obstacle vehicle as a series of obstacles linked together by the vehicle constraints. Thus, each vehicle is represented by multiple collision regions corresponding to each obstacle. The second method models the vehicle as having a single collision region with multiple scenario distributions. This extension's safety/risk guarantee is shown both theoretically and experimentally. Experiments are conducted in simulation on an urban straight road and at a T-Junction with multiple obstacle vehicles, and performances are compared, not only between these two methods but also with each obstacle modelled as a single disc, which is the baseline implementation. Conclusions are then made based on the performance metrics, and further improvements are proposed. It is shown that modelling the vehicle as a series of linked scenarios improves over the baseline method and the multiple obstacle discs implementation in terms of safety and computation times respectively.
Through this thesis, we introduce automated driving, its general pipeline stack, and industry standards for autonomy. We then get ourselves up-to-date with the latest trends and advances in motion planning, decision-making, and control of automated vehicles. Once that is covered, we narrow our focus towards using a scenario-based approach to safe trajectory planning. We delve in-depth into safety constraints guaranteed by this approach and discuss previous results obtained by using this method with pedestrians in an urban setting.
This thesis aims to extend this previously used scenario-based planning method to a multi-vehicle implementation. Two methods of modelling scenario distributions(assumed to be Gaussian) are proposed, implemented, and compared using evaluation metrics like computation times, safety, and time taken to reach the goal. The first method models each obstacle vehicle as a series of obstacles linked together by the vehicle constraints. Thus, each vehicle is represented by multiple collision regions corresponding to each obstacle. The second method models the vehicle as having a single collision region with multiple scenario distributions. This extension's safety/risk guarantee is shown both theoretically and experimentally. Experiments are conducted in simulation on an urban straight road and at a T-Junction with multiple obstacle vehicles, and performances are compared, not only between these two methods but also with each obstacle modelled as a single disc, which is the baseline implementation. Conclusions are then made based on the performance metrics, and further improvements are proposed. It is shown that modelling the vehicle as a series of linked scenarios improves over the baseline method and the multiple obstacle discs implementation in terms of safety and computation times respectively.