Person | TU Delft Repository

M.A. Sharifi Kolarijani

View Orcid Profile

View Pure Profile

Authored

5 records found

Finite-Dimensional Approximation in Dual Domain

With Applications in Opinion Dynamics and Dynamic Programming

Doctoral thesis - M.A. Sharifi Kolarijani

This thesis is comprised of two main parts. In the first part of the thesis, we study the nonlinear Fokker-Planck (FP) equation that arises as a mean-field (macroscopic) approximation of the bounded confidence opinion dynamics, where opinions are influenced by environmental noises and opinions of radicals (stubborn individuals). The distribution of radical opinions serves as an infinite-dimensional exogenous input to the FP equation, visibly influencing the steady opinion profile. We first establish the mathematical properties of the FP equation. In particular, we (i) show the well-posedness of the dynamic equation, (ii) provide existence result accompanied by a quantitative global estimate for the corresponding stationary solution, and, (iii) establish an explicit lower bound on the noise level that guarantees exponential convergence of the dynamics to stationary state. Combining the results in (ii) and (iii) readily yields the input-output stability of the system for sufficiently large noises. Next, using Fourier analysis, the structure of opinion clusters under the uniform initial distribution is examined. Specifically, two numerical schemes for (i) identification of order-disorder transition and (ii) characterization of initial clustering behavior are provided. The results of the analysis are validated through several numerical simulations of the continuum-agent model (partial differential equation) and the corresponding discrete-agent model (interacting stochastic differential equations) for a particular distribution of radicals. In the second part of the thesis, we focus on the value iteration algorithm for solving optimal control problems. We propose two novel numerical schemes for approximate implementation of the dynamic programming (DP) operation concerned with finite-horizon, optimal control of deterministic, discrete-time systems with input-affine dynamics. The proposed algorithms involve discretization of the state and input spaces and are based on an alternative path that solves the dual problem corresponding to the DP operation. We provide error bounds for the proposed algorithms, along with detailed analyses of their computational complexity. In particular, for a specific class of problems with separable data in the state and input variables, the proposed approach can reduce the typical time complexity of the DP operation from O(XU) to O(X +U), where X and U denote the size of the discrete state and input spaces, respectively. We next discuss the extensions of the proposed conjugate value iteration algorithm for problems with separable data. The extensions are three-fold: We consider (i) infinite-horizon, discounted cost problems with (ii) stochastic dynamics, while (iii) computing the conjugate of input cost numerically. In particular, we analyze the convergence, complexity, and error of the proposed algorithm under these extensions. The theoretical results are validated through multiple numerical examples.@en

Macroscopic Noisy Bounded Confidence Models with Distributed Radical Opinions

Journal article - M.A. Sharifi Kolarijani, Anton V. Proskurnikov, Anton V. Proskurnikov, P. Mohajerin Esfahani

In this article, we study the nonlinear Fokker-Planck (FP) equation that arises as a mean-field (macroscopic) approximation of bounded confidence opinion dynamics, where opinions are influenced by environmental noises and opinions of radicals (stubborn individuals). The distribut ...

Fast Approximate Dynamic Programming for Input-Affine Dynamics

Journal article - M.A. Sharifi Kolarijani, P. Mohajerin Esfahani

We propose two novel numerical schemes for the approximate implementation of the dynamic programming (DP) operation concerned with finite-horizon optimal control of discrete-time systems with input-affine dynamics. The proposed algorithms involve discretization of the state and i ...

Fast Approximate Dynamic Programming for Infinite-Horizon Markov Decision Processes

Conference paper - M.A. Sharifi Kolarijani, G.F. Max, P. Mohajerin Esfahani

In this study, we consider the infinite-horizon, discounted cost, optimal control of stochastic nonlinear systems with separable cost and constraints in the state and input variables. Using the linear-time Legendre transform, we propose a novel numerical scheme for implementation ...

Long-term Behavior of Mean-field Noisy Bounded Confidence Models with Distributed Radicals

Conference paper - M.A. Sharifi Kolarijani, A.V. Proskurnikov, A.V. Proskurnikov, P. Mohajerin Esfahani, P. Mohajerin Esfahani

In this paper, we consider the mean-field model of noisy bounded confidence opinion dynamics under exogenous influence of static radical opinions. The long-term behavior of the model is analyzed by providing a sufficient condition for exponential convergence of the dynamics to st ...

Contributed

5 records found

On the road from Model-Based Dynamical Programming to Model-Free Reinforcement Learning

A sample efficient approach

Master thesis - P. Mur Uribe, P. Mohajerin Esfahani, M.A. Sharifi Kolarijani

This thesis introduces a new method, called Mixed Iteration, for controlling Markov Decision Processes when partial information is known about the dynamics of the Markov Decision Process. The algorithm uses sampling to calculate the expectation of partially known dynamics in stoc ...

Fast Dynamic Programming

A Numerical Method for Solving Dynamic Programming Problems

Master thesis - G.F. Max, T. Keviczky, P. Mohajerin Esfahani, M.A. Sharifi Kolarijani, A.J. van der Veen

A well-established method for finding the optimal control policy for a given dynamical system is to solve the problem iteratively going from its terminal state "backwards" in time, known as Dynamic Programming Algorithm. For a generic problem with discrete state/action space, the ...

Control of Dynamical Systems via Deep Kernel Learning

Master thesis - M.A.B. Tan, L. Laurenti, M. Kok, M.A. Sharifi Kolarijani

In the field of Systems and Control, optimal control problem-solving for complex systems is a core task. The development of accurate mathematical models to represent these systems’ dynamics is often difficult. This complexity comes from potential uncertainties, complex non-linearities, or unknown factors that might affect the system. Because of these challenges, there is a need for methods that can understand the dynamics using available data and control strategies that can work with such models without relying too much on expert knowledge or task-specific insights. These methods are essential for creating efficient and reliable solutions in a wide variety of applications within the discipline. The need for models that do not require expert knowledge has spurred the interest in applying machine learning methods to control problems. Probabilistic Inference for Learning COntrol (PILCO) is a model-based Reinforcement Learning (RL) algorithm known for its probabilistic approach to model-based RL. By employing Gaussian Process (GP) dynamics models, PILCO integrates uncertainties into its learning process, allowing it to derive control policies from limited data. PILCO’s use of the Squared Exponential (SE) kernel in its GP can restrict the learning capacity. Especially in higher-dimensional spaces, due to the SE kernel’s inherent smoothness assumption that might not capture complex or non-smooth dynamics effectively. The algorithm’s reliance on moment matching for approximating posterior distributions introduces another weakness, which can lead to inaccuracies in non-Gaussian or multi-modal contexts. These shortcomings may limit PILCO’s efficiency and scalability in more complex, higher-dimensional tasks or in situations where the underlying dynamics are not well-captured by the chosen kernel and approximation methods. This thesis introduces Deep Kernel PILCO (DKL PILCO), a novel framework that uses Deep Kernel Learning (DKL) for learning the dynamics, and the Unscented Transform (UT) to propagate the uncertainty. The effectiveness of this approach is demonstrated across various tasks, highlighting the potential of DKL and UT to enhance the scalability and efficiency of model-based RL methods such as PILCO, making it a promising candidate for real-world control applications.

Data-driven Dynamic Programming: a Peak Shaving Application

Master thesis - F.I. Silva Cruz Guerreiro Mestre, P. Mohajerin Esfahani, M.A. Sharifi Kolarijani, M. Mazo, Simon H. Tindemans

The rising number of electricity consumers poses a challenge to power generators and grid operators in maintaining a balanced grid. Peak shaving is a technique that consists of shifting electricity consumption from hours of high demand to times of low demand, and has been gaining ...

Conjugate Dynamic Programming

Master thesis - C. Rodopoulos, P. Mohajerin Esfahani, M.A. Sharifi Kolarijani, G.F. Max, A. Dabiri

In decision making problems, the ability to compute the optimal solution can pose a serious challenge. Dynamic Programming (DP) aims to provide a framework to deal with a category of such problems, namely ones that involve sequential decision making. By dividing the original cont ...