Distributed Actor-Critic Algorithms for Multiagent Reinforcement Learning Over Directed Graphs

Actor-critic (AC) cooperative multiagent reinforcement learning (MARL) over directed graphs is studied in this article. The goal of the agents in MARL is to maximize the globally averaged return in a distributed way, i.e., each agent can only exchange information with its neighboring agents. AC methods proposed in the literature require the...

Efficient Shapley Value Approximation Methods: for Cost Redistribution in Energy Communities

With the emergence of energy communities, where a number of prosumers (consumers with their own energy generation) invest in shared renewable generation capacity and battery storage, the issue of fair allocation of benefits and costs has become increasingly important. The Shapley value, a solution concept in cooperative game theory initially...

Local Stackelberg equilibrium seeking in generalized aggregative games

We propose a two-layer, semi-decentralized algorithm to compute a local solution to the Stackelberg equilibrium problem in aggregative games with coupling constraints. Specifically, we focus on a single-leader, multiple follower problem, and after equivalently recasting the Stackelberg game as a mathematical program with complementarity...

Approximate solutions to the optimal flow problem of multi-area integrated electrical and gas systems

We formulate the optimal flow problem in a multi-area integrated electrical and gas system as a mixed-integer optimization problem by approximating the non-linear gas flows with piece-wise affine functions, thus resulting in a set of mixed-integer linear constraints. For its solution, we propose a novel algorithm that consists in one stage for...

Informative Path Planning in Random Fields via Mixed Integer Programming

We present a new mixed integer formulation for the discrete informative path planning problem in random fields. The objective is to compute a budget constrained path while collecting measurements whose linear estimate results in minimum error over a finite set of prediction locations. The problem is known to be NP-hard. However, we strive to...

SeqClu-PV: An extension of online K-medoids to efficiently cluster sequences real-time

Real-time sequence clustering is the problem of clustering an infinite stream of sequences in real time with limited memory. A variant of the k-medoids algorithm called <i>SeqClu </i>is the suggested approach, representing a cluster with <i>p </i>most representative sequences of the cluster, called prototypes, to solve the problem of maintaining...

A distributed forward-backward algorithm for stochastic generalized Nash equilibrium seeking

We consider the stochastic generalized Nash equilibrium problem (SGNEP) with expected-value cost functions. Inspired by Yi and Pavel (2019), we propose a distributed generalized Nash equilibrium seeking algorithm based on the preconditioned forward-backward operator splitting for SGNEPs, where, at each iteration, the expected value of the...

Isochronous partitions for region-based self-triggered control

In this article, we propose a region-based self-triggered control (STC) scheme for nonlinear systems. The state space is partitioned into a finite number of regions, each of which is associated to a uniform interevent time. The controller, at each sampling time instant, checks to which region does the current state belong, and correspondingly...

Learning state representation for deep actor-critic control

Deep Neural Networks (DNNs) can be used as function approximators in Reinforcement Learning (RL). One advantage of DNNs is that they can cope with large input dimensions. Instead of relying on feature engineering to lower the input dimension, DNNs can extract the features from raw observations. The drawback of this end-to-end learning is that it...

Improved Approximation Algorithm for k-level Uncapacitated Facility Location Problem (with Penalties)

We study the k-level uncapacitated facility location problem (k-level UFL) in which clients need to be connected with paths crossing open facilities of k types (levels). In this paper we first propose an approximation algorithm that for any constant k, in polynomial time, delivers solutions of cost at most α_k times OPT, where α<sub...