Recent control algorithms for Markov decision processes (MDPs) have been designed using an implicit analogy with well-established optimization algorithms. In this article, we adopt the quasi-Newton method (QNM) from convex optimization to introduce a novel control algorithm coined as quasi-policy iteration (QPI). In particular, QPI is based on a novel approximation of the 'Hessian' matrix in the policy iteration algorithm, which exploits two linear structural constraints specific to MDPs and allows for the incorporation of prior information on the transition probability kernel. While the proposed algorithm has the same computational complexity as value iteration, it exhibits an empirical convergence behavior similar to that of QNM with a low sensitivity to the discount factor.

This brief presents a mathematical framework for modeling the dynamic effects of three fault categories and six fault variants in the ink channels of high-end industrial printers. It also introduces a hybrid approach that combines model-based and data-based methods to detect and isolate these faults effectively. A key challenge in these systems is that the same piezo device is used for actuation (generating ink droplets) and for sensing, and, as a consequence, sensing is only available when there is no actuation. The proposed fault detection (FD) filter, based on the healthy model, uses the piezo self-sensing signal to generate a residual, while taking the above challenge into account. The system is flagged as faulty if the residual energy exceeds a threshold. Fault isolation (FI) is achieved through linear regression (LR) or a k-nearest neighbors (KNN) approach to identify the most likely fault category and variant. The resulting hybrid fault detection and isolation (FDI) method overcomes traditional limitations of model-based methods by isolating different types of faults affecting the same entries (i.e., equations) in the ink channel dynamics. Moreover, it is shown to outperform purely data-driven methods in FI, especially when data is scarce. Experimental validation demonstrates superior FDI performance compared to state-of-the-art methods.

Inspired by the recent successes of Inverse Optimization (IO) across various application domains, we propose a novel offline Reinforcement Learning (ORL) algorithm for continuous state and action spaces, leveraging the convex loss function called “sub-optimality loss” from the IO literature. To mitigate the distribution shift commonly observed in ORL problems, we further employ a robust and non-causal Model Predictive Control (MPC) expert steering a nominal model of the dynamics using in-hindsight information stemming from the model mismatch. Unlike the existing literature, our robust MPC expert enjoys an exact and tractable convex reformulation. In the second part of this study, we show that the IO hypothesis class, trained by the proposed convex loss function, enjoys ample expressiveness and reliably recovers teacher behavior in MuJoCo benchmarks. The method achieves competitive results compared to widely-used baselines in sample-constrained settings, despite using orders of magnitude fewer parameters. To facilitate the reproducibility of our results, we provide an open-source package implementing the proposed algorithms and the experiments.

To improve the predictive capacity of system models in the input–output sense, this paper presents a framework for model updating via learning of modeling uncertainties in locally and globally Lipschitz nonlinear systems. First, we introduce a method to extend an existing known model with an uncertainty model so that stability of the extended model is guaranteed in the sense of set invariance and input-to-state stability. To achieve this, we provide two tractable semi-definite programs. These programs allow obtaining optimal uncertainty model parameters for both locally and globally Lipschitz nonlinear models, given uncertainty and state trajectories. Subsequently, in order to extract this data from the available input–output trajectories, we introduce a filter that incorporates an approximated internal model of the uncertainty and asymptotically estimates uncertainty and state realizations. This filter is also synthesized using semi-definite programs with guaranteed robustness with respect to uncertainty model mismatches, disturbances, and noise. Numerical simulations for a large data-set of a roll plane model of a vehicle illustrate the effectiveness and practicality of the proposed methodology in improving model accuracy, while guaranteeing stability.

Assuming access to prior knowledge about the unknown system parameter θ, given as a probability prior ϕ(θ), we study how such information can be incorporated into the adaptive control design of nonlinear systems. To this end, we propose a new parameter estimation law, and show that it pro-motes convergence of parameter estimates to high-probability regions of the parameter space. Our approach is particularly useful when employed as a single universal controller for a large population of systems with similar dynamic structure but different parameter values. In this scenario, our method, on average, leads to a more accurate parameter estimate, which consequently improves transient performance and accelerates state convergence. Numerical studies confirm the effectiveness of the proposed method.

This survey reviews recent developments in fault diagnosis for both linear and nonlinear dynamical systems, covering model-based and data-driven approaches as well as passive and active detection and estimation methods. A central focus is placed on the geometric interpretation of diagnosis filters and their connection to the concept of behavioral sets, providing an intuitive view of their performance. We also review optimization-based techniques that enhance the robustness of linear filters when applied to nonlinear or uncertain systems. Furthermore, we point out recent progress in active fault diagnosis, where input design plays a key role in improving detectability and estimation accuracy. To bridge theory and practice, we include a set of real-world industrial applications that demonstrate the implementation and effectiveness of these methods in realistic settings.

To increase system robustness and autonomy, in this article, we propose a nonlinear fault estimation filter for a class of linear dynamical systems, subject to structured uncertainty, measurement noise, and system delays, in the presence of additive and multiplicative faults. The proposed filter architecture combines tools from model-based control approaches, regression techniques, and convex optimization. The proposed method estimates the additive and multiplicative faults using a linear residual generator combined with nonlinear regression. An offline simulator allows us to numerically characterize the mismatch between an assumed linear model and a range of alternative linear models that exhibit different levels of structured uncertainty. Moreover, we show how the performance bounds of the estimator, valid in the absence of uncertainty, can be used to determine appropriate countermeasures for measurement noise. In the scope of this work, we focus particularly on a fault estimation problem for Society of Automotive Engineers (SAEs) level 4 automated vehicles, which must remain operational in various cases and cannot rely on the driver. The proposed approach is demonstrated in simulations and in an experimental setting, where it is shown that additive and multiplicative faults can be estimated in a real vehicle under the influence of model uncertainty, measurement noise, and delay.

We propose a data-driven, user-centric vehicle-to-grid (V2G) methodology based on multi-objective optimization to balance battery degradation and V2G revenue according to EV user preference. Given the lack of accurate and generalizable battery degradation models, we leverage input convex neural networks (ICNNs) to develop a data-driven degradation model trained on extensive experimental datasets. This approach enables our model to capture nonconvex dependencies on battery temperature and time while maintaining convexity with respect to the charging rate. Such a partial convexity property ensures that the second stage of our methodology remains computationally efficient. In the second stage, we integrate our data-driven degradation model into a multi-objective optimization framework to generate an optimal smart charging profile for each EV. This profile effectively balances the trade-off between financial benefits from V2G participation and battery degradation, controlled by a hyperparameter reflecting the user prioritization of battery health. Numerical simulations show the high accuracy of the ICNN model in predicting battery degradation for unseen data. Finally, we present a trade-off curve illustrating financial benefits from V2G versus losses from battery health degradation based on user preferences and showcase smart charging strategies under realistic scenarios.

We consider the constrained Linear Inverse Problem (LIP), where a certain atomic norm (like the `₁ norm) is minimized subject to a quadratic constraint. Typically, such cost functions are non-differentiable which makes them not amenable to the fast optimization methods existing in practice. We propose two equivalent reformulations of the constrained LIP with improved convex regularity: (i) a smooth convex minimization problem, and (ii) a strongly convex min-max problem. These problems could be solved by applying existing acceleration-based convex optimization methods which provide better Op¹{k²q theoretical convergence guarantee, improving upon the current best rate of Op¹{kq. We also provide a novel algorithm named the Fast Linear Inverse Problem Solver (FLIPS), which is tailored to maximally exploit the structure of the reformulations. We demonstrate the performance of FLIPS on the classical problems of Binary Selection, Compressed Sensing, and Image Denoising. We also provide open source MATLAB and PYTHON package for these three examples, which can be easily adapted to other LIPs.

This paper studies the problem of fault detection and estimation (FDE) for linear time-invariant (LTI) systems with a particular focus on frequency content information of faults, possibly as multiple disjoint continuum ranges, and under both disturbances and stochastic noise. To ensure the worst-case fault sensitivity in the considered frequency ranges and mitigate the effects of disturbances and noise, an optimization framework incorporating a mixed H_{_}/H₂ performance index is developed to compute the optimal detection filter. Moreover, a thresholding rule is proposed to guarantee both the false alarm rate (FAR) and the fault detection rate (FDR). Next, shifting attention to fault estimation in specific frequency ranges, an exact reformulation of the optimal estimation filter design using the restricted H_∞ performance index is derived, which is inherently non-convex. However, focusing on finite frequency samples and fixed poles, a lower bound is established via a highly tractable quadratic programming (QP) problem. This lower bound together with an alternating optimization (AO) approach to the original estimation problem leads to a suboptimality gap for the overall estimation filter design. The effectiveness of the proposed approaches is validated through applications of a non-minimum phase hydraulic turbine system and a multi-area power system.

We establish a collection of closed-loop guarantees and propose a scalable optimization algorithm for distributionally robust model predictive control (DRMPC) applied to linear systems, convex constraints, and quadratic costs. Via standard assumptions for the terminal cost and constraint, we establish distributionally robust long-term and stagewise performance guarantees for the closed-loop system. We further demonstrate that a common choice of the terminal cost, i.e., via the discrete-algebraic Riccati equation, renders the origin input-to-state stable for the closed-loop system. This choice also ensures that the exact long-term performance of the closed-loop system is independent of the choice of ambiguity set for the DRMPC formulation. Thus, we establish conditions under which DRMPC does not provide a long-term performance benefit relative to stochastic MPC. To solve the DRMPC optimization problem, we propose a Newton-type algorithm that empirically achieves superlinear convergence and guarantees the feasibility of each iterate. We demonstrate the implications of the closed-loop guarantees and the scalability of the proposed algorithm via two examples. To facilitate the reproducibility of the results, we also provide open-source code to implement the proposed algorithm and generate the figures.