When estimating a single subsystem (module) in a linear dynamic network with a prediction error method, a data-informativity condition needs to be satisfied for arriving at a consistent module estimate. This concerns a condition on input signals in the constructed, possibly MIMO (multiple input multiple output) predictor model being persistently exciting, which is typically guaranteed if the input spectrum is positive definite for a sufficient number of frequencies. Generically, the condition can be formulated as a path-based condition on the graph of the network model. The current condition has two elements of possible conservatism: (a) rather than focussing on the full MIMO model, one would like to be able to focus on consistently estimating the target module only, and (b) structural information, such as structural zero elements in the interconnection structure or known subsystems, should be taken into account. In this paper relaxed conditions for data-informativity are derived addressing these two issues, leading to relaxed path-based conditions on the network graph. This leads to experimental conditions that are less strict, i.e. require a smaller number of external excitation signals. Additionally, the new expressions for data-informativity in identification are shown to be closely related to earlier derived conditions for (generic) single module identifiability.

Learning-based control with safety guarantees usually requires real-time safety certification and modifications of possibly unsafe learning-based policies. The control barrier function (CBF) method uses a safety filter (SF) containing a constrained optimization problem to produce safe policies. However, finding a valid CBF for a general nonlinear system requires a complex function parameterization, which in general makes the policy optimization problem difficult to solve in real time. For nonlinear systems with nonlinear state constraints, this paper proposes the novel concept of state-action CBFs (SACBFs), which do not only characterize the safety at each state but also evaluate the control inputs taken at each state. SACBFs, in contrast to CBFs, enable a flexible parameterization, resulting in a SF that involves a convex quadratic optimization problem, which significantly alleviates the online computational burden. We propose a learning-based approach to synthesize SACBFs. The effect of learning errors on the effectiveness of SACBFs is addressed by constraint tightening and introducing a new concept called contractive-set CBFs. This ensures formal safety guarantees for the learned CBFs and control policies. Simulation results on an inverted pendulum with elastic walls validate the proposed CBFs in terms of constraint satisfaction and CPU time.

Handling model mismatch is a common challenge in model predictive control (MPC). While robust MPC is effective, its conservatism often makes it less desirable. Certainty-equivalence MPC (CE-MPC), which uses a nominal model, offers an appealing alternative due to its design simplicity and low computational costs. This paper investigates CE-MPC for uncertain nonlinear systems with multiplicative parametric uncertainty and input constraints that are inactive at the steady state. The primary contributions are two-fold. First, a novel perturbation analysis of the MPC value function is provided, without assuming the Lipschitz continuity of the stage cost, better tailoring the widely used quadratic cost and having broader applicability in value function approximation, learning-based MPC, and performance-driven MPC design. Second, the stability and performance analysis of CE-MPC are provided, quantifying the suboptimality of CE-MPC compared to the infinite-horizon optimal controller with perfect model knowledge. The results provide insights in how the prediction horizon and model mismatch jointly affect stability and the worst-case performance. Furthermore, the general results are specialized to linear quadratic control, and a competitive ratio bound is derived, serving as the first competitive-ratio bound for MPC of uncertain linear systems with input constraints and multiplicative uncertainty.

This work analyzes how the trade-off between the modeling error, the terminal value function error, and the prediction horizon affects the performance of a nominal receding-horizon linear quadratic (LQ) controller. By developing a novel perturbation result of the Riccati difference equation, a novel performance upper bound is obtained and suggests that for many cases, the prediction horizon can be either 1 or +∞ to improve the control performance, depending on the relative difference between the modeling error and the terminal value function error. The result also shows that when an infinite horizon is desired, a finite prediction horizon that is larger than the controllability index can be sufficient for achieving a near-optimal performance, revealing a close relation between the prediction horizon and controllability. The obtained suboptimality performance upper bound is applied to provide novel sample complexity and regret guarantees for nominal receding-horizon LQ controllers in a learning-based setting. We show that an adaptive prediction horizon that increases as a logarithmic function of time is beneficial for regret minimization.

Implementing model predictive control (MPC) in practice faces many subtle but prevalent problems, including modeling errors, solver errors, and actuator faults. In essence, the real control input applied to the system always deviates from the ideal one based on a perfect controller, resulting in an imperfect controller. In this letter, we provide a general analysis to quantify the suboptimality of MPC for Lipschitz-continuous nonlinear systems due to imperfect control inputs in terms of dynamic regret. Based on a general assumption about how the imperfect controller may improve over time, sublinear regret upper bounds are established for cases where the closed-loop system under the ideal controller is Lipschitz-contractive (i.e., its Lipschitz constant is smaller than one). In addition, we also discuss how the regret scales when the closed-loop system under the oracle controller is not Lipschitz-contractive. The results provide insights into designing suitable MPC strategies, especially for learning-based MPC.

Learning-based control, in particularReinforcement Learning (RL) reinforcementReinforcement learning, and optimization-based control, in particular model predictive control, each have their advantages and disadvantages for online, real-timeOptimal control optimal controlOptimal control of systems with complex dynamicsDynamic. However, both approaches are highly complementary and therefore there is an increased interest in combining their advantages in an integrated approach. In this chapter, we provide an overview of recent results, challenges, and opportunities on an integrated learning-based and optimization-based control approach. We focus in particular on piecewise affine systems as they are an extension of linear systemsLinear systems that can model or approximate hybridHybrid or nonlinearNonlinearbehaviorBehavior and as they still allow for effective numerical solutionSolution approaches.

Infinite-horizon optimal control of constrained piecewise affine (PWA) systems has been approximately addressed by hybrid model predictive control (MPC), which, however, has computational limitations, both in offline design and online implementation. In this article, we consider an alternative approach based on approximate dynamic programming (ADP), an important class of methods in reinforcement learning. We accommodate nonconvex union-of-polyhedra state constraints and linear input constraints into ADP by designing PWA penalty functions. PWA function approximation is used, which allows for a mixed-integer encoding to implement ADP. The main advantage of the proposed ADP method is its online computational efficiency. Particularly, we propose two control policies, which lead to solving a smaller-scale mixed-integer linear program than conventional hybrid MPC, or a single convex quadratic program, depending on whether the policy is implicitly determined online or explicitly computed offline. We characterize the stability and safety properties of the closed-loop systems, as well as the suboptimality of the proposed policies, by quantifying the approximation errors of value functions and policies. We also develop an offline mixed-integer-linear-programming-based method to certify the reliability of the proposed method. Simulation results on an inverted pendulum with elastic walls and on an adaptive cruise control problem validate the control performance in terms of constraint satisfaction and CPU time.

Approximate dynamic programming (ADP) faces challenges in dealing with constraints in control problems. Model predictive control (MPC) is, in comparison, well-known for its accommodation of constraints and stability guarantees, although its computation is sometimes prohibitive. This paper introduces an approach combining the two methodologies to overcome their individual limitations. The predictive control law for constrained linear quadratic regulation (CLQR) problems has been proven to be piecewise affine (PWA) while the value function is piecewise quadratic. We exploit these formal results from MPC to design an ADP method for CLQR problems with a known model. A novel convex and piecewise quadratic neural network with a local–global architecture is proposed to provide an accurate approximation of the value function, which is used as the cost-to-go function in the online dynamic programming problem. An efficient decomposition algorithm is developed to generate the control policy and speed up the online computation. Rigorous stability analysis of the closed-loop system is conducted for the proposed control scheme under the condition that a good approximation of the value function is achieved. Comparative simulations are carried out to demonstrate the potential of the proposed method in terms of online computation and optimality.

In this paper, we analyze the regret incurred by a computationally efficient exploration strategy, known as naive exploration, for controlling unknown partially observable systems within the Linear Quadratic Gaussian (LQG) framework. We introduce a two-phase control algorithm called LQG-NAIVE, which involves an initial phase of injecting Gaussian input signals to obtain a system model, followed by a second phase of an interplay between naive exploration and control in an episodic fashion. We show that LQG-NAIVE achieves a regret growth rate of Õ(√T), i.e., O(√T) up to logarithmic factors after T time steps, and we validate its performance through numerical simulations. Additionally, we propose LQG-IF2E, which extends the exploration signal to a 'closed-loop' setting by incorporating the Fisher Information Matrix (FIM). We provide compelling numerical evidence of the competitive performance of LQG-IF2E compared to LQG-NAIVE.

Networks of dynamical systems play an important role in various domains and have motivated many studies on the control and analysis of linear dynamical networks. For linear network models considered in these studies, it is typically pre-determined what signal channels are inputs and what are outputs. These models do not capture the practical need to incorporate different experimental situations, where different selections of input and output channels are applied to the same network. Moreover, a unified view of different network models is lacking. This work makes an initial step towards addressing the above issues by taking a behavioral perspective, where input and output channels are not pre-determined. The focus of this work is on behavioral network models with only external variables. By exploiting the concept of hypergraphs, novel dual graphical representations, called system graphs and signal graphs, are introduced for behavioral networks. Moreover, connections between behavioral network models and structural vector autoregressive models are established. In addition to their connections in graphical representations, it is shown that the regularity of interconnections is an essential assumption when choosing a structural vector autoregressive model.

Identifiability of a single module in a network of transfer functions is determined by whether a particular transfer function in the network can be uniquely distinguished within a network model set, on the basis of data. Whereas previous research has focused on the situations that all network signals are either excited or measured, we develop generalized analysis results for the situation of partial measurement and partial excitation. As identifiability conditions typically require a sufficient number of external excitation signals, this article introduces a novel network model structure such that excitation from unmeasured noise signals is included, which leads to less conservative identifiability conditions than relying on measured excitation signals only. More importantly, graphical conditions are developed to verify global and generic identifiability of a single module based on the topology of the dynamic network. Depending on whether the input or the output of the module can be measured, we present four identifiability conditions which cover all possible situations in single module identification. These conditions further lead to synthesis approaches for allocating excitation signals and selecting measured signals, to warrant single module identifiability. In addition, if the identifiability conditions are satisfied for a sufficient number of external excitation signals only, indirect identification methods are developed to provide a consistent estimate of the module. All the obtained results are also extended to identifiability of multiple modules in the network.

In a dynamic network of interconnected transfer functions, it is not necessary to use all the node signals for estimating a local transfer function. Given the network topology, detailed conditions are available for selecting inputs and outputs in a (MIMO) predictor model that warrants consistent and minimum variance estimation of a target module through the so-called local direct method. Motivated by the existing minimum-input signal selection approach that gradually incorporates additional signals, an alternative graphical algorithm for signal selection is developed in this work by directly exploiting the complete network graph. Then, as a straightforward application of existing analytical results, graphical conditions for consistent identification are derived for the novel signal selection approach. We show by an example that in some cases, for the consistent estimation of the target module, the developed method leads to fewer selected signals than the original minimum-input method.

For the identification of switched systems with measured states and a measured switching signal, this work aims to analyze the effect of switching strategies on the estimation error. The data is assumed to be collected from globally asymptotically or marginally stable switched systems under switches that are arbitrary or subject to an average dwell time constraint. Then the switched system is estimated by the least-squares (LS) estimator. To capture the effect of the parameters of the switching strategies on the LS estimation error, finite-sample error bounds are developed in this work. The obtained error bounds show that the estimation error is logarithmic of the switching parameters when there are only stable modes; however, when there are unstable modes, the estimation error bound can increase linearly as the switching parameter changes. This suggests that in the presence of unstable modes, the switching strategy should be properly designed to avoid the significant increase of the estimation error.

Identifiability of linear dynamic networks requires the presence of a sufficient number of external excitation signals. The problem of allocating a minimal number of external signals for guaranteeing generic network identifiability in the full measurement case has been recently addressed in the literature. Here we will extend that work by explicitly incorporating the situation that some network modules are known, and thus are fixed in the parametrized model set. The graphical approach introduced earlier is extended to this situation, showing that the presence of fixed modules reduces the required number of external signals. An algorithm is presented that allocates the external signals in a systematic fashion.