This work extends and compares some recent model+learning-based methodologies for optimal control with input saturation. We focus on two methodologies: a model-based actor-critic (MBAC) strategy, and a nonlinear policy iteration strategy. To evaluate the performance of the algorithms, these strategies are applied to the swinging up an inverted pendulum. Numerical simulations show that the neural network approximation in the MBAC strategy can be poor, and the algorithm may converge far from the optimum. In the MBAC approach neither stabilization nor monotonic convergence can be guaranteed, and it is observed that the best value function is not always corresponding to the last one. On the other side the nonlinear policy iteration approach guarantees that every new control policy is stabilizing and generally leads to a monotonically decreasing cost.

Using a setting in which the input is communicated among neighbors (without exchanging any distributed observer variables), the problem of synchronizing an acyclic network of linear uncertain agents has been formulated recently as a distributed model reference adaptive control (MRAC) where each agent tries to converge to the model defined by its neighbors. In this work we show how to parametrize the distributed MRAC in cyclic and undirected graphs.

This paper establishes a novel adaptive hierarchical formation control method for uncertain heterogeneous nonlinear agents described by Euler–Lagrange (EL) dynamics. Formation control is framed as a synchronization problem where a distributed model reference adaptive control is used to synchronize the EL systems. The idea behind the proposed adaptive formation algorithm is that each agent must converge to the model defined by its hierarchically superior neighbors. Using a distributed inverse dynamics structure, we prove that distributed nonlinear matching conditions between connected agents hold, so that matching gains exist to make the entire formation converge to same homogeneous dynamics: to compensate for the presence of uncertainties, estimation laws are devised for such matching gains, leading to adaptive synchronization. An appropriately designed distributed Lyapunov function is used to derive asymptotic convergence of the synchronization error. The effectiveness of the proposed methodology is supported by simulations of a formation of Unmanned Aerial Vehicles (UAVs).