Over the last couple of decades the demand for high precision and enhanced performance of physical systems has been steadily increasing. This demand often results in miniaturization and complex design, thus increasing the need for complex nonlinear control methods. Some of the state of the art nonlinear methods are stymied by the requirement of full state information, model and parameter uncertainties, mathematical complexity, etc. For many scenarios it is nearly impossible to consider all the uncertainties during the design of a feedback controller. Additionally, while designing a modelbased nonlinear control there is no standard mechanism to incorporate performance measures. Some of the mentioned issues can be addressed by using online learning. Animals and humans have the ability to share, explore, act or respond, memorize the outcome and repeat the task to achieve a better outcome when they encounter the same or a similar scenario. This is called learning from interaction. One instance of this approach is reinforcement learning (RL). However, RL methods are hindered by the curse of dimensionality, non-interpretability and non-monotonic convergence of the learning algorithms. This can be attributed to the intrinsic characteristics of RL, as it is a modelfree approach and hence no standard mechanism exists to incorporate à priori model information. In this thesis, learning methods are proposed which explicitly use the available system knowledge. This can be seen as a new class of approaches that bridge model-based and model-free methods. These methods can address some of the hurdles mentioned earlier. For example, i) a prior system information can speed up the learning, ii) new control objectives can be achieved which otherwise would be extremely difficult to attain using only model-based methods, iii) physical meaning can be attributed to the learned controller. The developed approach is as follows: themodel of the given physical system is represented in the port-Hamiltonian (PH) form. For the system dynamics in PH form a passivity-based control (PBC) law is formulated, which often requires the solution to a set of partial differential equations (PDEs). Instead of finding an analytical solution, the PBC control law is parameterized using an unknown parameter vector. Then, by using a variation of the standard actor-critic learning algorithm, the unknown parameters can be learned online. Using the principles of stochastic approximation theory, a proof of convergence for the developed method is shown. The proposedmethods are evaluated for the stabilization and regulation ofmechanical and electro-mechanical systems. The simulation and experimental results show comparable learning curves. In the final part of the thesis a novel integral reinforcement learning approach is developed to solve for the optimal output tracking control problem for a set of linear heterogeneous multi-agent systems. Unlike existing methods, this approach does not need to solve either the output regulator equation or requires a p-copy of the leader’s dynamics in the agent’s control law. A detailed numerical evaluation has been conducted to show the feasibility of the developed method.@en

In this thesis, a swarm is defined as: ”A swarm is a large number of homogenous, unsophisticated agents that interact locally among themselves and their environment, without any central control or management to yield a global behaviour to emerge.” The bigger perspective is to have the units within a swarm operate for a potentially infinite amount of time, thus making algorithms and energy use as effective as possible. The research is split up in different sections: sensors, communication and behaviour. The behaviour is split again: individual behaviour, anti collision behaviour and swarming behaviour. The research on the sensors and communication points out that the Bluetooth Low Energy (BLE) sensor is the most suitable for an autonomous Zebro Swarm. BLE can be used to transfer data as well as serve as a range sensor. As a communication device, BLE offers the ability to advertise data an has a range of 50 meters which is sufficient for application in a swarm. As a sensor, BLE offers fairly accurate measurements in distances below 1 meter. Due to the properties of radio signal strength and the always present radio frequency noise, the accuracy decreased with increasing distances. The individual behaviour aims at exploring as much area as possible, with the amount of energy available. The energy use therefore has to be minimised and the explored area maximised. Since little is known on the energy use, this is assumed constant. Maximising area, however, has some key elements. The first is to keep moving; standing still results in energy use without exploring new terrain. The second is to make only gentle turns; the rotational centre must be outside of the detection distance. When this centre lies inside of the detection distance, the inner turn will cover area for a longer period making the turn ineffective. The anti collision behaviour should prevent units from making physical contact. By using a contactless sensor an approaching collision can be detected. Changing the parameters of the movement of the units at the moment they enclose more than a certain value prevents the collision. The anti separation behaviour should prevent units from separating from eachother. This can be done by making units turn 180 degrees once the relative distance reaches a certain limit. The turning on a certain limit is a simple algorithm and thus requires little processing power. Apart from the anti collision behaviour and the anti separation behaviour, there is another approach to deal with these behaviours. Since the units are able to communicate with each other, they can also advertise their location to other units. There are several methods for a unit to acquire their location. These methods are divided in approximate and exact location estimation methods. The Trial and Error method is placed in the first category, the exact location estimation methods are: Time Based, Internal Reference, External Reference and Radar. The External Reference method is considered the most applicable of these methods since it is relatively accurate, simple and provides a direct location.

Sequential composition is an effective supervisory control method for addressing control problems in nonlinear dynamical systems. It executes a set of controllers sequentially to achieve a control specification that cannot be realized by a single controller. Sequential composition focuses on the interaction between a collection of pre-designed controllers, where each of them is associated with a domain of attraction (DoA) and a goal set. By design, if the goal set of one controller lies in the DoA of another controller (this is called the prepare relation), the supervisor can instantly switch from the first controller to the second without affecting the stability of the system. As these controllers are designed offline, sequential composition cannot address unmodeled situations that might occur during runtime. Moreover, sequential composition has not been developed for cooperative settings where the collaboration of multiple systems is required in order to fulfill the control specifications. This thesis studies automatic synthesis of supervisory control systems using the framework of sequential composition. First, a learning sequential composition control algorithm is developed so as to learn new controllers on demand, by means of reinforcement learning. Once learning is completed, the supervisory control structure is augmented with the learned controllers. As a consequence, the supervisor is able to cope with unforeseen situations for which new controllers are required. Second, a cooperative sequential composition control algorithm is proposed to enable the coordination between a set of sequential composition controllers, without any change in their low-level structures. Finally, a robot contact language is designed for the manipulation of multiple objects by multiple robots.@en