The advancement and development of human modeling have greatly benefited from principles used in robotics, for instance, multibody dynamics laid the foundations for physics engines of human movement simulation, and the robotics and control theory were used to contextualize human sensorimotor control. There are many common interests and interconnections between the fields of human modeling and robotics. In recent years, as robots have become safer and smarter, they actively participate in our lives and help us in various scenarios. Roboticists need tools and data from human modeling to build next-generation robots that better assist humans. In this survey, we focus on the connections between physical human-robot interaction and human modeling. On one hand, human neuromusculoskeletal and sensorimotor control models provide novel insights into the human response that robots can utilize to improve human performance. On the other hand, robots are becoming instrumental in quantifying the performance of the (neuro)musculoskeletal system. Thus, the combined use of human modeling and robotic methods in physical human-robot interaction can lead to both improved human understanding and functional assistance.

Today’s humanoid robots are complex mechanical systems with many degrees of freedom that are built to achieve locomotion skills comparable to humans. In order to synthesize whole-body motions, real-tme capable direct methods of optimal control are a subject of contemporary research. To this end, Nonlinear Model Predictive Control is the method of choice to realize motions on the physical robot using model-based optimal control. However, the complexity of the problem results in a high computational time that falls short of the expectations of robotic experimenters and control engineers. In this article, we show how advanced NMPC methods can be applied to improve the control rate by a factor of 10–16 up to 190Hz. This is achieved by thread-based parallelization of two controllers and by efficiently reusing control problem linearizations of the last iteration to provide fast feedback by one controller while the other controller prepares the next nonlinear step including the evaluation of the multi-body dynamics and the respective sensitivities. This way, the bottleneck of the roll-out of up to 130 ms can partly be side-stepped by repeated calls of the much faster feedback phase of ~5ms. This enables a realization of a squatting task on the actual 2D-robot Leo of Delft University of Technology, which was not possible using a conventional Nonlinear Model Predictive Control scheme.

Model-free reinforcement learning and nonlinear model predictive control are two different approaches for controlling a dynamic system in an optimal way according to a prescribed cost function. Reinforcement learning acquires a control policy through exploratory interaction with the system, while nonlinear model predictive control exploits an explicitly given mathematical model of the system. In this article, we provide a comprehensive comparison of the performance of reinforcement learning and nonlinear model predictive control for an ideal system as well as for a system with parametric and structural uncertainties. The comparison is based on two different criteria, namely the similarity of trajectories and the resulting rewards. The evaluation of both methods is performed on a standard benchmark problem: a cart–pendulum swing-up and balance task. We first find suitable mathematical formulations and discuss the effect of the differences in the problem formulations. Then, we investigate the robustness of reinforcement learning and nonlinear model predictive control against uncertainties. The results demonstrate that nonlinear model predictive control has advantages over reinforcement learning if uncertainties can be eliminated through identification of the system parameters. Otherwise, there exists a break-even point after which model-free reinforcement learning performs better than nonlinear model predictive control with an inaccurate model. These findings suggest that benefits can be obtained by combining these methods for real systems being subject to such uncertainties. In the future, we plan to develop a hybrid controller and evaluate its performance on a real seven-degree-of-freedom walking robot.

This chapter reviews different methods for the control of legged locomotion with a special focus on bipedal locomotion. All locomotion systems are governed by complex nonlinear, hybrid dynamics, and are redundant, underactuated and often unstable, which makes their control a very challenging task.The chapter starts with a presentation of different concepts of stability and robustness of locomotion considering nominal walking situations as well as the reaction to larger external perturbations. Then, optimal control is discussed as a guiding principle of human and robot motion, and dynamic multibody system models as well as different optimization problem formulations for the generation, control and analysis of locomotion are shown. Constant or variable compliance plays an important role in biological and bio-inspired locomotion, but needs to be properly adapted in the design and control process which also can be addressed by optimal control. Next, impedance control in locomotion is discussed, looking at passive and active impedance and different approaches to emulated appropriate impedances for robots. The chapter also reviews control approaches for legged locomotion based on template models, i.e. very simple representations of the original locomotor system, with a focus using template models for the design of suitable controllers. The state of the art of passive dynamic walking robots as well as powered and almost passive dynamic robots is summarized and their achievements in terms of energy-efficiency, stability, robustness and versatility re discussed. Hybrid zero dynamics is presented as a control synthesis framework that reduces the complexity of whole-body dynamics control and allows to develop efficient controllers for dynamic walking and running motions. Finally, a control approach for locomotion based on the concept of central pattern generators is presented which helps to control locomotion of legged robots and gives insight into human movement control.