Model-plant mismatch compensation using reinforcement learning

Learning-based approaches are suitable for the control of systems with unknown dynamics. However, learning from scratch involves many trials with exploratory actions until a good control policy is discovered. Real robots usually cannot withstand the exploratory actions and suffer damage. This problem can be circumvented by combining learning...

Viewpoint optimization for aiding grasp synthesis algorithms using reinforcement learning

Grasp synthesis for unknown objects is a challenging problem as the algorithms are expected to cope with missing object shape information. This missing information is a function of the vision sensor viewpoint. The majority of the grasp synthesis algorithms in literature synthesize a grasp by using one single image of the target object and...

Active vision via extremum seeking for robots in unstructured environments: Applications in object recognition and manipulation

In this paper, a novel active vision strategy is proposed for optimizing the viewpoint of a robot's vision sensor for a given success criterion. The strategy is based on extremum seeking control (ESC), which introduces two main advantages: 1) Our approach is model free: It does not require an explicit objective function or any other task...

Evaluation of physical damage associated with action selection strategies in reinforcement learning

Reinforcement learning techniques enable robots to deal with their own dynamics and with unknown environments without using explicit models or preprogrammed behaviors. However, reinforcement learning relies on intrinsically risky exploration, which is often damaging for physical systems. In the case of the bipedal walking robot Leo, which is...

Benchmarking model-free and model-based optimal control

Model-free reinforcement learning and nonlinear model predictive control are two different approaches for controlling a dynamic system in an optimal way according to a prescribed cost function. Reinforcement learning acquires a control policy through exploratory interaction with the system, while nonlinear model predictive control exploits an...

Accelerating reinforcement learning on a robot by using subgoals in a hierarchical framework

Reinforcement learning is a way to learn control tasks by trial and error. Even for simple motor control tasks, however, this can take a long time. We can speed up learning by using prior knowledge, but this is not always available, especially for an autonomous agent. One way to add limited prior knowledge is to use subgoals, defining points...

The Effects of Large Disturbances on On-Line Reinforcement Learning for aWalking Robot

Reinforcement Learning is a promising paradigm for adding learning capabilities to humanoid robots. One of the difficulties of the real world is the presence of disturbances. In Reinforcement Learning, disturbances are typically dealt with stochastically. However, large and infrequent disturbances do not fit well in this framework; essentially,...

Automated Design of Application-Specific Smart Camera Architectures

Parallel heterogeneous multiprocessor systems are often shunned in embedded system design, not only because of their design complexity but because of the programming burden. Programs for such systems are architecture-dependent: the application developer needs architecture-specific knowledge to implement his algorithms, as each processor has its...

Biological applications of an LCoS-BASED PROGRAMMABLE ARRAY MICROSCOPE (PAM)

We report on a new generation, commercial prototype of a programmable array optical sectioning fluorescence microscope (PAM) for rapid, light efficient 3D imaging of living specimens. The stand-alone module, including light source(s) and detector(s), features an innovative optical design and a ferroelectric liquid-crystal-on-silicon (LCoS)...