Kinematic Synthesis using Reinforcement Learning

Abstract

Advanced tools such as machine learning are slowly finding their way into the modern scientist’s toolbox . In the design of mechanical systems however hardly any machine learning applications are being used. Research into the viability of such an application is therefore necessary.
We have performed such research, using a specific type of machine learning, known as reinforcement learning, for the synthesis of kinematic mechanisms. Reinforcement learning is an experience-based learning strategy which has proven particularly successful in learning to play games, like chess, blackjack or Go. In this research it is shown that the sequentially alternating nature of game-playing between actions and reward can also be observed in mechanism design by posing design challenges in a game-like format. We have used a decision- tree based mechanism representation developed by Lipson [1] to create such a game-like world in which mechanisms can be designed. To train an actor to navigate this game-like world both Monte Carlo and Temporal Di erence learning have been applied, in combination with a neural network as nonlinear value function approximator. Moreover a kinematic simulator and scoring modules have been implemented to evaluate synthesized mechanisms.
We demonstrated the successful implementation of the framework and learning algorithm by synthesizing mechanism for two separate path tracing objectives: straight lines and figure- eights. A set of recommended algorithm settings was extracted from a parameter sweep and grid search including a total of 560 test runs. Straight line mechanisms were obtained with a fixed maximum number of 10 nodes, drawing lines with aspect ratios up to 1:1168. Additionally a mechanism was synthesized capable of drawing figure-eight patterns.
We conclude that the use of reinforcement learning in the context of mechanical system design is viable. More specifically using the presented method kinematic synthesis for path tracing objectives can be performed. The current research cultivates the land for future e orts to bridge the gap between the challenges faced by mechanical system design groups and the advancing solutions developed by the computer sciences.