<p>Soft robotics is a rapidly growing field of study still in the nascent stage.&nbsp;As opposed to&nbsp;conventional robots comprised of ridged bodies, soft robots are made of soft and extensible materials&nbsp;like polymers.&nbsp;These robots accomplish tasks via shape changing by use of pneumatic or fluidic actuation, serving a diverse range of potential applications that are often outside of the possibilities of conventional robots. For example, soft robots capable of locomotion have been shown to operate in unstructured environments with little to no sensory feedback. Due to soft robots’ complex dynamic behavior inherent from their soft material processors (e.g., nonlinear and nonhomogeneous elasticity and viscosity), the traditional physics-based modeling methods and,&nbsp;subsequently,&nbsp;model-based motion control methods cannot be directly applied. In addition, soft robots are continuum robots, that is they have infinitely many degrees of freedom thus, they can be described as uncontrollable and unobservable systems. This paper proposed novel motion control of a pneumatically actuated quadruped soft robot via reinforcement learning (RL) where the control policy modeled after a central pattern generator (CPG). RL is a subset of machine learning where an agent seeks to find the most advantageous actions by maximizing a cumulative reward. In biology a CPG, which is a cluster of nerves in the spinal cord that controls rhythmic behavior like breathing, walking, or swimming with little to no input signals from the central nervous system. Engineers and scientists have developed CPG based controllers which model repetitive behavior such as walking which have been extensively used in classic robots, and recently have garnered attention in soft robots due to (1) CPGs are model-free algorithms, (2) they can describe complex oscillatory motions using a limited number of parameters, and (3) these algorithms do not require sensory feedback. A learning algorithm called Parameter Exploring Policy Gradient (PEPG) searches the parameter to find the optimal policy. The PEPG algorithm was chosen because it (1) allows learning in continuous, partially observable environments, (2) it is model free, and (3) the parameter space is sampled symmetrically. The PEPG algorithm operates as follows, first a perturbation is randomly selected and a then a symmetric parameter is sampled. The reward function is evaluated with the sampled parameters. Lastly, the standard deviation of which the perturbations are sampled from and the set of parameters are updated every epoch. By choosing random location, to which the robot must walk to, the PEPG find four distinct, optimal gaits: forward motion, backward motion, left turn, and right turn. Lastly, a PID controller is designed to control the transition between gaits for trajectory tracking.</p>

Reinforcement Learning Based Motion Control for a Quadruped Pneumatically Actuated Soft Robot

Paper Proceedings: https://asmedigitalcollection.asme.org/IMECE/IMECE2021/volume/85543

ASME 2021 International Mechanical Engineering Congress and Exposition (IMECE2021)

Supporting the need among researchers worldwide for additional online and multimedia content, ASME now offers the ASME Conference Video Collection. This platform will collect and provide long-term access to conference presentation videos, encompassing all technology areas within ASME's technical conference and events program. About 30,000-40,000 engineers across all mechanical engineering and related disciplines participate in ASME conferences each year. This includes authors of approximately 6,000 to 8,000 technical papers published by ASME, the majority of whom also submit video presentations for our virtual conferences. The video collection will be available as an add-on to existing product packages or as a separate subscription.