Soft actuators are often combined with rigid robot bodies or embedded in soft robots to control them. Soft actuators are mainly categorized as pneumatic actuators (SPAs) [7274], cable-driven actuators [75], electroactive polymers (EAPs) [76], and shape memory alloys (SMAs) [7779] based on their actuation methods. Fig 3 depicts the inputs and outputs of machine learning models used in soft actuators.

(A) Actuators, (B) Actuators in practical uses.

Due to the high degree of freedom of hyper-elastic materials [80], it is difficult to realize accurate proprioception or control of soft robots using soft actuators. To control them, high-dimensions of soft morphology should be actuated with less control inputs. In addition, time-varying material characteristics limit the dynamic modeling of soft actuators. In detail, the degradations of soft matter, i.e., creep, fatigue, and friction, known as critical factors of time-varying material characteristics, are often occurred, which limits the dynamic modeling of the soft actuators. For example, frictions between the cable and cable sheath in a cable-driven approach make cable tensions highly fluctuate, which in result increases the hysteresis of cable-driven actuators and shortens the lifetime [81]. At present, machine learning methods have been extensively applied to the modeling of soft actuators that have high degree of freedom and to the generating control strategies in order to deal with the aforementioned non-linearity issues.

This section introduces existing machine learning-based researches conducted on soft pneumatic actuators and cable-driven actuators, among other actuators such as EAPs and SMAs.

Soft pneumatic actuators (SPAs) have been extensively researched due to their flexible motions with simple morphological structures and versatility. To improve the functionality of SPAs, various sensors have been integrated for training data obtained from soft pneumatic actuators. Given that the solid-state sensors traditionally used in rigid robots may limit the flexible movements of SPAs, soft and flexible sensors have frequently been integrated to obtain contacts or bending motions of SPAs. In addition, the simple internal pressure sensor data of the SPA was used to improve the functionality of the soft gripper [82]. RNNs were employed for the SPAs, which were integrated with soft resistive sensors to obtain the contact forces and the bending motions [10]. Instead of embedding the sensors into the SPAs, a camera sensor was used to obtain the states of the actuators. To track the 3D trajectories of the SPA, an inverse model was also employed for training, as the application of the nonparametric and online learning of locally-weighted projection regression for endoscopy applications [83]. Jung et al. developed a proprioceptive sensing method of a soft pneumatic actuator based on the GP regression by incorporating with an extended Kalman filtering for state estimation and sliding mode control for the feedback control strategy [84].

Obtaining a kinematic or dynamic model of a soft robot has been a challenge in model-based control strategies. To overcome such limitation, learning algorithms have been applied to acquire the kinematic or dynamic model of soft robots based on SPAs [8588]. An FNN and radial basis function (RBF) neural networks were applied to the inverse or forward kinematic modeling of a soft continuum robot based on SPAs including 3-Dimensional motions [85, 86]. M. Gillespie et al. and P. Hyatt et al. proposed a predictive model based on the neural networks, and a learning method for the linearized discrete state space representation of soft robots [87, 88]. G. Fang et al. developed a learning method based on the local Gaussian Process Regression (GPR) to estimate the motion of SPAs using the kinematic model from the control inputs to the manipulator configurations based on the sequential camera images [89]. Instead of the inverse, or forward kinematic modeling, an asymmetric hysteresis of a pneumatic artificial muscle was modeled by integrating the Convolutional Neural Network and an existing extended up-parallel Prandtl-Ishlinskii model. J. Kim used a Gaussian Process Regression to learn control policy for a simple tripod mobile robot based on membrane vibration actuators [90].

M. Rolf et al. developed learning strategies to obtain an inverse model, which indicates the relationship between the target position and the required control inputs [91]. Instead of modeling the dynamics of a soft robot itself, hysteresis was also predicted for a pneumatic artificial muscle over a wide range of input by combining conventional hysteresis model and the CNN [92]. M. Wiese et al. studied hyperparameter optimizations to model SPAs using a simple FNN [93].

Another approach for controlling the pneumatically actuated soft robot is a model-free learning algorithm, which is a learning method to calculate the control policy without an analytical model. Reinforced learning algorithms such as Q-learning have been usually used for the model-free approaches [94]. In general, the objective of reinforcement learning is to find the control policy that maximizes the expected discount return, which is the weighted sum of rewards received by the agent for the system [95]. X. You et al. and S. Satheeshbabu et al. developed and implemented a multi-segment soft manipulator for planar motions using the Q-learning algorithm [96, 97]. J. Kim et al. used a model-free reinforcement learning algorithm to control a pneumatic actuated tripod mobile robot. They used an adaptive soft actor-critic (ASAC) algorithm and a reinforcement algorithm to obtain an accurate dynamic model of the robot [98].

Commercially available sensors, like depth cameras, film-based flex sensors, and potentiometers, are generally used to estimate the configurations of SPAs with machine learning techniques [85, 88, 89, 91]. On the other hand, as the traditional sensors can be relatively too rigid to be compatible with SPAs that are highly deformable, soft sensors have often been integrated with SPAs to estimate the configurations of soft robots. However, the non-linear behaviors of the soft sensors may cause delays when estimating states of soft robots. For example, T. Thuruthel et al. suggested SPAs integrated with a soft sensor using cPDMS (carbon-polydimethylsiloxane) and film-based flex sensors to estimate contact forces or configurations; they reported that the proposed learning-based model showed longer delays when using the soft sensors compared to the film-based flex sensors due to the high-dimensional deformability of soft sensors [10]. Based on this perspective, it would be an open issue for the future direction to develop soft sensors with fast responses. For instance, the development of three-dimensional printing-based fabrication of soft sensors would be a possible solution to estimate the configurations of SPAs in that it tends to have consistent and fast responses [99]. At the same time, machine learning algorithms need to be developed to overcome the nonlinear dynamic responses of soft sensors when integrated with SPAs. Although [92] showed potentials in that the large hysteresis of the SPAs could be reduced via machine learning algorithms, it was limited to simple linear motions of Pneumatic Artificial Muscles (PAMs). Thus, it is necessary to develop algorithms that deal with the non-linear and hysteresis behaviors of soft sensor-embedded systems with fast responses, as a future goal.

In cable-driven or tendon-driven soft robots, the actuators are situated outside of the robot structures; therefore, they do not interfere with movements of the soft bodies. Instead, cables connected to the actuators transmit the tensions through the cable paths or routings, which are embedded in a soft structure [100]. When it comes to controlling the soft robots, a major problem for the cable-driven mechanisms comes from non-linearity and hysteresis. These issues are mainly caused by high friction between a cable and cable path due to tension of the cable and the bending of the cable path [101].

A supervised learning-based method was proposed with respect to soft manipulators as a solution of the inverse statics problem to realize effective grasping. M. Giorelli et al. implemented an FNN for non-constant curvature manipulators to solve inverse kinematics [102, 103]. The performance of the FNN-based model was experimentally tested by comparing with model-based numerical approach and Jacobian-based method, which requires numerical resolution of integrals along the structure as proposed in [104, 105], for a conical soft manipulator driven by two cables. Based on the results, the FNN showed better performances and faster convergence than the model-based numerical method; however, FNN required model optimization and a bigger dataset [102].

Model-free control strategies based on RNNs were developed to learn the dynamics of robots. For a soft robot with friction-manipulation mechanisms driven by a motor-tendon combination that is capable of terrestrial locomotion, the model-free control framework was experimentally applied to the robot designs while changing the shape of tendon paths, friction mechanisms, and environmental conditions [100]. Nakajima et al. demonstrated a soft silicone arm system that can be employed to deal with the transient dynamics of the soft materials based on RNN and suggested its applicability to a real-world problem [106]. Ansari et al. conducted a study on a soft robot arm module actuated by tendon-based and pneumatic-based actuators for a bathing task for elderly people. Model-free control using reinforcement learning was developed to simultaneously increase the stiffness and positioning capacities [107]. Thuruthel et al. tested a tendon-driven soft manipulator under a simulated environment, in addition to a pneumatically-driven soft manipulator, using model-based reinforcement learning for closed-loop dynamic control. For the forward dynamic model, an RNN was used. Based on the learned dynamic models, a trajectory optimization was implemented to develop an open loop controller; however, the authors reported that the open loop controller is not robust against external disturbances [108]. To overcome this limitation, a model-based policy learning method for the closed-loop dynamic control of a soft robotic manipulator using an RNN was proposed. The representation of the policy architecture allows for the stability of the controller with respect to changes in the control frequency, sensory noise, and dynamics. With respect to the simulation of tendon-driven soft manipulators and experimental evaluations of under-actuated pneumatically-driven soft manipulators, sufficient accuracy levels were maintained, and the control frequency was decreased by a maximum factor of 5 [109].

Previously, machine learning in cable-driven or tendon-driven actuators of soft robots was focused on increasing the performances of position control. Rather than position control, soft manipulators and soft wearable robots with cable-driven actuators require end-effectors’ force or cable tension to generate proper contacting forces in accordance with various object characteristics. However, due to the non-linear characteristics of friction and fatigue with the cable and the cable path, degradations of the cable over time under various loads and situations is still limited in soft robotic systems. As future works, a real-time applicable learning method should be developed by collecting time sequence data of cable tension and the configuration of the soft robot to estimate the precise force control of the soft robots [101].

Ionic polymer-metal composite (IPMC) flexible actuators are generally composed of ion exchange polymer films, with electrodes on both sides, which have relatively low voltages (< 4 V), can generate large strains (> 40%) and are capable of sensing and actuating under harsh conditions [110]. However, the IPMC materials have time-varying performance changes and mechanical hysteresis as well as high maneuverability and agile capabilities, thus making it difficult to plan paths of IPMC manipulators. H. Wang et al. implemented a six-segment IPMC flexible manipulator; the paths were encoded using a Gaussian mixture model (GMM). Moreover, the recommended paths were generated using Gaussian mixture regression (GMR). The verification of the learned paths was conducted using an IPMC manipulator. They reported that the data from the operator were required, the generalized trajectories from the GMM and GMR could not always ensure the complete reproduction of the demonstrated task, and the approach was effective under static environments [24]. J. D. Carrico et al. presented machine learning with Bayesian optimization for the effective motion control of 3D-printed soft IPMC actuators in a soft crawling robot platform. However, there were challenging issues when it comes to controlling IPMC actuators. First, performance degradation occurred when the actuator operated such that the current voltage was higher than electrolysis voltage of the hydrating solvent. Second, the performance of the conventional control methods deteriorated over time. Thus, future works in controlling IPMCs will be predicting and planning the performance degradations using real-time degradation data [111].

Dielectric elastomer actuators (DEAs), consist of thin elastomer membranes between two compliant electrodes, are known to have rapid responses, large voltage-induced deformations, and noise-free operations. However, viscoelastic materials of the DEAs exhibit complex time-dependent behavior, such as creep, hysteresis, and the Maxwell stress that is related to the deformation of the actuators. As a result, the actual actuations based on the electromechanical coupling are very non-linear and time dependent [112]. In the case of a cuttlefish robot with a DEA as the jet-actuator, reinforcement learning algorithms such as Q-learning were used as the actuation strategy. The experimental results verified that the optimized control using reinforcement learning can enhance the actuation performances [113]. Li et al. conducted a study on DEA control. Based on deep reinforcement learning, a model-free method can be employed to achieve the dynamic feedback control of DEAs under the consideration of their time-dependent characteristics. Experiments were conducted on circular and rectangular DEA configurations to test their accuracy and robustness with respect to changes in the material properties and structures [112].

Shape memory actuators (SMAs) generate relatively large displacements and high force/weight ratios. However, SMAs have difficulties when modeling and controlling them when the space is continuous because the relationship between strain and temperature is hysteric and changed abruptly [113]. Recent studies that involved neural networks on SMAs were focused on SMA identification and modeling [114]. C. Cheng et al. proposed an SMA-actuated multiple-DOF soft robot with a simplified adaptive neural network control algorithm for the improvement of the accuracy of position control [62].

Several applications implementing soft actuators have aimed to perform tasks other than calibrations, control, or proprioception. For instance, soft wearable devices were employed to obtain body poses or fingertip forces due to contact. In such tasks, the human-related applications increase the complexity of soft robots with additional non-linearity, which can degrade performances. In addition, the human-related applications are complex for several reasons. First, human physical factors are different from person to person, like the height, weight, muscle strength, and patterns of human motions. Second, there are several different muscles involved when generating a single motion.

In several studies, learning-based methods were proposed for the manipulation of wearable hand robots. Ha et al. realized the position control of a soft wearable glove with pneumatic actuators using pressure and vision data [115]. In particular, deep learning allowed for position control in an open-loop without prior knowledge such as the user characteristics. Kim et al. proposed VIDEO-Net for the detection of human grasping by the recognition of arm behavior and hand/object interactions using a first-person-view camera [116]. The performance of VIDEO-Net was verified using a soft wearable hand robot for disabled people. Kang et al. proposed a learning-based fingertip force estimation method for wearable hand robots based on the tendon-sheath mechanism. In addition, a bending time-gradient LSTM (BT-LSTM) was proposed to mitigate the influence of the factors that decrease the accuracy of fingertip force estimations: (1) non-linearity and hysteresis of wearable robots and human hands, and (2) dynamic angular changes in the tendon-sheath [101]. Schlagenhauf et al. tested LR to control a tendon-driven soft robot hand, Cyberglove. They compared learning-based approaches, including kNN, LR, FNN, and deep reinforcement learning, when controlling soft foam robot hands; they found that kNN outperformed the other three methods under the simulated environment [117].

For soft manipulators and grippers, machine learning algorithms are primarily employed to obtain proprioception and control the robots to desired positions. Unlike rigid robots, soft robots have a high number of DOFs; thus, they are difficult to model and control. To solve this problem, machine learning models are extensively used. In particular, reinforcement learning-based methods are primarily applied, unlike other soft robotic fields. Scimeca et al. utilized an FNN to learn tactile image information. Moreover, an integration system with a tactile sensor was proposed to obtain internal pressure distributions based on the neural network [118]. In [23], a neural network controller for continuum robots was proposed. The controller comprised an FNN controller and a nonlinear feedback controller for the manipulation of an OCTARM VI manipulator [119121]. You et al. proposed a Q-learning method for the control of a honeycomb pneumatic network (HPN [122]) manipulator. Satheeshbabu et al. proposed an open-loop position controller based on deep reinforcement learning for a manipulator (BR2 manipulator [123]). Watson and Morimoto proposed to localize the tip of soft continuum robots that have potential to be usable as medical devices in which the medical field needs accurate control to guarantee safety. They used a LSTM to localize the magnet at the tip of the robot compared to existing analytic and hybrid methods [124]. In [125], a hybrid model for controlling a modular collaborative Variable-Stiffness-Link (VSL) robots has been proposed. It consisted of forward kinematics and inverse kinematics whose models are 7-layer FNN. The open-loop model was compared with a traditional model-based method, and showed that their model outperformed the traditional model. [126] proposed a learning-based approach for proprioception of three-dimensional soft sensorized robots. Unlike existing studies, it uses embedded sensor information. It also predicts 3-dimensional configuration of the robots based on the sensor data. The paper used LSTM, which was compared with 2-layer FNN, and showed that the RNN-based model reasonably estimates the steady-state configuration of the soft robots.

Due to the aforementioned material characteristics, it is difficult to analytically or empirically model soft actuators using traditional methods, thus making it difficult to design controllers. On the other hand, machine learning methods have been used to control soft actuators with reliable results in limited workspaces. A major disadvantage of using machine learning in control, compared to physical models, is the requirement of large number of datasets. For example, when it comes to reinforcement learning, it requires a lot of rollouts to train the algorithms to obtain desired controller policies.

Overall, soft actuators commonly show mechanical hysteresis and functional degradation over time. When soft actuators are employed in robotic applications, reliability is a dominant issue. Soft actuators are made of soft materials; these materials are highly non-linear compared to rigid materials, such as large distribution of elasticity and high dimensionality. This leads to a difficulty to predict an appropriate lifetime of the model [127]. Thus, as a future direction, applying a prognostic method will be useful to estimate the performance and the lifetime of soft actuators for the practical implementations [128, 129]. Since the data-driven approaches are widely spread in prognostics field due to its ability of quick implementations and developments, machine learning will be an applicable tool to predict the time-dependent nonlinear performance of the soft actuators. 

Note: The content above has been extracted from a research article, so it may not display correctly.



Q&A
Please log in to submit your questions online.
Your question will be posted on the Bio-101 website. We will send your questions to the authors of this protocol and Bio-protocol community members who are experienced with this method. you will be informed using the email address associated with your Bio-protocol account.



We use cookies on this site to enhance your user experience. By using our website, you are agreeing to allow the storage of cookies on your computer.