Improving Reinforcement Learning Algorithm Based on Non-Negative Matrix Factorization Method for Controlling an Arm Model

Abstract

Purpose: Reinforcement Learning (RL) is attracting great interest because it enables systems to learn by interacting with the environment. This study aims to enhance the RL algorithm to become more similar to human motor control by combining it with the Non-negative matrix factorization (NMF) method.

Materials and Methods: In the study, the signals recorded from six muscles involved in arm-reaching movement without carryinga certain weight.were pre-processed, and the optimal number of synergy patterns was extracted using NMF and the Variance Account For (VAF) methods. This, in turn, contributes to reducing the calculations. Subsequently, the robustness of the two-link arm model with six muscles was evaluated under various noise levels applied to the action coefficient matrix. Finally, the average synergy pattern was done on the mentioned arm model, and the RL algorithm controlled it by producing the action coefficient matrix.

Results: The average VAF% was 97.25±0.45%, and the number of synergies was four. The tip-of-the-arm model was able to reach the target after an average of 100 episodes.

Conclusion: The results indicated that the similarity in the extracted synergy patterns helps to model a system that is more similar to motor control. Additionally, the results of the synergistic patterns revealed that the two-link arm model with six muscles was suitable for the model. While controlling the model with the RL algorithm, the desired end-point position and path were achieved.