TY - GEN
T1 - Reinforcement Learning-Based Parameter Optimization for Whole-Body Admittance Control with IS-MPC
AU - Figueroa, Nicolas
AU - Tafur, Julio
AU - Kheddar, Abderrahmane
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - Maintaining stability in bipedal walking remains a significant challenge in humanoid robotics, largely due to the numerous involved hyperparameters. Traditional methods for determining these hyperparameters, such as heuristic approaches, can be both time-consuming and potentially suboptimal. In this paper, we present an approach aimed at enhancing the stability of bipedal gait, particularly when faced with floor perturbations and speed variations. Our main contribution is the integration of intrinsically stable model predictive control (IS-MPC) and whole-body admittance control within a closed-loop reinforcement learning system. We devised a reinforcement learning plugin, implemented in the mc-rtc framework, that allows the control system to continuously monitor the robot's current states, maintain recursive feasibility, and optimize parameters in real-time. Furthermore, we propose a reward function derived from a combination of changes in single and double support time, postural recovery, divergent control of motion, and action generation grounded in training optimization. In the course of this research, we conducted experiments on a real humanoid robot to validate initial aspects of our work. The integrated module's effectiveness was further assessed through comprehensive simulations.
AB - Maintaining stability in bipedal walking remains a significant challenge in humanoid robotics, largely due to the numerous involved hyperparameters. Traditional methods for determining these hyperparameters, such as heuristic approaches, can be both time-consuming and potentially suboptimal. In this paper, we present an approach aimed at enhancing the stability of bipedal gait, particularly when faced with floor perturbations and speed variations. Our main contribution is the integration of intrinsically stable model predictive control (IS-MPC) and whole-body admittance control within a closed-loop reinforcement learning system. We devised a reinforcement learning plugin, implemented in the mc-rtc framework, that allows the control system to continuously monitor the robot's current states, maintain recursive feasibility, and optimize parameters in real-time. Furthermore, we propose a reward function derived from a combination of changes in single and double support time, postural recovery, divergent control of motion, and action generation grounded in training optimization. In the course of this research, we conducted experiments on a real humanoid robot to validate initial aspects of our work. The integrated module's effectiveness was further assessed through comprehensive simulations.
UR - http://www.scopus.com/inward/record.url?scp=85186265664&partnerID=8YFLogxK
U2 - 10.1109/SII58957.2024.10417367
DO - 10.1109/SII58957.2024.10417367
M3 - Conference contribution
AN - SCOPUS:85186265664
T3 - 2024 IEEE/SICE International Symposium on System Integration, SII 2024
SP - 1405
EP - 1410
BT - 2024 IEEE/SICE International Symposium on System Integration, SII 2024
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2024 IEEE/SICE International Symposium on System Integration, SII 2024
Y2 - 8 January 2024 through 11 January 2024
ER -