MECHANICAL ENGINEERING MS THESIS DEFENSE BY BERK GÜLER

Title: Deep Reinforcement Learning to Optimize Task Performance in Human-Robot Co-Manipulation

Speaker: Berk Güler

Time: May 2, 2023,  5:30 pm

Thesis Committee Members:

Prof. Çağatay Başdoğan  (Advisor, Koç University)

Asst. Prof. Özgür S. Oğuz (Bilkent University)

Asst. Prof. Yusuf Aydın (MEF University)

Abstract:

We propose a two-layer machine learning (ML) approach, which utilizes an artificial neural network (ANN) model as a precursor for a deep reinforcement learning (DRL) model, to optimize task performance during human-robot co-manipulation of heavy objects. In the first layer, the ANN model estimates the human intention to accelerate or decelerate the object (which precedes the actual acceleration or deceleration of the object due to its large inertia). This probabilistic estimation is then used to calculate the gain of an adaptive admittance controller, which proactively alters the robot’s contribution to the task. In the second layer, the DRL model fine-tunes this gain and optimizes the task performance by minimizing jerk in movement and physical effort made by the human. Since online training of a DRL model for a physical human-robot interaction task is highly time-consuming and can potentially be dangerous for the human due to abrupt changes in controller gain, a data-driven human force model was developed by a conditional variational auto-encoder (C-VAE) for offline training of the DRL model via simulations. For this purpose, experimental data was collected from six subjects under three different fixed gains of the admittance controller (minimum, nominal, and maximum) to train and validate the DRL model. For online testing and comparison, the same co-manipulation task was performed with five new subjects under the controllers utilizing adaptive admittance gain profiles generated in real-time by the ANN model alone and the proposed two-layer approach (ANN + DRL). The experimental results show that the optimized gain profile obtained by the two-layer approach leads to a significant decrease in human effort and jerk compared to the initial profile provided by the ANN model.