MECHANICAL ENGINEERING PhD THESIS DEFENSE BY ZAID AL-SAADI



Title: Salience of Haptic Features for Interactive Behavior Classification in Physical Human-Human/Robot Collaboration.

Speaker: Zaid Al-Saadi

Time: April 24, 2023, 02:30 PM

Place: Online via Zoom

Thesis Committee Members:

Prof. Dr. Çağatay Başdogan (Advisor, Koç University)

Prof. Dr. Sözer (Koç University)

Prof. Dr. Duygun Erol Barkana (Yeditepe University)

Assoc. Prof. Dr. Mehmet Dogar (University of Leeds)

Asst. Prof. Dr. Yusuf Aydın (MEF University)

Asst. Prof. Dr. Ayşe Küçükyılmaz (Co-advisor, University of Nottingham)

 

Abstract:

In recent years, a considerable amount of research has been directed toward designing robots to cooperate with humans in daily activities that require close physical interaction. One particular area of interest is collaborative object transportation, where human dyads would rely heavily on haptic sensations to coordinate their actions.

Humans are sensitive to interactive behaviors during joint action, and can leverage haptic cues (i.e., the forces they apply and sense through their interactions with the manipulated object) to easily distinguish between interaction states, such as harmony and conflict. Such discrimination between interaction states is currently missing in physical human-robot interaction (pHRI), and collaboration can be improved by making the robot proactive against changes in interaction behaviors.

We trust that understanding physical human-human interactions (pHHI) is crucial to implement a robotic system that is able to classify dyadic interactive behaviors. This dissertation is an effort to provide insight into the inherently important role of haptics in differentiating distinct interaction behavior classes during the collaboration of two physically-linked partners. To the best of our knowledge, ours is the first study to design interaction-specific haptic features to improve identification of conflict-related pHHI behaviors in physical co-manipulation. This haptic information works as a basis in pHRI to enable proactive robotic partners that can accommodate varying interactive behaviors.

In the first part of this dissertation, we explore the prominence of haptic data to extract information about underlying interaction patterns within physical human-human interaction (pHHI). We work on a joint object transportation scenario involving two human partners, and show that haptic features, based on force/torque information, suffice to identify human interactive behavior patterns. We categorize the interaction into four discrete behavior classes. These classes describe whether the partners work in harmony or face conflicts while jointly transporting an object through translational or rotational movements. In our experimental study, we collect data from 12 human dyads and verify the salience of haptic features by achieving a correct classification rate over 91% using a Random Forest classifier.

The second part of this dissertation proposes a machine learning (ML) approach to detect and resolve motion conflicts that may occur between a human and a proactive robot during the execution of a collaborative task. Our approach aims to distinguish harmonious and conflicting behaviors by looking at dyadic interaction patterns rather than inferring solely about the individual state of the human partner. In doing so, we challenge the traditionally adopted dualistic attitude in physical HRI, which considers the human and the robot as distinct and separable entities, and instead, consider the kinesthetic information generated through the teamwork to describe the interactive quality of collaboration. We demonstrate that features derived from haptic (force/torque) data are sufficient to classify if the human and the robot harmoniously manipulate the object or they face a conflict. A conflict resolution strategy is implemented to get the robotic partner to proactively contribute to the task via online trajectory planning whenever interactive motion patterns are harmonious, and to follow the human lead when a conflict is detected. An admittance controller regulates the physical interaction between the human and the robot during the task. This enables the robot to follow the human passively when there is a conflict. An artificial potential field approach is used to make the robot proactive when partners work in harmony. An experimental study is designed to create scenarios involving harmonious and conflicting interactions between human and robot partners during the collaborative manipulation of an object, and to create a dataset to train and test our ML model. The results of the study show that ML successfully detects the conflicts and our conflict resolution mechanism reduces the human force and effort significantly compared to the case of a passive robot that always follows the human partner and a proactive robot that cannot resolve conflicts.