United States

As collaborative robots become more common in manufacturing scenarios and adopted in hybrid human-robot teams, we should develop new interaction and communication strategies to ensure smooth collaboration between agents. In this paper, we propose a novel communicative interface that uses Mixed Reality as a medium to perform Kinesthetic Teaching (KT) on any robotic platform. We evaluate our proposed approach in a user study involving multiple subjects and two different robots, comparing traditional physical KT with holographic-based KT through user experience questionnaires and task-related metrics.

IEEE RO-MAN 2024

Kinesthetic Teaching in Robotics: A Mixed Reality Approach

kinesthetic teaching

mixed reality

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

technical paper

Recently, electroencephalographic (EEG) signals have been used to enhance Human-Robot Interaction (HRI). In particular, Error-Related Potentials (ErrPs) have been exploited since very few years. These potentials are evoked when there is a mismatch between the command given by the subject and the movements of the robot, or if the users expectation is different from the robot or other human behavior. These signals can be used to improve and customize the robot system, as feedback to better adapt the robot to human needs. This work aims to investigate and detect the ErrPs during different interaction tasks. We set up an experiment divided into five different tasks, where every task has 120 events with a 25%- 35% probability of error. The robot used in the experiment is a Baxter robot and the commands from the subject to the robot are sent in two different ways: with a keyboard or with a motion capture device. This work aims to reproduce a simplified teleoperated pick and place task. However, the achieved results do not allow to correctly identify the ErrPs, but exhibit only some minor differences between trials with and without errors. Hence, we here analyze the reasons behind such negative results, focusing on the challenges of the structure and the setup of the experiment. We analyze the possible problems and provide some recommendations to overcome them in similar use cases.

Challenges in Detecting and Analyzing EEG Error-Related Potentials: Lessons from a Case Study in HRI

Using socially assistive robots (SARs) as specialized companions for those living with depression to manage symptoms provides a unique opportunity for exploration of robotic systems as comfort objects. Moreover, the robotic components allow for specialized behavioral responses to particular stimuli, as preferred by the user. We have conducted semi-structured interviews with 10 participants about the zoomorphic robots Therabot desired behaviors and focus groups with five additional participants regarding the preferred sensors within the Therabot system. In this paper, using the data from interviews and focus groups, we explore SAR input and output for depression management. While participants overall expected the robot to respond in much similar ways as a well-trained service animal, they expressed interest in the robot understanding unique information about the environment and the user, such as when the user might need interaction.

The Ins and Outs of Socially Assistive Robots: Sensors and Behaviors of a Therapeutic Robot for Depression Management

The global aging population is growing, leading to an increased need for unpaid carers to support older adults. Socially Assistive Robots (SARs) can play an important role in supporting unpaid carers. However, as with any socio-technical system, building and maintaining trust is critical for achieving full potential and adoption of a SAR. Despite the growing interest in robotics across various domains, there has been limited research on time-dependant patterns of trust during Human Robot Interactions (HRI) and how unpaid carers perceive SARs as a support tool for care. We present a study in which we a) investigate the level of trust that unpaid carers have in using SARs for taking care of older adults, b) qualitatively explore the factors that influence trust considering the dynamic and context dependant nature of care. To explore SARs as a support tool from the perspective of unpaid carers we recruited 15 caregivers, we conducted co-design sessions, interviews and carers interacted with a Pepper robot evaluating trust before and after interaction. We discuss how the level of trust of unpaid carers in SARs increases after interaction and how unpaid carers trust in SARs is impacted by socially and culturally intelligent robots. Our work contributes to identify factors influencing trust that are important for carers while interacting with SARs, offering insights for the development and implementation of future SAR-based as a support tool that can assist them in taking care of older adults.

An Exploratory Analysis of Trust in Socially Assistive Robot Interactions with Unpaid Carers of Older Adults

Engaging in regular exercise offers numerous mental and physical health benefits. Many people, after the Covid-19 pandemic, started to work out at home. However, performing exercises incorrectly can result in injuries. For this reason, technology can be an effective tool for encouraging physical activity at home, and social robots have emerged as a potential solution for providing fitness training in a safe and engaging manner. A physical robot is often perceived as more socially attractive compared to virtual agents. Despite the growing popularity and existing literature on social robotics-based fitness solutions, there is a lack of research examining the accuracy of these systems and interventions at home during physical exercise recognition.In this paper, we propose a novel approach utilizing the Ubtech alpha mini social robot, which possesses motor capabilities to demonstrate exercises. We endowed the robot with the capability to recognize exercise correctness and motivate users to engage in physical activities. In this project, first of all, a new dataset of physical exercises was created, including 1,500 videos of physical exercises. Then, a new approach to recognize physical exercises from image sequences was presented. This method was implemented in a personal social robot to recognize physical exercises using a deep learning model. The results of the study validate the effectiveness of the system in recognizing physical exercises through the camera onboard a personal social robot.

Alpha Mini Social Robot As a Fitness Trainer at Home

With the increase of prevalence in autism, the need for efficient public health support has been amplified. Socially-assistive robots (SARs) have been found effective in engaging and interacting with autistic children, however, the perception intelligence during interaction still needs more domain-specific knowledge in terms of understanding children's behaviors. The Family Observation Schedule-Second Version (FOS-II) is one of the key methods in assessing parent-child interactions in developmental disabilities, yet its manual annotation demands considerable time and effort. This study proposes a multi-modal artificial intelligence (AI) model using video and audio inputs for automated FOS-II annotation. Utilizing advanced deep learning for behavior recognition, this method offers rapid, cost-effective FOS-II scaling. It will thus enhance the capability of socially assistive robots to understand human behaviors and support the advancement of digital health research for children with autism. The visual perception in home settings are most likely based on uncontrolled environments, so it is crucial to develop algorithms that can robustly work with video-log data with uncontrolled quality. Ultimately, it aims to ease the burden on parents and caregivers, streamlining the monitoring and treatment of challenging behaviors in autism.

Multimodal and Multi-Lingual Deep Neural Network for Interactive Behavior Style Recognition from Uncontrolled Video-Logs of Children with Autism

This qualitative study explored 20 grade 3-5 childrens perceptions of learning STEM vocabulary with an educational social robot. A semi-structured interview protocol was used to elicit childrens perceptions. Interviews were recorded and transcribed into a database reflecting one talk-turn per row (911 talk-turns total). Two coders used emergent coding and constant comparative method to identify talk-turns that reflected childrens perceptions of the assets and drawbacks of learning STEM vocabulary with the robot. Findings identified new assets and drawbacks about the robots instruction, and new drawbacks about the robots speech, which were not explored in previous research. Findings suggest design implications, including designing robots with the capacity for more individualization of instruction and adjustable movement and speech features based on learners preferences.

If a Robot Was Teaching, Then Everybody Would Definitely Like School Better: An Analysis of Grade 3-5 Children's Perceptions of Learning STEM Vocabulary with a Social Educational Robot

Capturing real-world 3D spaces as point clouds is efficient and descriptive, but it comes with sensor errors and lacks object parametrization. These limitations render point clouds unsuitable for various real-world applications, such as robot programming, without extensive post-processing (e.g., outlier removal, semantic segmentation). On the other hand, CAD modeling provides high-quality, parametric representations of 3D space with embedded semantic data, but requires manual component creation that is time-consuming and costly. To address these challenges, we propose a novel solution that combines the strengths of both approaches. Our method for 3D workcell sketching from point clouds allows users to refine raw point clouds using an Augmented Reality (AR) interface that leverages their knowledge and the real-world 3D environment. By utilizing a toolbox and an AR-enabled pointing device, users can enhance point cloud accuracy based on the devices position in 3D space. We validate our approach by comparing it with ground truth models, demonstrating that it achieves a mean error within 1cm - significant improvement over standard LiDAR scanner apps.

Precise Workcell Sketching from Point Clouds Using an AR Toolbox

Developments in the field of social robotics open interesting opportunities for applications in healthcare, education, and services. For this, studying engagement in human-robot interaction (HRI) is crucial for improving the quality of interactive experiences. Questionnaires are powerful in describing voluntary behavior; however, engagement is often an implicit non-voluntary behavior that reaches awareness only once initiated. Inspired by research in cognitive psychology, we propose a behavioral feature to quantify engagement in HRI through the measurement of spontaneous movement and spectral wavelet analysis. For this, we conducted an experiment during which participants listened to sad stories narrated by a moving social robot. Throughout the experiment, we tracked the participants' spontaneous and non-voluntary sway movements with a motion capture system. The experiments were conducted with three robotic platforms (Buddy, Pepper, and Nao). Results showed that spontaneous body sway can be modulated by social robots within no-goal-oriented interaction. This opens up the application possibility of our methodology to various modes of HRI such as collaboration.

Exploring Engagement in Human-Robot Interaction through the Quantification of Human Spontaneous Movement

Socially Assistive Robots (SARs) represent a valid support to professional caregivers in providing care to person with need. To improve the quality of the interaction, SARs should be able to automatically assess user engagement. In this work, we addressed this problem by investigating user engagement dynamics during a robot-to-human handover task, considering 3 main components of engagement: affective, cognitive, and behavioral. For this study, we automatically extracted 10 visual features from the camera recordings of 31 participants. Each individual engaged in eight consecutive sessions with a robot manipulator designed with social cues. Our statistical analysis indicates that prolonged interaction with the robot could influence user engagement. Namely, we observed a decrease in positive emotions (affective), a more regulated quantity of motion (behavioral), and a reduced attention on the robot tasks (cognitive). Overall, the results of this study suggests that engagement dynamics can be described by the selected behavioral features, and that the a more predictable robots behavior could negatively influence user engagement.

Investigating User Engagement Dynamics in Robot-To-Human Handovers with a Social Manipulator

HRI research has evolved to take a broader, more inclusive view of how culture influences our interaction with robots. As we delve deeper into cultural integration in HRI, it has become evident that while integrating cultural aspects offers new opportunities, it requires careful consideration due to the heightened sensitivity to the fluid nature of cultural dynamics. Our study focuses on a particular case and examines the role of context and personal preferences in a restaurant setting. We investigate how preferences for cross-cultural greetings performed by a humanoid robot can change based on the restaurant theme and describe what factors influence these preferences by looking at two different groups who participated based on different ethnic greetings. Our study reveals insight into how ethnicity, percentage of life lived in Western countries, personality variations, and implementation of cultural aspects influence the likability of robotic greeting gestures. Our investigations highlight the complexity of creating culturally adaptive robots that demonstrate the cultural norms and gestures that align with the expectations of the respective cultural groups.

Premium content

Downloads

Next from IEEE RO-MAN 2024

Challenges in Detecting and Analyzing EEG Error-Related Potentials: Lessons from a Case Study in HRI

Similar lecture

Investigating Mixed Reality for Communication between Humans and Mobile Manipulators