United States

A robots ability to effectively recognize human emotions is critical in human-robot collaboration. However, most of the current collaborative robots were designed to improve productivity. Few of these robots consider human emotions. This situation would cause humans to be unwilling to work with robots for a long time. Motivated by this gap, this research developed a human emotion recognition system for enhancing the interaction abilities of collaborative robots. In this project, both speech and facial information were analyzed for robust human emotion recognition in complex working environments like manufacturing assembly environments. In the experiment, the developed system has been tested through three different human-robot co-assembly scenarios: (1) the robot effectively assists humans in finishing the task, (2) the robot is slow in response, leading to the task failing, (3) the robot frequently picks up wrong tools leading to task failure. Experimental results have demonstrated the effectiveness of the developed system in recognizing human co-workers emotions when the human and the robot were working in the above scenarios. It further shows the developed system has the potential to contribute to the development of an empathic collaborative robot companion in the manufacturing area.

IEEE RO-MAN 2024

A Speech and Facial Information Based Emotion Recognition System of Collaborative Robot for Empathic Human-Robot Collaboration

speech and facial information

human-robot collaboration

emotion recognition

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. This theme also touches upon people's expectations and perceptions of robots, as well as other social, cognitive and cultural factors that will influence human-robot interactions. We will have a special session/track on this topic, invite targeted submissions and include plenary speakers who are specialist in this area. We sincerely hope that this focus will help enlighten the design-thinking of all of us as we fabricate and program our own various systems.

To access RO-MAN Virtual 2021, [please register here.](https://ro-man2021.org/registration/)

IEEE RO-MAN 2021

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. 

technical paper

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

Human joints are unique in the sense that their stiffness is not fixed and rather adjustable which aids in interacting with the physical world. A robot with servo-adjustable stiffness is intrinsically safe and capable of imitating human-like interactions. Apart from safety, soft joints are efficient in executing cyclic tasks which opens the door to have an agile behaviour. In this paper we extend our prior work on variable stiff joints and use two of them to design a two-link planar robot. Here we modify the dynamic model for this manipulator to include the variable stiffness term in addition to the joint torques as inputs, followed by joint torque calibration method, and then a simplified manipulator control template. This paper serves as an introduction to our ongoing work on vSEA and a foray into true manipulator space application.

R-R Manipulator Using Variable-Stiff Joints for Safe Human-Robot-Interaction

This paper introduces a new companion robot as an open research platform for studying human-robot interaction: The ZENIT Enabling Natural Interaction Technology. Open-source solutions for companion robots are often complicated to build or not thought of holistically. ZENIT is based on a few low-cost components, namely a smartphone and a stationary robot arm in conjunction with an average computer and a separate camera. This concept makes it possible not to be tied to specific hardware. Our platform aims to enable easy use of the multimodality of human communication channels such as body language, spatial behavior, facial expressions and speech so that ZENIT can understand its interlocutor and express itself appropriately. The fusion of various open-source software components in an adaptable, lightweight and resource-efficient distributed system simplifies its replication. The system demonstrated an adequate response time (speech: 2.69s, facial expressions: 0.55s, bodily movements: 0.32s) for perception and expression combined, as well as reliability in public testing. Evaluation of the robot's expressive capabilities in an online survey (N=23) revealed users were able to understand six of eight emotions conveyed by ZENIT using facial expressions and/or body language (except for contempt and fear) and perceived its non-verbal feedback as significantly more expressive when facial expressions were accompanied with bodily movements.

Fusing Components for an Attentive and Emotionally Expressive Companion Robot: Meet ZENIT

Rapid human-robot interactions require fast hardware platforms with minimal latency and high reliability. In response, we present a cost-effective, electrically actuated, tendon-driven robotic hand. This hand features a unique spool-free actuation mechanism that achieves a limit-to-limit flexion movement in less than 60 ms, matching human speed. To our knowledge, it is among the fastest electric motor tendon-driven robotic hands available today. The high speed of the robotic hand was successfully demonstrated in public by playing Rock Paper Scissors at a science fair. This research work outlines the design methodology and introduces a simulation-optimization framework that allows users to preview the motion of the hand, quantify the actuation performance, and customize the design parameters prior to fabrication. The proposed actuation mechanism, along with the simulation and optimization tools, illustrates design principles and computational methods applicable to other dynamic human-robot applications that require fast reaction times. The Dextra hand design is available at https://sensorsini.github.io/dextra-robot-hand.

A Rapid and Robust Tendon-Driven Robotic Hand for Human-Robot Interactions Playing Rock-Paper-Scissors

With recent technological improvements, the performance of conversational robots such as context-aware dialogue is expected to be dramatically enhanced. While context-aware utterance is considered one of the crucial functions of conversational robots, only a few studies investigate the relationship between context awareness and user preference.  As is the case with human social relationships, human-robot interactions are under the effect of user's attributes such as personality, and thus considering these factors of users is essential to continue interacting with robots. In this paper, we evaluate the preference for context awareness based on personality trait effects in an open-domain dialogue using a conversational android robot system. Two types of utterances, context-aware and context-free, are implemented in the robot system, and the participants talk with and evaluate our conversational system. The preference for two systems and the personality traits of each participant are collected. We analyze the relationship between preferences and personalities. The results suggest that the participants' preference for whether an utterance is context-aware or context-free is distinguished by some personality traits.

Evaluation of Preference on Context-Aware Utterances Based on Personality Traits Using a Conversational Android Robot System

Robots deployed into real-world task-based environments may need to provide assistance, troubleshooting, and on-the-fly instruction for human users. While previous work has considered how robots can provide this assistance while co-located with human teammates, it is unclear how robots might best support users once they are no longer co-located. We propose the use of Augmented Reality as a medium for conveying long-distance task guidance from humans existing robot teammates, through Augmented Reality facilitated Robotic Guidance (ARRoG). Moreover, because there are multiple ways that a robot might project its identity through an Augmented Reality Head Mounted Display, we identify two candidate designs inspired by existing interaction patterns in the human-robot interaction (HRI) literature (re-embodiment-based and telepresence-based identity projection designs), present the results of a design workshop to explore how these designs might be most effectively implemented, and the results of a human-subject study intended to validate these designs

Designing Augmented Reality Robot Guidance Interactions through the Metaphors of Re-Embodiment and Telepresence

This work describes an immersive control system for a quadruped robot, designed to track the head movements of the operator wearing a virtual reality eye-wear, while also utilizing joystick commands for locomotion control. The article details the implemented closed-loop velocity control approach and the locomotion task specifications. The proposed method has been implemented on a Spot robot from Boston Dynamics, with Meta Quest 2 virtual reality system. Evaluation of the approach involved a user study, where participants engaged in immersive control of the quadruped robot within an indoor experimental environment and provided feedback through standardized questionnaires. Pairwise comparison of the resulting data revealed significant advantages for the proposed immersive control system over a standard remote controller, with enhanced performance observed in the second trial of using the control system. However, participants lacking experience with virtual reality systems reported increased distress symptoms following the experiment.

Immersive Control of a Quadruped Robot with Virtual Reality Eye-Wear

Understanding human emotions is paramount for effective human-human interactions. As technology advances, social robots are increasingly being developed with the capability to discern and respond to human emotions, with the ultimate aim of providing assistance and companionship. However, existing research on emotion recognition for human-robot interaction predominantly focuses on facial expressions or verbal speech, neglecting other potential mediums of emotional expression. In this study, we shed light on the significance of considering various forms of emotional expression, mainly non-verbal vocalization known as vocal bursts, which have been overlooked in the context of emotion modeling for human-robot interaction. Vocal bursts, characterized by brief and intense vocal utterances, represent a rich source of emotional cues that can significantly enhance the capabilities of social robots in understanding and responding to human emotions. Driven by the increasing interest in vocal bursts within speech and affective computing research, we propose a baseline model for affective vocal burst recognition that can outperform large audio models. The proposed baseline model achieves weighted F1 scores of 0.606, 0.342, and 0.287 on 10, 24, and 30 emotion classes, respectively. Additionally, we identify challenges that must be addressed to enhance affective vocal burst recognition for human-robot interaction. Code available at github.com/rahullabs/Sigh

Sigh!!! There Is More Than Just Faces and Verbal Speech to Recognize Emotion in Human-Robot Interaction

Telepresence robots can enhance communication experiences by providing a sense of physical presence, embodiment and may evoke co-presence. In spite of that, telepresence robots have not made it fully to consumer markets. In this paper, we investigate how different levels of controlling a telepresence robot (teleoperation, shared control, and no control) influence presence. To this aim, we conducted a study (N=45) where participants were evenly distributed to one of the robot control conditions. The task involved navigating an unknown room and listening to stories told by a person co-located with the robot. We collected subjective impressions of presence using the temple presence inventory and performed a thematic content analysis on a post-experiment interview. Our results suggest nuances in perceived presence under different levels of robot control after performing a thematic content analysis. Co-presence can be experienced during teleoperation and shared control, and teleoperation may evoke negative sentiments if it does not provide enough spatial information during navigation. However, our results did not point to significant differences in spatial or social presence. We consider that these findings encourage further discussions on how presence is perceived in robot-mediated communication.

An Exploratory Study on the Impact of Varying Levels of Robot Control on Presence in Robot-Mediated Communication

Looking at state-of-the-art robots, the majority have a white surface. This raises the questions of why robots are designed predominantly white, and how surface color affects the robots social perception. This not only reinforces human stereotypes but also influences the acceptance and usage of robots. Accordingly, this online study with a 2 (black vs. white) x 3 (Pepper robot vs. Temi robot vs. Kuka robot) within-subjects design (N = 100) investigated the effect on racist stereotypical perception and the role of anthropomorphism using implicit (Implicit Association Test) and explicit (Robotic Social Attributes Scale) measures and questionnaires on behavioral intentions. Our results show that robot color influences the implicit perception of robots but rarely affects explicit perception and behavior measures. Moreover, no effects of the robots' anthropomorphism on racist perceptions were found. As more effects of anthropomorphism on our outcome variables were found than for robot color, we assume that robot color activates stereotypes in terms of color rather than race.

All Too White? Effects of Anthropomorphism on the Stereotypical Perception of Robot Color

The rapid advancements of drones have demonstrated the versatility and promising potential of human-swarm systems (HSS) across various domains. However, human performance within these systems may be impaired by factors such as limited domain knowledge and mental stress, often leading to cognitive overload and hindering the efficiency and effectiveness of human-swarm teaming. Consequently, the accurate monitoring of cognitive load levels is crucial for optimizing HSS performance. To address the challenges of existing measurement methods, which are often expensive, time-consuming, or lack real-time capabilities, we propose a Large Language Model driven cognitive load prediction framework. This framework integrates comprehensive task context, domain knowledge, and behavior analysis to provide fast and cost-effective predictions in complex scenarios. By leveraging the capabilities of Large Language Models and employing reinforcement learning to model the cognitive load generation, our framework aims to offer real-time insights into human-related factors causing high cognitive load and predict cognitive levels over time, ultimately enhancing the performance of HSS teaming.

Premium content

Downloads

Next from IEEE RO-MAN 2024

R-R Manipulator Using Variable-Stiff Joints for Safe Human-Robot-Interaction

Similar lecture

Robot Facial Expression Framework for Enhancing Empathy in Human-Robot Interaction