United States

This paper presents the design of a novel social robot, namely Patbot, to engage people in playful interactions to help reduce their anxiety in waiting environments. We introduce the rationale and design decisions made during the development of the robot. We evaluated the robot&#39;s usability through an experiment within a simulated waiting environment, followed by interviews with the participants. The study indicated a positive effect of interacting with Patbot regarding STAI-6 anxiety measures. Other quantitative and qualitative measures, including Godspeed questionnaires, observation, and post-interviews, revealed positive impressions made by Patbot regarding its likability and animacy and that it could engage individuals in intuitive and playful activities. Our findings suggest the promise of designing social robots for entertainment and relaxation to enhance people&#39;s emotional experiences in waiting environments.

IEEE RO-MAN 2024

Patbot: Designing a Social Robot to Reduce Anxiety in Waiting Environments

robot companions and social robots; robots in art and entertainment; non-verbal cues and expressiveness

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. This theme also touches upon people's expectations and perceptions of robots, as well as other social, cognitive and cultural factors that will influence human-robot interactions. We will have a special session/track on this topic, invite targeted submissions and include plenary speakers who are specialist in this area. We sincerely hope that this focus will help enlighten the design-thinking of all of us as we fabricate and program our own various systems.

To access RO-MAN Virtual 2021, [please register here.](https://ro-man2021.org/registration/)

IEEE RO-MAN 2021

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. 

This paper presents the design of a novel social robot, namely Patbot, to engage people in playful interactions to help reduce their anxiety in waiting environments. We introduce the rationale and design decisions made during the development of the robot. We evaluated the robot's usability through an experiment within a simulated waiting environment, followed by interviews with the participants. The study indicated a positive effect of interacting with Patbot regarding STAI-6 anxiety measures. Other quantitative and qualitative measures, including Godspeed questionnaires, observation, and post-interviews, revealed positive impressions made by Patbot regarding its likability and animacy and that it could engage individuals in intuitive and playful activities. Our findings suggest the promise of designing social robots for entertainment and relaxation to enhance people's emotional experiences in waiting environments.

technical paper

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

We explore the potential of mobile motion as an expression modality for home social robots with a low expressivity. To gauge the potential of mobile motion, we examine its relative efficacy in terms of emotion perception accuracy, emotion intensity, and impression (anthropomorphism and animacy) compared to screen-based facial expressions. Additionally, we explore how users perceive the emotional intensity of expressions based on different degrees of expression by manipulating emotional features such as motion size and speed. Motion expressions are less accurate than facial expressions, but perform on par with facial expressions in other metrics. Its dynamic and expressive features elicit powerful emotional conveyance, in contrast to the low emotional impact associated with the monotonous nature of screen-based facial expressions. Further research is needed on ME for specific emotions, but in general, the higher the degree of expression, the more intense the emotion conveyed. We show that the degree of expression, i.e., the combination of emotional features and their modulation, can be utilized to express the emotional intensity, situational dependence, and personality of robots. In conclusion, we argue that mobile motion is a promising method to compensate for the weaknesses of screen-based facial expression, which is the dominant expression modality for low expressivity robots.

Exploring the Potential of Wheel-Based Mobile Motion As an Emotional Expression Modality

Integrating intelligent systems, such as robots, into dynamic group settings poses challenges due to the mutual influence of human behaviors and internal states. A robust representation of social interaction dynamics is essential for effective human-robot collaboration. Existing approaches often narrow their focus to facial expressions or speech, overlooking the broader context. We propose employing an adapted Temporal Graph Networks to comprehensively represent social interaction dynamics while enabling its practical implementation. Our method incorporates temporal multi-modal behavioral data including gaze interaction, voice activity and environmental context. This representation of social interaction dynamics is trained as a link prediction problem using annotated gaze interaction data. The F1-score outperformed the baseline model by 37.0%. This improvement is consistent for a secondary task of next speaker prediction which achieves an improvement of 29.0%. Our contributions are two-fold, including a model to representing social interaction dynamics which can be used for many downstream human-robot interaction tasks like human state inference and next speaker prediction. More importantly, this is achieved using a more concise yet efficient message-passing method, significantly reducing the message size from 768 to 14 while outperforming the baseline model.

Modeling Social Interaction Dynamics Using Temporal Graph Networks

A crucial part of traditional reinforcement learning (RL) is the initial exploration phase, in which trying available actions randomly is a critical element. As random behavior might be detrimental to a social interaction, this work proposes a novel paradigm for learning social robot behavior - the use of shielding to ensure socially appropriate behavior during exploration and learning. We explore how a data-driven approach for shielding could be used to generate listening behavior. In a video-based user study (N=110), we compare shielded exploration to two other exploration methods. We show that the shielded exploration is perceived as more comforting and appropriate than a straightforward random approach. Based on our findings, we discuss the potential for future work using shielded and socially guided approaches for learning idiosyncratic social robot behaviors through RL.

Shielding for Socially Appropriate Robot Listening Behaviors

This paper presents a system for diversity-aware autonomous conversation leveraging the capabilities of large language models (LLMs). The system adapts to diverse populations and individuals, considering factors like background, personality, age, gender, and culture. The conversation flow is guided by the structure of the system's pre-established knowledge base, while LLMs are tasked with various functions, including generating diversity-aware sentences. Achieving diversity-awareness involves providing carefully crafted prompts to the models, incorporating comprehensive information about users, conversation history, contextual details, and specific guidelines. To assess the system's performance, we conducted both controlled and real-world experiments, measuring a wide range of performance indicators.

Enhancing LLM-Based Human-Robot Interaction with Nuances for Diversity Awareness

This paper presents a comprehensive framework to enhance Human-Robot Collaboration (HRC) in real-world scenarios. It introduces a formalism to model articulated tasks, requiring cooperation between two agents, through a smaller set of primitives. Our implementation leverages Hierarchical Task Networks (HTN) planning and a modular multisensory perception pipeline, which includes vision, human activity recognition, and tactile sensing. To showcase the system's scalability, we present an experimental scenario where two humans alternate in collaborating with a Baxter robot to assemble four pieces of furniture with variable components. This integration highlights promising advancements in HRC, suggesting a scalable approach for complex, cooperative tasks across diverse applications.

A Modular Framework for Flexible Planning in Human-Robot Collaboration

Vertical take-off and landing (VTOL) aircraft do not require a prolonged runway, thus allowing them to land almost anywhere. In recent years, their flexibility has made them popular in development, research, and operation. When compared to traditional fixed-wing aircraft and rotorcraft, VTOLs bring unique challenges as they combine many maneuvers from both types of aircraft. Pilot workload is a critical factor for safe and efficient operation of VTOLs. In this work, we conduct a user study to collect multimodal data from 28 pilots while they perform a variety of VTOL flight tasks. We analyze and interpolate behavioral patterns related to their performance and perceived workload. Finally, we build machine learning models to estimate their workload from the collected data. Our results are promising, suggesting that quantitative and accurate VTOL pilot workload monitoring is viable. Such assistive tools would help the research field understand VTOL operations and serve as a stepping stone for the industry to ensure VTOL safe operations and further remote operations.

How Is the Pilot Doing: VTOL Pilot Workload Estimation by Multimodal Machine Learning on Psycho-Physiological Signals

Virtual Reality (VR) environments are used as a way to let humans experiment with algorithms, techniques, and situations in various application areas, including emergency management, serious games, smart manufacturing, and precision agriculture. They are especially relevant when experiments in the real world may be harmful to human operators. As VR environments are the closest possible faithful replicas of real environments, many recent works focus on the employment of such tools as a means to generate synthetic datasets that can be used for training machine and deep learning models, especially in situations where obtaining real datasets can be difficult. In this paper, we introduce a strategy to generate a dataset for pose estimation in the challenging scenario of precision agriculture. Finally, the quality of the generated dataset was evaluated.

Generating and Evaluating Synthetic Data in Virtual Reality Simulation Environments for Pose Estimation

A robots ability to effectively recognize human emotions is critical in human-robot collaboration. However, most of the current collaborative robots were designed to improve productivity. Few of these robots consider human emotions. This situation would cause humans to be unwilling to work with robots for a long time. Motivated by this gap, this research developed a human emotion recognition system for enhancing the interaction abilities of collaborative robots. In this project, both speech and facial information were analyzed for robust human emotion recognition in complex working environments like manufacturing assembly environments. In the experiment, the developed system has been tested through three different human-robot co-assembly scenarios: (1) the robot effectively assists humans in finishing the task, (2) the robot is slow in response, leading to the task failing, (3) the robot frequently picks up wrong tools leading to task failure. Experimental results have demonstrated the effectiveness of the developed system in recognizing human co-workers emotions when the human and the robot were working in the above scenarios. It further shows the developed system has the potential to contribute to the development of an empathic collaborative robot companion in the manufacturing area.

A Speech and Facial Information Based Emotion Recognition System of Collaborative Robot for Empathic Human-Robot Collaboration

Human joints are unique in the sense that their stiffness is not fixed and rather adjustable which aids in interacting with the physical world. A robot with servo-adjustable stiffness is intrinsically safe and capable of imitating human-like interactions. Apart from safety, soft joints are efficient in executing cyclic tasks which opens the door to have an agile behaviour. In this paper we extend our prior work on variable stiff joints and use two of them to design a two-link planar robot. Here we modify the dynamic model for this manipulator to include the variable stiffness term in addition to the joint torques as inputs, followed by joint torque calibration method, and then a simplified manipulator control template. This paper serves as an introduction to our ongoing work on vSEA and a foray into true manipulator space application.

R-R Manipulator Using Variable-Stiff Joints for Safe Human-Robot-Interaction

This paper introduces a new companion robot as an open research platform for studying human-robot interaction: The ZENIT Enabling Natural Interaction Technology. Open-source solutions for companion robots are often complicated to build or not thought of holistically. ZENIT is based on a few low-cost components, namely a smartphone and a stationary robot arm in conjunction with an average computer and a separate camera. This concept makes it possible not to be tied to specific hardware. Our platform aims to enable easy use of the multimodality of human communication channels such as body language, spatial behavior, facial expressions and speech so that ZENIT can understand its interlocutor and express itself appropriately. The fusion of various open-source software components in an adaptable, lightweight and resource-efficient distributed system simplifies its replication. The system demonstrated an adequate response time (speech: 2.69s, facial expressions: 0.55s, bodily movements: 0.32s) for perception and expression combined, as well as reliability in public testing. Evaluation of the robot's expressive capabilities in an online survey (N=23) revealed users were able to understand six of eight emotions conveyed by ZENIT using facial expressions and/or body language (except for contempt and fear) and perceived its non-verbal feedback as significantly more expressive when facial expressions were accompanied with bodily movements.

Premium content

Downloads

Next from IEEE RO-MAN 2024

Exploring the Potential of Wheel-Based Mobile Motion As an Emotional Expression Modality

Similar lecture

Hierarchical Motion Planning Framework for Manipulators in Human-Centered Environments