Italy

Human-robot communication is key to establishing transparency of robot states, and promoting psychological safety of the people interacting with a robot. Such communication is especially challenging for appearance-constrained robots, as they do not afford to employ commonly-used anthropomorphic modalities such as facial expressions. 
We explore how an appearance-constrained quadruped robot can use LEDs and gestures to generate affective expressions in order to non-verbally communicate. Traditionally, affect has been used as a signalling paradigm in Human-Robot Interaction where expressions are designed with the goal of communicating a particular emotion such as happiness. Instead, this work explores a generalizable mapping from informative messages (e.g., &quot;I am listening&quot;) to abstract affective expressions. We modulate the expressions by a model of affect, taking inspirations from Affect Control Theory (ACT). To explore designing the expressions, we first conducted pilot semi-structured interviews consulting stakeholders who regularly operate appearance-constrained robots (N=12) to understand the communication requirements. Then, we designed a set of six affective expressions and evaluated them in a crowdsourced study (N=450). Findings suggest that the expressions can significantly improve effective communication of a robot&#39;s awareness and intent, and promote psychological safety of people interacting with it.

IEEE RO-MAN 2022

User Requirements for a Robot Teleoperation system for General Medical Examination

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. This theme also touches upon people's expectations and perceptions of robots, as well as other social, cognitive and cultural factors that will influence human-robot interactions. We will have a special session/track on this topic, invite targeted submissions and include plenary speakers who are specialist in this area. We sincerely hope that this focus will help enlighten the design-thinking of all of us as we fabricate and program our own various systems.

To access RO-MAN Virtual 2021, [please register here.](https://ro-man2021.org/registration/)

IEEE RO-MAN 2021

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. 

Human-robot communication is key to establishing transparency of robot states, and promoting psychological safety of the people interacting with a robot. Such communication is especially challenging for appearance-constrained robots, as they do not afford to employ commonly-used anthropomorphic modalities such as facial expressions. 
We explore how an appearance-constrained quadruped robot can use LEDs and gestures to generate affective expressions in order to non-verbally communicate. Traditionally, affect has been used as a signalling paradigm in Human-Robot Interaction where expressions are designed with the goal of communicating a particular emotion such as happiness. Instead, this work explores a generalizable mapping from informative messages (e.g., "I am listening") to abstract affective expressions. We modulate the expressions by a model of affect, taking inspirations from Affect Control Theory (ACT). To explore designing the expressions, we first conducted pilot semi-structured interviews consulting stakeholders who regularly operate appearance-constrained robots (N=12) to understand the communication requirements. Then, we designed a set of six affective expressions and evaluated them in a crowdsourced study (N=450). Findings suggest that the expressions can significantly improve effective communication of a robot's awareness and intent, and promote psychological safety of people interacting with it.

technical paper

**31st IEEE International Conference on Robot & Human Interactive Communication**
 This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.
 
 The conference covers a wide range of topics related to Robot and Human Interactive Communication, involving theories, methodologies, technologies, empirical and experimental studies. Papers related to the study of robotic technology, psychology, cognitive science, artificial intelligence, human factors, ethics and policies, interaction-based robot design and other topics related to human-robot interaction are welcome.
 
 **CONFERENCE THEME:**
 Social, asocial, and antisocial robots
 
 Robots are starting to populate our lives: at work, in our homes, hotels, shops. However, not everybody is comfortable with robots finding a place in society. For example, people may be afraid that robots will replace humans in their jobs. This belief is exacerbated by the media that, on one side, raise expectations of robots’ capabilities, while amplifying fears and worries on the other side.
 
 In the next years, due to the increasing request for interaction with people in social environments, academic research and market analyses predict that robots will become more and more “social”. But, at the same time, robots may incur the risk of being perceived as “asocial” as their pervasiveness in everyday environments may become an obstacle to social interaction between humans. Or they may even be rejected as “antisocial” if researchers do not take care to develop robots that behave according to the customs and habits of the communities where they are expected to operate.
 
 To prevent this from happening, robotic researchers can no longer take the luxury of being mere technicians. They must be capable of addressing all the challenges raised by future robot-populated communities in a multidisciplinary way. They need to share their knowledge with the public to make people aware of what robots really are, what expectations we can have from them, the risks we may incur, and the possible mitigation plans.
 
 While being open to any kind of contribution related to Human-Robot Interactive Communication, the RO-MAN 2022 conference will emphasize these aspects of robotic research. How can we create robots capable of being welcomed as our assistants and companions in society while, at the same time, not being perceived as “asocial” or “antisocial?” How can we create robots perceived as facilitators of the natural development of human beings and communities, rather than obstacles?
 
 Contributions from different disciplines, not exclusively related to technological development, will be welcomed as crucial to pave the way to global acceptance of future robots with an informed attitude.
 
 [![](https://assets.underline.io/uploads/markdown_image/1/image/af259abe105b3791c498ae1552d75710.png)](https://drive.google.com/file/d/18w73BZPB_JVOgcrcdZDqYpEupwVt4CnU/view?usp=sharing)

A registration is required to access this event page. If you have not already done so, please use the below link to register for the event. 

[Register Here](http://www.smile.unina.it/ro-man2022/welcome-to-ieee-ro-man-2022/)

Registration Is Required

31st IEEE International Conference on Robot & Human Interactive Communication
This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.

In the last few years, Extended reality (XR) has enabled novel forms of sensory experiences and social interplay, which can be hardly experienced in real life. 
However, the full potential of XR has not been exploited yet, since vision remains the main interaction modality, and the time- and space- modulation of the sense of self - which could open interesting perspectives in several scenarios - is still largely unexplored.
To pave the path to a multi-modal manipulation of the sense of time and space in immersive XR, in this work we discuss the preliminary outcomes of the first investigation in the visual-tactile domain of two well known perceptual illusions affecting spatial and temporal perception, i.e. Tau and Kappa effects, respectively. 
We compared the effects originated from unimodal stimulation (i.e., only visual or tactile) with the same effects induced by convergent bimodal stimulation (i.e., visual and tactile), delivered to the forearm. 
Results show that both Tau and Kappa effects are affected by the multi-modality of the stimulation, and that the perceptual bias differently affects time- or space- perception based on the modality used for stimulus delivery. Our results, although preliminary, seem to suggest that multimodal perceptual illusions could be a viable solution for time- and space- modulation of the sense of self in immersive XR and advanced social human-robot interaction.

From Task Analysis to Wireframe Design: An Approach to User-Centered Design of a GUI for Mobile HRI at Assembly Workplaces

Domestic robots and agents are widely sold to the grand public, leading us to ethical issues related to the data harvested by such machines. While users show a general acceptance of these robots, concerns remain when it comes to information security and privacy. Current research indicates that there’s a privacy-security trade-off for better use, but the anthropomorphic and social abilities of a robot are also known to modulate its acceptance and use. To explore and deepen what literature already brought on the subject we examined how users perceived their robot (Replika, Roomba©, Amazon Echo©, Google Home©, or Cozmo©/Vector©) through an online questionnaire exploring acceptance, perceived privacy and security, anthropomorphism, disclosure, perceived intimacy, and loneliness. The results supported the literature regarding the potential manipulative effects of robot’s anthropomorphism for acceptance but also information disclosure, perceived intimacy, security, and privacy.

Designing the mobile robot Kevin for a life science laboratory

Robot-initiated touch is a promising mode of expression that would allow robot caregivers to perform physical tasks (instrumental touch) and provide comfort (affective touch) in healthcare settings. To understand the factors that shape how people respond to touch from a robotic caregiver, we conducted a crowdsourced study (N=163) examining how robot-initiated touch (present or absent), the robot’s intent (instrumental or affective), robot appearance (Nao or Stretch), and robot tone (empathetic or serious) impact the perceived quality of care. Results show that participants prefer instrumental to affective touch, view the robot as having greater social attributes (higher warmth, higher competence, and lower discomfort) after robot-initiated touch, are more comfortable interacting with the human-like Nao than the more machine-like Stretch, and favor consistent robot tone and appearance. From these results, we derived three design guidelines for caregiving robots in healthcare settings.

Learning Personalized Human-Aware Robot Navigation Using Virtual Reality Demonstrations from a User Study

Social robots are often critiqued as being too ‘robotic’ and unemotional. For affective human-robot interaction (HRI), robots must detect sentiment and express emotion and empathy in return. We explored the extent to which people can detect emotions, empathy and sentiment from speech expressed by a computer system, with a focus on changes in prosody (pitch, tone, volume) and how people identify sentiment from written text, compared to a sentiment analyzer. 89 participants identified empathy, emotion and sentiment from audio and text embedded in a survey. Empathy and sentiment were best expressed in the audio, while emotions were the most difficult detect (67%, 67% and 42% respectively). We found moderate agreement (70%) between the sentiment identified by the participants and the analyzer. There is potential for computer systems to express affect by using changes in prosody, as well as analyzing text to identify sentiment. This may help to further develop affective capabilities and appropriate responses in social robots, in order to avoid ‘robotic’ interactions. Future research should explore how to better express negative sentiment and emotions, while leveraging multi-modal approaches to HRI.

ClassMate Robot: A Robot to Support Teaching and Learning Activities in Schools

During the last years, many solutions have been proposed to achieve a natural Human-Robot Interaction (HRI) and Communication paving the way to new paradigms of understanding and adaptation based on mutual affective perception.
Especially in human-robot social interaction, it is helpful not only that people can understand the robot’s behavioral state, but also robots possess the ability to detect, interpret and adaptively react to human affective responses. Typical approaches are able to assess humans’ affective responses from the observation of overt behavior. However, there are cases in which the overt observable behaviors could not match with the internal states (e.g., people with diseases compromising normal emotional responses). In such cases, having an objective measure of the users’ state from ‘inside’ is of paramount importance. This work presents an affect detection model able to provide a measure of the human affective state, with particular focus on the stress state, from the analysis of EEG users’ activity during the interaction with a social humanoid robot endowed with diverse affective elicitation behaviors. We argue that monitoring the stress state of a human during HRI is necessary to adapt the robot behavior in a way to avoid possible counterproductive effects of its use.

Enhancing Affective Robotics via Human Internal State Monitoring

Dexterous in-hand manipulation is a peculiar and useful human skill. This ability requires the coordination of many senses and hand motion to adhere to many constraints. These constraints vary and can be influenced by the object characteristics or the specific application. One of the key elements for a robotic platform to implement reliable in-hand manipulation skills is to be able to integrate those constraints in their motion generations. These constraints can be implicitly modelled, learned through experience or human demonstrations. We propose a method based on motion primitives dictionaries to learn and reproduce in-hand manipulation skills. In particular, we focused on fingertip motions during the manipulation, and we defined an optimization process to combine motion primitives to reach specific fingertip configurations. The results of this work show that the proposed approach can generate manipulation motion coherent with the human one and that manipulation constraints are inherited even without an explicit formalization.

In-hand manipulation planning using human motion dictionary

Deep Learning (DL) based methods for object detection achieve remarkable performance at the cost of computationally expensive training and extensive data labeling. Robots embodiment can be exploited to 
mitigate this burden by acquiring automatically annotated training data via a natural interaction with a human showing the object of interest, handheld. However, learning solely from this data may introduce biases (the so-called domain shift), and prevents adaptation to novel tasks. While Weakly-supervised Learning offers a well-established set of techniques to cope with these problems in general-purpose Computer Vision, its adoption in challenging robotic domains is still at a preliminary stage. In this work, we target the scenario of a robot trained in a teacher-learner setting to detect handheld objects. The aim is to improve detection performance in different settings by letting the robot explore the environment with a limited human labeling budget. We compare several techniques for WSL in detection pipelines to reduce model re-training costs without compromising accuracy,
proposing solutions which target the considered robotic scenario. We show that the robot can improve adaptation to novel domains, either by interacting with a human teacher (Active Learning) or with an autonomous supervision (Semi-supervised Learning). We integrate our strategies into an on-line detection method, achieving efficient model update capabilities with few labels. We experimentally benchmark our method on challenging robotic object detection tasks under domain shift.

From Handheld to Unconstrained Object Detection: a Weakly-supervised On-line Learning Approach

Finding effective grasp strategies constitutes one of the main challenges in robotic manipulation, especially when
using soft, underactuated, and non-anthropomorphic hands.

Learning Grasping Strategies for a Soft Non-Anthropomorphic Hand from Human Demonstrations

We develop and evaluate two human-robot handover controllers that allow end-users to specify timing parameters for the robot reach motion, and that provide feedback if the robot cannot satisfy those constraints. 
End-user tuning with feedback is a useful controller feature in settings where robots have to be re-programmed for varying task requirements but end-users do not have programming knowledge. 
The two controllers we propose are both receding-horizon controllers that differ in their objective function, and their user specified parameters, and subsequently their user-interface: 
One controller uses a minimum cumulative jerk (MCJ) objective function, and the other a minimum cumulative error (MCE) objective function. 
We implemented the controllers on a collaborative robot and conducted two controlled experiments to compare the user experience and performance of these controllers vis-à-vis a baseline proportional velocity (PV) controller. 
In each experiment, participants ($n=30$) interactively tuned the controller parameters, and collaborated with a robot to perform a time-constrained repetitive task. 
We found that the timing controller with the MCE implementation can provide a better user experience, both while setting the parameters ($p=0.011$) and performing the handovers with the robot ($p<0.001$), and fewer failures ($p=0.016$) compared to the PV controller, however the MCJ implementation did not provide better user experience compared to the PV controller.
The MCJ controller also resulted in more failures than the PV controller.
These results could inform the design of usable and effective end-user configurable controllers for human-robot interaction.

Transparent Learning from Demonstration for Robot-Mediated Therapy

Humanoid robots provide a unique opportunity for personalized interaction using emotion recognition. However, emotion recognition performed by humanoid robots in complex social interactions is limited in the flexibility of interaction as well as personalization and adaptation in the responses. We designed an adaptive learning system for real-time emotion recognition that elicits its own ground-truth data and updates individualized models to improve performance over time. A Convolutional Neural Network based on off-the-shelf ResNet50 and Inceptionv3 are assembled to form an ensemble model which is used for real-time emotion recognition through facial expression. Two sets of robot behaviors, general and personalized, are developed to evoke different emotion responses. The personalized behaviors are adapted based on user preferences collected through a pre test survey. The performance of the proposed system is verified through a 2-stage user study and tested for the accuracy of the self-supervised retraining. We also evaluate the effectiveness of the personalized behavior of the robot in evoking intended emotions between stages using trust, empathy and engagement scales. The participants are divided into two groups based on their familiarity and previous interactions with the robot. The results of emotion recognition indicate a 12% increase in the F1 score for 7 emotions in stage 2 compared to pre-trained model. Higher mean scores for trust, engagement, and empathy are observed in both participant groups. The average similarity score for both stages was 82% and the average success rate of eliciting the intended emotion increased by 8.28% between stages, despite their differences in familiarity thus offering a way to mitigate novelty effect patterns among user interactions.

Premium content

Downloads

Next from IEEE RO-MAN 2022

From Task Analysis to Wireframe Design: An Approach to User-Centered Design of a GUI for Mobile HRI at Assembly Workplaces

Similar lecture

Robot Facial Expression Framework for Enhancing Empathy in Human-Robot Interaction