Italy

The goal of this study is to develop an autonomous conversational robot that acquires knowledge about human society in the real world. In this study, we define experience data as image data obtained from a camera (visual information) and dialogue data obtained from a microphone (auditory information). From experience data, the robot acquires knowledge about the co-occurrence between humans and items, that is, whether it is usual or not for humans to have the items. Not only does the proposed system acquire knowledge, but also generates utterances based on acquired knowledge. We have implemented the proposed method on an android, a humanlike robot. We conducted an experiment in which the android was placed in a real-world environment (shopping mall) and interacted with visitors. The percentage of positive responses to the robot’s questions based on the acquired knowledge suggests that this system can acquire knowledge from experience data.

IEEE RO-MAN 2022

Theory of Mind and Delegation to Robotic Virtual Agents

**31st IEEE International Conference on Robot & Human Interactive Communication**
 This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.
 
 The conference covers a wide range of topics related to Robot and Human Interactive Communication, involving theories, methodologies, technologies, empirical and experimental studies. Papers related to the study of robotic technology, psychology, cognitive science, artificial intelligence, human factors, ethics and policies, interaction-based robot design and other topics related to human-robot interaction are welcome.
 
 **CONFERENCE THEME:**
 Social, asocial, and antisocial robots
 
 Robots are starting to populate our lives: at work, in our homes, hotels, shops. However, not everybody is comfortable with robots finding a place in society. For example, people may be afraid that robots will replace humans in their jobs. This belief is exacerbated by the media that, on one side, raise expectations of robots’ capabilities, while amplifying fears and worries on the other side.
 
 In the next years, due to the increasing request for interaction with people in social environments, academic research and market analyses predict that robots will become more and more “social”. But, at the same time, robots may incur the risk of being perceived as “asocial” as their pervasiveness in everyday environments may become an obstacle to social interaction between humans. Or they may even be rejected as “antisocial” if researchers do not take care to develop robots that behave according to the customs and habits of the communities where they are expected to operate.
 
 To prevent this from happening, robotic researchers can no longer take the luxury of being mere technicians. They must be capable of addressing all the challenges raised by future robot-populated communities in a multidisciplinary way. They need to share their knowledge with the public to make people aware of what robots really are, what expectations we can have from them, the risks we may incur, and the possible mitigation plans.
 
 While being open to any kind of contribution related to Human-Robot Interactive Communication, the RO-MAN 2022 conference will emphasize these aspects of robotic research. How can we create robots capable of being welcomed as our assistants and companions in society while, at the same time, not being perceived as “asocial” or “antisocial?” How can we create robots perceived as facilitators of the natural development of human beings and communities, rather than obstacles?
 
 Contributions from different disciplines, not exclusively related to technological development, will be welcomed as crucial to pave the way to global acceptance of future robots with an informed attitude.
 
 [![](https://assets.underline.io/uploads/markdown_image/1/image/af259abe105b3791c498ae1552d75710.png)](https://drive.google.com/file/d/18w73BZPB_JVOgcrcdZDqYpEupwVt4CnU/view?usp=sharing)

A registration is required to access this event page. If you have not already done so, please use the below link to register for the event. 

[Register Here](http://www.smile.unina.it/ro-man2022/welcome-to-ieee-ro-man-2022/)

Registration Is Required

31st IEEE International Conference on Robot & Human Interactive Communication
This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.

technical paper

Patients with neurological gait impairments fear to
fall while performing locomotor tasks like sit to stand, stand to
sit, and level-ground walking without assistance. This prevents
them from participating in daily life. Multi-directional BodyWeight Support (BWS) systems aim to assist training in a safe
environment to overcome this limitation. To ensure the safety
and comfort during training, BWS systems should effectively
and automatically assist patients with forces that support the
desired locomotor task.
Instead of manual switching of controllers through the therapist, we propose a machine learning-based motion onset
recognition model that aims at automatic controller switching
(4 gait related task classes in this paper). In addition to the data
provided by the gait rehabilitation robot FLOAT, data of three
Inertial Measurement Units (IMUs) attached to the sternum
and middle of the outer thighs on 6 healthy participants were
used to predict motion onset of these 4 gait-related tasks.
Four train data sets have been built up from the synchronously
obtained data from the IMUs and the BWS system by dividing
them into observation windows with sizes of 100 ms and 200 ms
and overlap factors of 50% and 90%. From each training
dataset, 108 time-domain features have been extracted, ranked,
and reduced using the Minimum Redundancy Maximum Relevance (MRMR) method from each train set before applying it to
the classifier. The dominant features were applied for training
and comparing four different classifiers. The performance of
the classifiers has been evaluated based on leave one participant
out cross-validation. The ensemble classifier obtained from the
train data set with a window size of 100 ms and 90% overlap
achieved the best performance with an F1 score of 83.7%.

THE LMA12-O Framework for Emotional Robot Eye Gestures

This paper presents a novel quality assessment approach for the prediction of the subjective haptic experience
in time-delayed teleoperation. With the rapid development of haptic technology in remote robot control and virtual reality, new control schemes and hardware systems are developed to
provide high-quality human-in-the-loop teleoperation service. Our subjective experiments indicate that the existing objective quality assessment metrics do not sufficiently correlate with the subjective haptic experience of the users. This gap requires expensive and time-consuming subjective experiments to be conducted. To avoid time-consuming experiments and provide
a fast and accurate subjective experience prediction, we make
an attempt to analyze and explain the mismatch between the subjective and objective haptic signal quality metrics. To this end, extensive subjective experiments and case studies have
been conducted for teleoperation with time delay and haptic data reduction. Based on our experimental results, we propose a quality assessment approach that predicts the subjective quality of experience using multiple objective metrics. For the one-dimensional spring model, the Spearman’s rank order,
Kendall’s rank-order and Pearson’s Linearity correlation coefficient (SROCC, KLOCC, and PLCC) between the predictions of our model and the results of subjective experiment show
remarkable improvement in the correlation between subjective and objective quality assessment.

Human-robot co-manipulation of soft materials: enable a robot manual guidance using a depth map feedback

Theory of Mind (ToM) refers to the ability to make inferences about other's mental states. Such ability is fundamental for human social activities such as empathy, teamwork, and communication. As intelligent agents come to be involved in diverse human-agent teams, they will also be expected to be socially intelligent in order to become effective teammates. In this paper, we describe a computational ToM model which observes team behaviors and infers their mental states in an urban search and rescue (US\&R) task. Our modular ToM model approximates human inference by explicitly representing beliefs, belief updates, and action prediction/generation using Deep Neural Networks (DNNs). To validate our model we compare its performance to the gold standard of human observers asked to make the same inferences. The ToM model proved superior to the average judgments of human observers on all four tests of inference and better than 90th percentile observers on three of the four. While the learning bias provided by modularizing belief and prediction proved sufficient for the simple inferences tested, substantial refinement will be needed to replicate the complex nuanced chains of inference observed in human social interaction.

Theory of Mind Modeling in Search and Rescue Teams

Humans have an immediate perception of shapes and surroundings through their senses and their cognitive capabilities. This innate ability enables the manual inspection of components in manufacturing environments. Trained inspectors combine their senses and handling skills with bespoke non-destructive testing instrumentation. However, manual inspections can be slow for large and/or complex geometries and prone to human factors. Automated non-destructive testing systems have emerged in recent years, to increase data acquisition speed, part coverage and inspection reliability. These tools work well when the robotic inspection takes place in a well-structured environment and an accurate part model is available. However, precisely registering the position of a part in the robot reference system makes the inspection setup very time-consuming. Furthermore, the geometry of a part may differ from its digital model, spoiling the inspection accuracy. This work introduces a new approach that mimics the human perception capability and gives full manipulation autonomy to robotic sensing applications. We use a single robotized sensor to introduce a fully autonomous single-pass geometric and volumetric inspection of complex parts. Our approach can be used to solve some key challenges in quality assurance for Industry 4.0 and can find applicability beyond robotic non-destructive testing.

Mimicking Human Autonomy in Industrial Robotic Enabled Sensing

Theory of Mind is the process of ascribing mental states to other individuals we interact with. It is used for sense-making of the observed actions and prediction of future actions. Previous studies revealed that humans mentalize about artificial agents, but it is not entirely clear how and to what extent. At the same time mentalizing about humans is often influenced by biases such as an egocentric bias. 
We present a study investigating differences in participants' ToM and their susceptibility to an egocentric bias when observing humans vs robots.
The participants observed an autonomous robot, a controlled robot, and a human in the same scenarios. The agents had to find an object in a laboratory. While watching the agents, participants had to make several action predictions as an implicit measure of ToM, potentially revealing an egocentric bias.
At the end, questions about the agent’s responsibility, awareness and strategy were asked. 
The results indicate that while participants generally performed ToM for all types of agents,
both the scenario as well as the agent type appear to influence participants' likelihood of exhibiting an egocentric bias.

Differences and Biases in Mentalizing About Humans and Robots

Simulation tools are essential for robotics research, especially for those domains in which safety is crucial, such as Human-Robot Collaboration (HRC). However, it is challenging to simulate human behaviors, and existing robotics simulators do not integrate functional human models. This work presents Open-VICO, an open-source toolkit to integrate virtual human models in Gazebo focusing on vision-based human tracking. In particular, Open-VICO allows to combine in the same simulation environment realistic human kinematic models, multi-camera vision setups, and human-tracking techniques along with numerous robot and sensor models thanks to Gazebo. The possibility to incorporate pre-recorded human skeleton motion with Motion Capture systems broadens the landscape of human performance behavioral analysis within Human-Robot Interaction (HRI) settings. To describe the functionalities and stress the potential of the toolkit four specific examples, chosen among relevant literature challenges in the field, are developed using our simulation utils: i) 3D multi-RGB-D camera calibration in simulation, ii) creation of a synthetic human skeleton tracking dataset based on OpenPose, iii) multi-camera scenario for human skeleton tracking in simulation, and iv) a human-robot interaction example. The key of this work is to create a straightforward pipeline which we hope will motivate research on new vision-based algorithms and methodologies for lightweight human-tracking and flexible human-robot applications.

Open-VICO: An Open-Source Gazebo Toolkit for Multi-Camera-based Skeleton Tracking in Human-Robot Collaboration

The ability to impute mental states to oneself or others, or Theory of Mind (ToM), has been intrinsically linked
to trust between humans. However, less is known about how a robot mimicking ToM affects users’ trust and behaviour.
We explore this through an online study, where we compare three robot personas in a cooperative maze navigation task: one neutral, one that explains its reasoning in technical terms, and one that mimics ToM. We show that ToM influences human decision-making behaviour and trust in a way that makes it more appropriate with respect to the competencies of the robot. This is key for human-robot collaboration and adoption of robotics moving forward.

Exploring Theory of Mind for Human-Robot Collaboration

In this study, we extend our robot trust model into
a multimodal setting in which the Nao robot leverages audiovisual data to perform a sequential multimodal pattern recalling
task while interacting with a human partner who has different
guiding strategies: reliable, unreliable, and random. Here, the
humanoid robot is equipped with a multimodal auto-associative
memory module to process audio-visual patterns to extract
cognitive load (i.e., computational cost) and an internal reward
module to perform cost-guided reinforcement learning. After
interactive experiments, the robot associates a low cognitive load
(i.e., high cumulative reward) yielded during the interaction
with high trustworthiness of the guiding strategy of the partner.
At the end of the experiment, we provide a free choice to the
robot to select a trustworthy instructor. We show that the robot
forms trust in a reliable partner. In the second setting of the
same experiment, we endow the robot with an additional simple
theory of mind module to assess the efficacy of the instructor
in helping the robot perform the task. Our results show that
the performance of the robot is improved when the robot bases
its action decisions on factoring in the instructor assessment.

Trustworthiness assessment in multimodal human-robot interaction based on cognitive load

Expressiveness is a critical feature for the communication between humans and robots, and it helps humans to better understand and accept a robot. Emotions can be expressed through a variety of modalities: kinesthetic (via facial expression), body posture and gestures, auditory, thus the acoustic features of speech, and semantic, thus the content of what is said. One of the most effective modalities to communicate emotions is through facial expressions. Social robots often show facial expressions with coded animations. However, the robot must be able to express appropriate emotional responses according to the interaction with people. In this work, we consider verbal interactions between humans and robots and propose a system composed of two modules for the generation of facial emotions by recognising the arousal and valence values of a written sentence. The first module, based on Bidirectional Encoder Representations from Transformers, is deployed for emotion recognition in a sentence. The second, an Auxiliary Classifier Generative Adversarial Network, is proposed for the generation of facial movements for expressing the recognised emotion in terms of valence and arousal.

Action Unit Generation through Dimensional Emotion Recognition from Text

This paper examines how conversational agents (CAs) can communicate emotions non-verbally using light as communication modality. Therefore, we manipulated the CA Alexa to demonstrate emotions (joy and sorrow) using different light modalities (Echo Dot ring, a Hue lamp and the combination) with either a congruent verbal context (party or funeral) or no verbal context. In an online study 167 participants evaluated the perceived emotion of Alexa, their own emotional state as well as the perception of Alexa after watching a video with a user interacting with Alexa. Although the perceived emotions of Alexa were not affected by the experimental conditions, the results indicate that the perception of Alexa as well as the user’s emotion is affected by the displayed communication modality. As external light can be used to manipulate the users’ perception of CAs, the findings give relevant implications for the design of CAs.

Premium content

Theory of Mind and Delegation to Robotic Virtual Agents

Downloads

Next from IEEE RO-MAN 2022

THE LMA12-O Framework for Emotional Robot Eye Gestures

Similar lecture

From Handheld to Unconstrained Object Detection: a Weakly-supervised On-line Learning Approach