United States

Co-speech gesture generation on artificial agents has gained attention recently, mainly when it is based on data-driven models. However, end-to-end methods often fail to generate co-speech gestures related to semantics with specific forms, i.e., Symbolic and Deictic gestures. In this work, we identify which words in a sentence are contextually related to Symbolic and Deictic gestures. Firstly, we appropriately chose 12 gestures recognized by people from the Italian culture, which different humanoid robots can reproduce. Then, we implemented two rule-based algorithms to label sentences with Symbolic and Deictic gestures. The rules depend on the semantic similarity scores computed with the RoBerta model between sentences that heuristically represent gestures and sub-sentences inside an objective sentence that artificial agents have to pronounce. We also implemented a baseline algorithm that assigns gestures without computing similarity scores. Finally, to validate the results, we asked 30 persons to label a set of sentences with Deictic and Symbolic gestures through a Graphical User Interface (GUI), and we compared the labels with the ones produced by our algorithms. For this scope, we computed Average Precision (AP) and Intersection Over Union (IOU) scores, and we evaluated the Average Computational Time (ACT). Our results show that semantic similarity scores are useful for finding Symbolic and Deictic gestures in utterances.

IEEE RO-MAN 2024

Labeling Sentences with Symbolic and Deictic Gestures Via Semantic Similarity

gesture identification

semantic similarity

large language models

[![](https://assets.underline.io/uploads/markdown_image/1/image/81a3c0317d24f663b49b996875024d45.png)](https://aclanthology.org/volumes/2022.coling-1/)



### THE CONFERENCE WE KNOW AND WE WANT.
**COLING**, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes but also in industrial research departments including tech-startups. COLING provides opportunities for all these communities to showcase their exciting discovery.

In fall of 2022, COLING will be held in Gyeongju in a hybrid format. All participants can either present at the venue site or join virtually. As more people get vaccinated, we are happy to provide safer environments for our colleagues. We believe that COLING 2022 will be one of the conferences, free from the pandemic. The hybrid format gives presenters and sponsors a valuable opportunity to promote their companies in both an online and in-person venue. For the first time in a long time, customers can interact with their sponsor's products first-hand. The online venue, too, gives sponsors the chance to network with those unable to attend the in-person session.

The hybrid format ultimately lends itself to greater exposure for our sponsors: COLING2022 will let you reach more potential partners and customers than ever before!

To gain access to this event page you are required to register. Find more information and pay the registration fee on event the organizer’s website **[https://coling2022.org/reg](https://coling2022.org/reg)**

COLING 2022

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

technical paper

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

Using multiple robots in presentation effectively attracts and provides information to people compared to a single robot. Many robotics researchers employed this approach to realize effective robot-presentation systems in real environments. These systems implicitly equalize the amount of speech in conversational content between multiple robots; however, it is not known whether that rule is actually important. For example, the effects of the second robot that provides only a back-channel have not yet been investigated. If we unveil the relationships between the presentation effects and the amount of speech in multiple robot presentations, it is crucial in designing the content of multiple robots' conversations. For this purpose, we conducted a video-based web-survey. Our finding showed that product recommendations by multiple robots make them more positive, and a part of the advantages of equalizing the amount of speech compared to biasing the amount of speech in presentation.

Two Is Better Than One: Explanations by Multiple Robots Strengthen the Feeling of Kawaii Toward Objects

When designing the motions of wearable robotic systems, physical Human-Robot Interaction (pHRI) is a crucial consideration. This study suggests a coordinated motion planner for Supernumerary Robotic Limbs (SRLs) to minimize the applied moment from the robotic systems operation to the human body. Given task motion trajectories, moment compensation motions are generated to reduce the asymmetric moment elements while maintaining the other element below a nominal value. The performance of the motion planner is evaluated by the simulation and the hardware experiment. An object pick-and-place scenario is used as a goal task. The simulation results demonstrate that under different conditions, the compensation motions significantly decrease the generated roll and yaw moments when compared to the results without the motions. The hardware experiment validates its practicality as a real-time motion controller of a physical robotic system.

Strategies for Moment Compensation in Supernumerary Robotic Limbs Manipulation Tasks

Vascular dysfunction, trauma, and other disorders often cause a loss in mobility. Brain-computer-aided prosthetics have the potential to help restore critical motor functions and carry out normal daily activities. Through brain-computer interfaces (BCI), electroencephalogram (EEG) brain signals can be used to generate prosthetic gesture commands and control external devices. However, due to the complexity posed by noise in EEG signals, it can be difficult to extract useful features and classify a wide range of motor tasks. In this study, an EEG Motor Movement and Imagery Dataset collected by the New York State Department of Health using a BCI2000 system, is used to perform detailed motor task classification. This dataset includes both real and imagined motions across six different tasks. Convolutional neural networks (CNN) are built to classify six tasks, ultimately leading to binary classification between real and imagined movements, together with motion and rest. The latter models achieve accuracies of 91.2% and 74.4%, respectively. Our approach has the potential for broader applications where comprehensive classifications are needed to discern multiple prosthetic movement tasks.

Multi and Binary Task Classification for Real and Imagined Motor Movements Using EEG Signals

As robots become more and more integrated in human spaces, it is increasingly important for them to be able to explain their decisions to the people they interact with. These explanations need to be generated automatically and in real-time in response to decisions taken in dynamic and often unstructured environments. However, most research in explainable human-robot interaction only considers explanations (often manually selected) presented in controlled environments. We present an explanation generation method based on counterfactuals and demonstrate its use in an ``in-the-wild&quot; experiment using automatically generated and selected explanations of autonomous interactions with real people to assess the effect of these explanations on participants' ability to predict the robot's behaviour in hypothetical scenarios. Our results suggest that explanations aid one's ability to predict the robot's behaviour, but also that the addition of counterfactual statements may add some burden and counteract this beneficial effect.

What Would I Do If...? Promoting Understanding in HRI through Real-Time Explanations in the Wild

Bio-inspired robotic systems are capable of adaptive learning, scalable control, and efficient information processing. Enabling real-time decision-making for such systems is critical to respond to dynamic changes in the environment. We focus on dynamic target tracking in open areas using a robotic six-degree-of-freedom manipulator with a bird-eye view camera for visual feedback, and by deploying the Neurodynamical Computational Framework (NeuCF). NeuCF is a recently developed bio-inspired model for target tracking based on Dynamic Neural Fields (DNFs) and Stochastic Optimal Control (SOC) theory. It has been trained for reaching actions on a planar surface toward localized visual beacons, and it can re-target or generate stop signals on the fly based on changes in the environment (e.g., a new target has emerged, or an existing one has been removed). We evaluated our system over various target-reaching scenarios. In all experiments, NeuCF had high end-effector positional accuracy, generated smooth trajectories, and provided reduced path lengths compared with a baseline cubic polynomial trajectory generator. In all, the developed system offers a robust and dynamic-aware robotic manipulation approach that affords real-time decision-making.

Adaptive Environment-Aware Robotic Arm Reaching Based on a Bio-Inspired Neurodynamical Computational Framework

With the latest developments in sensors, battery, and Artificial Intelligence (AI) technologies, robots can perform missions in unstructured environments (e.g., disaster sites). However, their adaptation speed is still slow due to their limited onboard computing capability and dynamic disruption from environments, leading to inaccurate planning and delayed emergency reactions. Even though reinforcement learning increases robots' exploration speed by fusing effective guidance, the numerous interactions make the learning time-consuming and even risky (e.g., collisions). To fundamentally improve robot adaptation speed, this work seeks help from quantum power. A novel Quantum Exploration based Dreamer model (QED) was developed to facilitate reinforcement learning explorations. QED based on stochastic quantum walker quickly explores environments, evaluates action quality, and obtains a global optimal exploration strategy; then, these high-quality exploration samples will be used to facilitate reinforcement learning speed. A theoretical benefit of QED is facilitating reinforcement learning speed without changing the underlying learning architecture, which makes the proposed QED applicable to general robot learning scenarios. To validate QED effectiveness, a robot path-planning task in an obstacle-dense environment was designed. The number of needed training episodes validated the effectiveness. The results show that QED achieves around ten times faster policy learning compared with Monte Carlo tree-based reinforcement learning methods and vanilla reinforcement learning methods.

Quantum Exploration-Based Reinforcement Learning for Efficient Robot Path Planning in Sparse-Reward Environment

This paper presents a new software framework for HRI experimentation with the sixth version of the common NAO robot produced by the United Robotics Group. Embracing the common demand of researchers for better performance and new features for NAO, the authors took advantage of the ability to run ROS2 onboard on the NAO to develop a framework independent of the APIs provided by the manufacturer. Such a system provides NAO with not only the basic skills of a humanoid robot such as walking and reproducing movements of interest but also features often used in HRI such as: speech recognition/synthesis, face and object detention, and the use of Generative Pre-trained Transformer (GPT) models for conversation. The developed code is therefore configured as a ready-to-use but also highly expandable and improvable tool thanks to the possibilities provided by the ROS community. The code repository is: https://github.com/antbono/OAN

Open Access NAO (OAN): A ROS2-Based Software Framework for HRI Applications with the NAO Robot

People struggle to form accurate expectations of robots because we typically associate behavior (and capability) with the physical entity even when there are clear indicators of different software programs dictating behavior at different times. This is a harmful prior, as commercially available, visibly similar robots do not necessarily share any common ground in terms of capability, safety, or behavior. Prior efforts to calibrate peoples expectations of robots have not extended to anchoring on the robots control software rather than its embodiment. In this work, we leverage social participation and flexible identity presentation to facilitate coworkers' associations of robot capability with the currently running software rather than physical entity itself. By linking each of a robot's controllers to a social identity, we enable collaborators to more easily differentiate between them. In a human subjects study (n=30), participants who experienced our social identity signal understood differences between the robot's two controllers and prevented an unreliable controller from harming perceptions of the robot's other controller.

Robot Social Identity Performance Facilitates Contextually-Driven Trust Calibration and Accurate Human Assessments of Robot Capabilities

HRI research using autonomous robots in real-world settings can produce results with the highest ecological validity of any study modality, but many difficulties limit such studies feasibility and effectiveness. We propose VID2REAL HRI, a research framework to maximize real-world insights offered by video-based studies. The VID2REAL HRI framework was used to design an online study using first-person videos of robots as real-world encounter surrogates. The online study (n = 385) distinguished the within-subjects effects of four robot behavioral conditions on perceived social intelligence and human willingness to help the robot enter an exterior door. A real-world, between-subjects replication (n = 26) using two conditions confirmed the validity of the online studys findings and the sufficiency of the participant recruitment target (22) based on a power analysis of online study results. The VID2REAL HRI framework offers HRI researchers a principled way to take advantage of the efficiency of video-based study modalities while generating directly transferable knowledge of real-world HRI. Code and data from the study are provided at vid2real.github.io/vid2realHRI.

Vid2RealHRI: Align Video-Based HRI Study Designs with Real-World Settings

In Japan, home robots have become popular and are widely loved. In particular, Sonys aibo users have formed a big community, and a society where humans and robots live in harmony is becoming a reality. In order for robots to be more accepted in the society, exploratory analysis and explicit description of their bonds are important. However, the scale for measuring relationships with robots has not yet been defined, so a suitable measure was created from the Friendship Function Scale to measure the relationship of friends and the Pet Awareness Scale to measure the attitude toward pets. Using the proposed scale, we investigated the role of home robots, subjective intimacy, and attitudes toward robots among 54 robot owners. The survey found that more than half of them considered robots to be pets or children. In addition, the average value of subjective intimacy was very high to indicate a very high level of attachment to the robot. Furthermore, as a result of factor analysis of the scale items, five factors were extracted: Comfort, Irreplaceability, Trust from the robot, Trust to the robot, and Social life. At this time, there was no bias in how users felt about comfort and irreplaceability, but there was a bias in how users felt about trust. Additionally, those with the highest level of subjective intimacy felt significantly more trusted by the robot than those with the other level of intimacy, suggesting that recognition of trust from the robot may be an important bond-forming strategy. In the future, we will aim to form further bonds by investigating relationships with robots that play various roles, as well as in regions other than Japan.

Premium content

Downloads

Next from IEEE RO-MAN 2024

Two Is Better Than One: Explanations by Multiple Robots Strengthen the Feeling of Kawaii Toward Objects

Similar lecture

Towards Understanding the Relation between Gestures and Language