United States

In real-world scenarios, robots come closer to humans in many applications, sharing the same workspace or even manipulating the same objects. To ensure safe and intuitive collaboration, it is crucial to have an accurate knowledge of the human&#39;s 3D position in space, which should be estimated with high precision, high frequency and low latency. However, individual sensors such as inertial measurement units (IMUs) or cameras cannot meet all requirements for reliable human pose estimation under conditions such as long operating times, large distances and occlusions. In this study, we highlight the limitations of different visual pose methods and present a fused approach for real-time estimation of the 3D position of the human pelvis using machine learning-based visual pose from a monocular camera and an IMU sensor suit. The multimodal fusion is based on the Invariant Extended Kalman filter (InEKF) on Lie Groups, which fuses drift-free visual poses with high-frequency inertial measurements in a loosely-coupled manner. The evaluation is performed on a recorded dataset of multiple subjects performing various experimental scenarios. The results show that the fused approach can increase the accuracy and robustness of the estimates, taking a step closer towards smooth human-robot collaboration.

IEEE RO-MAN 2024

Fusion of Inertial Sensor Suit and Monocular Camera for 3D Human Pelvis Pose Estimation

imu sensor suit

visual pose

3d human pose estimation

kalman filter

sensor fusion

machine learning

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


ICPR 2020

In real-world scenarios, robots come closer to humans in many applications, sharing the same workspace or even manipulating the same objects. To ensure safe and intuitive collaboration, it is crucial to have an accurate knowledge of the human's 3D position in space, which should be estimated with high precision, high frequency and low latency. However, individual sensors such as inertial measurement units (IMUs) or cameras cannot meet all requirements for reliable human pose estimation under conditions such as long operating times, large distances and occlusions. In this study, we highlight the limitations of different visual pose methods and present a fused approach for real-time estimation of the 3D position of the human pelvis using machine learning-based visual pose from a monocular camera and an IMU sensor suit. The multimodal fusion is based on the Invariant Extended Kalman filter (InEKF) on Lie Groups, which fuses drift-free visual poses with high-frequency inertial measurements in a loosely-coupled manner. The evaluation is performed on a recorded dataset of multiple subjects performing various experimental scenarios. The results show that the fused approach can increase the accuracy and robustness of the estimates, taking a step closer towards smooth human-robot collaboration.

technical paper

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

Frailty assessment plays a pivotal role in providing older adults care. However, the current process is time-consuming and only measures patients' completion time for each test. This paper introduces a set of algorithms to be used in robots to autonomously perform frailty assessments. In doing so we aim at reducing therapists' burden and provide additional frailty-related metrics that can enhance the effectiveness of diagnosis. We conducted a pilot study with 22 elderly participants and compared our system's performance with that of medical professionals to assess its precision. The results demonstrate that our approach achieved performances close to that of its human counterpart. This research represents an important step forward in the integration of social robotics in healthcare, offering potential benefits for patient care and clinical decision-making.

Exploring the Potential of a Robot-Assisted Frailty Assessment System for Elderly Care

This study explores the complex dynamics between humans and robots, focusing on how emotional states influence proxemics. We conducted a user study using a standard mobile robot to investigate whether emotions elicited from a loudspeaker, affect human perception of robot proximity. Based on previous research on Human-Human Interaction (HHI), we analysed participants responses to robots displaying different behaviours. Participants observed the robots approach while experiencing positive or negative emotions. Our findings suggest that emotional states induced by external stimuli can affect participants perception of robot proximity. In detail, the results indicate that while comfortable stopping distances were unaffected by participants emotional state, individuals who experienced positive emotions judged the same proxemics distance used while performing an avoidance behaviour to be more acceptable compared to the case of negative emotions. This study describes the extent to which our emotions can alter the perception of robot behaviours, ultimately affecting our acceptance of these novel social agents.

Are Emotions Important? a Study on Social Distances for Path Planning Based on Emotions

The development of social robots with advanced conversational capabilities aiming to engage humans in natural interactions has recently surged. This paper investigates the dynamic aspect of Human-Robot Interaction (HRI), focusing on the regulation of interpersonal distance based on human emotion. Through a user study with a humanoid robot, we explore how participants perceive and respond to rule-based versus randomly generated robot behaviours in adjusting interpersonal space during an unconstrained conversation. Results suggest that participants perceive the robot using rule-based behaviours as more socially competent and adaptable to human behaviour and emotions compared to the random ones. These findings highlight the importance of considering subtle nonverbal cues and adapting robot behaviour based on human emotions to improve the quality of HRI, and consequently facilitate the successful integration of human natural nuances in robots.

Too Close to You? a Study on Emotion-Adapted Proxemics Behaviours

Complex manipulation tasks often require robots with complementary capabilities to collaborate. We introduce a benchmark for LanguagE-Conditioned Multi-robot MAnipulation (LEMMA) focused on task allocation and long-horizon object manipulation based on human language instructions in a tabletop setting. LEMMA features 8 types of procedurally generated tasks with varying degree of complexity, some of which require the robots to use tools and pass tools to each other. For each task, we provide 800 expert demonstrations and human instructions for training and evaluations. LEMMA poses greater challenges compared to existing benchmarks, as it requires the system to identify each manipulators limitations and assign sub-tasks accordingly while also handling strong temporal dependencies in each task. To address these challenges, we propose a modular hierarchical planning approach as a baseline. Our results highlight the potential of LEMMA for developing future language-conditioned multi-robot systems.

LEMMA: Learning Language-Conditioned Multi-Robot Manipulation

Recent advancements in medical science and technology have led to a remarkable increase in the lifespan of the global elderly population. However, this demographic often struggles with mobility issues, leading to a sedentary lifestyle fueled by concerns over their physical capabilities. Traditional treadmill gait training, although beneficial, often becomes monotonous and lacks the real-world feedback necessary for engaging and effective rehabilitation. Addressing this gap, research into virtual walking systems utilizing avatar robots has gained traction. Despite the progress, several challenges remain where the systems prioritize visual feedback without considering the crucial need for alerting users to potential dangers and obstacles. This lack of comprehensive environmental awareness and feedback undermines both user engagement and safety. To address this problem, this paper proposes an algorithm that employs B-splines for precise free space detection, integrated with a safety stop mechanism for an avatar robot. This novel approach enhances user awareness of their surroundings through a sophisticated Graphic User Interface (GUI) that leverages Augmented Reality (AR) technology. By superimposing free space boundaries and warning messages directly onto a real-time camera feed, the system provides an intuitive and immersive navigation aid. The efficacy of our proposed GUI was rigorously tested across a series of realistic scenarios, comparing teleoperation control performance with and without the augmented interface. Our findings reveal that our GUI markedly enhances user safety and navigational effectiveness, fostering a deeper understanding of and interaction with the surrounding environment, thereby redefining user experience in virtual mobility assistance.

A B-Spline Approach for Improved Environmental Awareness in Virtual Walking System Using Avatar Robot

Research on underwater human-robot teaming is particularly promising because of complementary sensory skills that may overcome unique challenges associated with the combination of limited connectivity and low visibility. Nevertheless, testing human-robot collaboration under water, especially in complex, real-world scenarios, poses severe safety, cost, and time constraints that significantly hinder academic research in this space. This paper presents a novel cyber-physical extended-reality (XR) testbed, referred to as Underwater Real-Time Human Autonomous Systems Collaboration (UnRealTHASC), designed to enable human-robot interactions in simulated photorealistic environments with mixed real and virtual wearables, and advanced autonomous underwater sensors in-the-loop. Novel sensor interfaces are designed to integrate real and virtual sensors for measuring physiological and cognitive human states underpinning decision-making abilities. Physics-based human and robot motion models are developed along with new sensor simulations in order to capture the couplings between underwater behaviors and perception based on measurements from optical and sonar sensors. Real-time data acquisition pipelines are created to access and share data from both real and virtual sensors and robots, such that new methods for online planning and collaboration may be tested via human-in-the-loop demonstrations.

UnRealTHASC – a Cyber-Physical XR Testbed for Underwater Real-Time Human Autonomous Systems Collaboration

Quadruped robots are showing impressive abilities to navigate the real world. If they are to become more integrated into society, social trust in interactions with humans will become increasingly important. Additionally, robots will need to be adaptable to different humans based on individual preferences. In this work, we study the social interaction task of learning optimal handshakes for quadruped robots based on user preferences. While maintaining balance on three legs, we parameterize handshakes with a Central Pattern Generator consisting of an amplitude, frequency, stiffness, and duration. Through 10 binary choices between handshakes, we learn a belief model to fit individual preferences for 25 different subjects. Our results show that this is an effective strategy, with 76% of users feeling happy with their identified optimal handshake parameters, and 20% feeling neutral. Moreover, compared with random and test handshakes, the optimized handshakes have significantly decreased errors in amplitude and frequency, lower Dynamic Time Warping scores, and improved energy efficiency, all of which indicate robot synchronization to the user's preferences

Learning Human-Robot Handshaking Preferences for Quadruped Robots

To facilitate human-robot interaction (HRI) tasks in real-world scenarios, service robots must adapt to dynamic environments and understand the required tasks while effectively communicating with humans. To accomplish HRI in practice, we propose a novel indoor dynamic map, task understanding system, and response generation system. The indoor dynamic map optimizes robot behavior by managing an occupancy grid map and dynamic information, such as furniture and humans, in separate layers. The task understanding system targets tasks that require multiple actions, such as serving ordered items. Task representations that predefine the flow of necessary actions are applied to achieve highly accurate understanding. The response generation system is executed in parallel with task understanding to facilitate smooth HRI by informing humans of the subsequent actions of the robot. In this study, we focused on waiter duties in a restaurant setting as a representative application of HRI in a dynamic environment. We developed an HRI system that could perform tasks such as serving food and cleaning up while communicating with customers. In experiments conducted in a simulated restaurant environment, the proposed HRI system successfully communicated with customers and served ordered food with 90% accuracy. In a questionnaire administered after the experiment, the HRI system of the robot received 4.2 points out of 5. These outcomes indicated the effectiveness of the proposed method and HRI system in executing waiter tasks in real-world environments.

Unified Understanding of Environment, Task, and Human for Human-Robot Interaction in Real-World Environments

This paper addresses the challenge of selecting explanations for XAI (Explainable AI)-based Intelligent Decision Support Systems (IDSSs). IDSSs have shown promise in improving user decisions through XAI-generated explanations along with AI predictions, and the development of XAI made it possible to generate a variety of such explanations. However, how IDSSs should select explanations to enhance user decision-making remains an open question. This paper proposes X-Selector, a method for selectively presenting XAI explanations. It enables IDSSs to strategically guide users to an AI-suggested decision by predicting the impact of different combinations of explanations on a users decision and selecting the combination that is expected to minimize the discrepancy between an AI suggestion and a user decision. We compared the efficacy of X-Selector with two naive strategies (all possible explanations and explanations only for the most likely prediction) and two baselines (no explanation and no AI support) in virtual stock-trading support scenarios. The results suggest the potential of X-Selector to guide users to AI-suggested decisions and improve task performance under the condition of a high AI accuracy.

User Decision Guidance with Selective Explanation Presentation from Explainable-AI

Apology behavior design is becoming important for social robots that work in daily environments because of their widespread use. Robotics researchers have reported that multiple robots can effectively achieve more acceptance and trust in apology situations. Unfortunately, such effects have only been confirmed in a single country: Japan. Some studies investigated cultural differences in apologies between Japan and other cultures and reported how the former influences the perceived function and meaning of apologies. Therefore, we conducted a web-based survey to investigate whether using multiple robots in apology situations is effective in another country. We compared such perceived feelings as forgiveness and trust toward a robots apologies between Japan and the U.S. by using the visual stimuli of one and two robots. The experiment results showed that U.S. people felt that multiple robot apologies are more acceptable than apologies from just one robot, similar to results with Japanese participants. Perceived trust did show a different phenomenon between the two countries.

Premium content

Downloads

Next from IEEE RO-MAN 2024

Exploring the Potential of a Robot-Assisted Frailty Assessment System for Elderly Care

Similar lecture

Can You Trust Your Pose? Confidence Estimation in Visual Localization