United States

The efficiency of collaborative mobile robot applications is influenced by the inherent uncertainty introduced by humans&#39; presence and active participation. This uncertainty stems from the dynamic nature of the working environment, various external factors, and human performance variability. The observed makespan of an executed plan will deviate from any deterministic estimate. This raises questions about whether a calculated plan is optimal given uncertainties, potentially risking failure to complete the plan within the estimated timeframe. This research addresses a collaborative task planning problem for a mobile robot serving multiple humans through tasks such as providing parts and fetching assemblies. To account for uncertainties in the durations needed for a single robot and multiple humans to perform different tasks, a probabilistic modeling approach is employed, treating task durations as random variables. The developed task planning algorithm considers the modeled uncertainties while searching for the most efficient plans. The outcome is a set of the best plans, where no plan is better than the other in terms of stochastic dominance. Our proposed methodology offers a systematic framework for making informed decisions regarding selecting a plan from this set, considering the desired risk level specific to the given operational context.

IEEE RO-MAN 2024

Risk-Aware Planning of Collaborative Mobile Robot Applications with Uncertain Task Durations

stochastic modeling

collaborative robot applications

task planning

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. This theme also touches upon people's expectations and perceptions of robots, as well as other social, cognitive and cultural factors that will influence human-robot interactions. We will have a special session/track on this topic, invite targeted submissions and include plenary speakers who are specialist in this area. We sincerely hope that this focus will help enlighten the design-thinking of all of us as we fabricate and program our own various systems.

To access RO-MAN Virtual 2021, [please register here.](https://ro-man2021.org/registration/)

IEEE RO-MAN 2021

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. 

The efficiency of collaborative mobile robot applications is influenced by the inherent uncertainty introduced by humans' presence and active participation. This uncertainty stems from the dynamic nature of the working environment, various external factors, and human performance variability. The observed makespan of an executed plan will deviate from any deterministic estimate. This raises questions about whether a calculated plan is optimal given uncertainties, potentially risking failure to complete the plan within the estimated timeframe. This research addresses a collaborative task planning problem for a mobile robot serving multiple humans through tasks such as providing parts and fetching assemblies. To account for uncertainties in the durations needed for a single robot and multiple humans to perform different tasks, a probabilistic modeling approach is employed, treating task durations as random variables. The developed task planning algorithm considers the modeled uncertainties while searching for the most efficient plans. The outcome is a set of the best plans, where no plan is better than the other in terms of stochastic dominance. Our proposed methodology offers a systematic framework for making informed decisions regarding selecting a plan from this set, considering the desired risk level specific to the given operational context.

technical paper

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

Visual planning methods are promising to handle complex settings where extracting the system state is challenging. However, none of the existing works tackles the case of multiple heterogeneous agents which are characterized by different capabilities and/or embodiment. In this work, we propose a method to realize visual action planning in multi-agent settings by exploiting a roadmap built in a low-dimensional structured latent space and used for planning. To enable multi-agent settings, we infer possible parallel actions from a dataset composed of tuples associated with individual actions. Next, we evaluate feasibility and cost of them based on the capabilities of the multi-agent system and endow the roadmap with this information, building a capability latent space roadmap (C-LSR). Additionally, a capability suggestion strategy is designed to inform the human operator about possible missing capabilities when no paths are found. The approach is validated in a simulated burger cooking task and a real-world box packing task.

Visual Action Planning with Multiple Heterogeneous Agents

In robotics, understanding human interaction with autonomous systems is crucial for enhancing collaborative technologies. We focus on human-swarm interaction (HSI), exploring how active robot groups of varying sizes affect operators' cognitive and perceptual reactions over distinct durations. We analyze the impact of different numbers of active robots within a 15-robot swarm on operators' time perception, emotional state, flow experience, and task difficulty perception. Our findings indicate that managing multiple active robots when compared to one active robot significantly alters time perception and flow experience, leading to a faster passage of time and increased flow. More active robots and extended durations cause increased emotional arousal and perceived task difficulty, highlighting the interaction between the number of active robots and human cognitive processes. These insights inform the creation of intuitive human-swarm interfaces and aid in developing swarm robotic systems aligned with human cognitive structures, enhancing human-robot collaboration.

From One to Many: How Active Robot Swarm Sizes Influence Human Cognitive Processes

In this paper, we set out to explore how people judge the personality of a non-anthropomorphic virtual agent during group interactions. Using a Wizard of Oz (WoZ) based approach, we observed that people judged the acceptability of a virtual agent's behavior from a tool-based lens, that is, if this robot and its behavior are useful to the team or not. Furthermore, we found that while people were able to acknowledge the virtual agent's personality and recognize its identity through social cues, the tool-based framing impacts these perceptions into a normative judgment of the robots' utility. We present two case studies that we think highlight this tool-based interpretation of robotic personality: robots expressing non-factual opinions and robots expressing humor. Finally, we suggest that researchers should consider the impact of this tool-based framing on people's perceptions of a robot's identity when designing robots for social interaction.

A Tool but Not a Peer: How Framing Affects People's Perceptions of AI Agents in Teams

Humanrobot work collaboration can be comple- mentary where both the robot and employee use their strengths to complete the task at hand. But who is ultimately responsible for the task? Employees may assume more or less responsibility when working with robots, which can have a direct impact on job outcomes. In this paper the authors use the job characteristics model as a framework to investigate changes in job characteristics and their impact on job responsibility and worker outcomes as a result of robot implementation in restaurants. The results identify changes in task significance, job autonomy, and feedback from the job as important predictors of changes in experienced responsibility toward work. The study also found a significant positive relationship between job responsibility and motivation to work.

Working with Robots in Restaurants: Job Responsibility and Work Motivation Changes

In human-robot collaboration, legible intent of the robot is critical to success as it enables the human to more effectively work with and around the robot. Environments where humans and robots collaborate are widely varied and in the real world are most often cluttered. However, prior work in legible motion utilizes primarily environments which are uncluttered. Success in these environments does not necessarily guarantee success in more cluttered environments. Furthermore, the prior work has been primarily performed based on results from robot-human studies and the problem has not been studied from the prospective of what people do to express intent to each other. Therefore, this work addresses a gap in current research into legible robot arm motion in the following ways: first we perform a human-human study in order to establish the factors which humans use to express their intent through body language, and second we perform the study in a cluttered and varied environment. Through the study we showed that the primary factors which people considered are: timing, kinematic parameters, hand gestures, object proximity, etc. The results also showed that legibility is correlated with perceived safety, perceived social intelligence, the collaborators contribution, and trust which further speaks to the importance of legible motion. Future work will utilize the pose data extracted from the study's video recordings to develop a model for legible motion.

Investigating Non-Verbal Cues in Cluttered Environments: Insights Into Legible Motion From Interpersonal Interaction

The human gaze is an important cue to signal intention, attention, distraction, and the regions of interest in the immediate surroundings. Gaze tracking can transform how robots perceive, understand, and react to people, enabling new modes of robot control, interaction, and collaboration. In this paper, we use gaze tracking data from a rich dataset of human motion (THÖR-MAGNI) to investigate the coordination between gaze direction and head rotation of humans engaged in various indoor activities involving navigation, interaction with objects, and collaboration with a mobile robot. In particular, we study the spread and central bias of fixations in diverse activities and examine the correlation between gaze direction and head rotation. We introduce various human motion metrics to enhance the understanding of gaze behavior in dynamic interactions. Finally, we apply semantic object labeling to decompose the gaze distribution into activity-relevant regions.

Human Gaze and Head Rotation During Navigation, Exploration and Object Manipulation in Shared Environments with Robots

Technology-supported learning is an integral part of everyday learning. Adaptive tutoring systems are already widespread, but they often lack the social component of learning. Social robots can provide personalized tutoring as well as social interaction and can thus address this challenge. In this contribution, we present a model to personalize the behavior of a social robot tutor for higher education which has both a theoretical as well as an empirical foundation: Theoretically, it is based on the well-known Self-Determination Theory (SDT) that assumes that intrinsic motivation is strongly related to the fulfillment of three basic human needs, namely the need for autonomy, competence, and relatedness. Empirically it is based on the results of four studies with university students that demonstrate that particularly adaptation on a social level is beneficial. The final model considers the personalization of the robotic tutor's behavior based on the current learning content and its social behavior and also considers a long-term perspective for multi-session learning. Thereby each of the three layers of SDT is addressed in multiple ways. This is one of the first models that contain a long-term perspective and is particularly designed for higher education.

Introducing a Model for (Long-Term) Personalization of the Behavior of a Social Robot Tutor Based on Self-Determination Theory and Empirical Findings

Advanced by rich perception and precise execution, robots possess immense potential to provide professional and customized rehabilitation exercises for patients with mobility impairments caused by strokes. Autonomous robotic rehabilitation significantly reduces human workloads in the long and tedious rehabilitation process. However, training a rehabilitation robot is challenging due to the data scarcity issue. This challenge arises from privacy concerns (e.g., the risk of leaking private disease and identity information of patients) during clinical data access and usage. Data from various patients and hospitals cannot be shared for adequate robot training, further compromising rehabilitation safety and limiting implementation scopes. To address this challenge, this work developed a novel federated joint learning (FJL) method to jointly train robots across hospitals. FJL also adopted a long short-term memory network (LSTM)-Transformer learning mechanism to effectively explore the complex tempo-spatial relations among patient mobility conditions and robotic rehabilitation motions. To validate FJL's effectiveness in training a robot network, a clinic-simulation combined experiment was designed. Real rehabilitation exercise data from 200 patients with stroke diseases (upper limb hemiplegia, Parkinson's syndrome, and back pain syndrome) were adopted. Inversely driven by clinical data, 300,000 robotic rehabilitation guidances were simulated. FJL proved to be effective in joint rehabilitation learning, performing 20% - 30% better than baseline methods.

Federated Joint Learning of Robot Networks in Stroke Rehabilitation

Robots that make use of multi-fingered or fully anthropomorphic end-effectors can engage in highly complex manipulation tasks. However, the choice of a suitable grasp for manipulating an object is strongly influenced by factors such as the physical properties of an object and the intended task. This makes predicting an appropriate grasping pose for carrying out a concrete task notably challenging. At the same time, current grasp type predictors rarely consider the task as a part of the prediction process. This work proposes a learning model that considers the task in addition to an object's visual features for predicting a suitable grasp type. Furthermore, we generate a synthetic dataset by simulating robotic grasps on 3D object models based on the BarrettHand end-effector. With an angular similarity of 0.9 and above, our model achieves competitive prediction results compared to grasp type predictors that do not consider the intended task for learning grasps. Finally, to foster research in the field, we make our synthesized dataset available to the research community.

Incorporation of the Intended Task into a Vision-Based Grasp Type Predictor for Multi-Fingered Robotic Grasping

It is crucial that users are empowered to use the functionalities of a robot to creatively solve problems on the fly. A user who has access to a Reinforcement Learning (RL) based robot may want to use the robot's autonomy and their knowledge of its behavior to complete new tasks. One technique is for the user to take control of some of the robot's action space through teleoperation, allowing the RL policy to simultaneously control the rest. We formalize this type of shared control as Partitioned Control (PC). However, this may not be possible using an out-of-the-box RL policy. For example, a user's control may bring the robot into a failure state from the policy's perspective, causing it to act unexpectedly and hindering the success of the user's desired task. In this work, we formalize this problem and present Imaginary Out-of-Distribution Actions, IODA, an initial algorithm which empowers users to leverage their expectations of a robot's behavior to accomplish new tasks. We deploy IODA in a user study with a real robot and find that IODA leads to both better task performance and a higher degree of alignment between robot behavior and user expectation. We also show that in PC, there is a strong and significant correlation between task performance and the robot's ability to meet user expectations, highlighting the need for approaches like IODA. Code is available at https://github.com/AABL-Lab/ioda_roman_2024

Premium content

Downloads

Next from IEEE RO-MAN 2024

Visual Action Planning with Multiple Heterogeneous Agents

Similar lecture

Pedestrian Density-Based Path Recognition and Risk Prediction for Autonomous Vehicles