United States

With the increase of prevalence in autism, the need for efficient public health support has been amplified. Socially-assistive robots (SARs) have been found effective in engaging and interacting with autistic children, however, the perception intelligence during interaction still needs more domain-specific knowledge in terms of understanding children&#39;s behaviors. The Family Observation Schedule-Second Version (FOS-II) is one of the key methods in assessing parent-child interactions in developmental disabilities, yet its manual annotation demands considerable time and effort. This study proposes a multi-modal artificial intelligence (AI) model using video and audio inputs for automated FOS-II annotation. Utilizing advanced deep learning for behavior recognition, this method offers rapid, cost-effective FOS-II scaling. It will thus enhance the capability of socially assistive robots to understand human behaviors and support the advancement of digital health research for children with autism. The visual perception in home settings are most likely based on uncontrolled environments, so it is crucial to develop algorithms that can robustly work with video-log data with uncontrolled quality. Ultimately, it aims to ease the burden on parents and caregivers, streamlining the monitoring and treatment of challenging behaviors in autism.

IEEE RO-MAN 2024

Multimodal and Multi-Lingual Deep Neural Network for Interactive Behavior Style Recognition from Uncontrolled Video-Logs of Children with Autism

children with autism

interactive behavior style recognition

multimodal deep neural network

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

AAAI 2026

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

With the increase of prevalence in autism, the need for efficient public health support has been amplified. Socially-assistive robots (SARs) have been found effective in engaging and interacting with autistic children, however, the perception intelligence during interaction still needs more domain-specific knowledge in terms of understanding children's behaviors. The Family Observation Schedule-Second Version (FOS-II) is one of the key methods in assessing parent-child interactions in developmental disabilities, yet its manual annotation demands considerable time and effort. This study proposes a multi-modal artificial intelligence (AI) model using video and audio inputs for automated FOS-II annotation. Utilizing advanced deep learning for behavior recognition, this method offers rapid, cost-effective FOS-II scaling. It will thus enhance the capability of socially assistive robots to understand human behaviors and support the advancement of digital health research for children with autism. The visual perception in home settings are most likely based on uncontrolled environments, so it is crucial to develop algorithms that can robustly work with video-log data with uncontrolled quality. Ultimately, it aims to ease the burden on parents and caregivers, streamlining the monitoring and treatment of challenging behaviors in autism.

technical paper

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

This qualitative study explored 20 grade 3-5 childrens perceptions of learning STEM vocabulary with an educational social robot. A semi-structured interview protocol was used to elicit childrens perceptions. Interviews were recorded and transcribed into a database reflecting one talk-turn per row (911 talk-turns total). Two coders used emergent coding and constant comparative method to identify talk-turns that reflected childrens perceptions of the assets and drawbacks of learning STEM vocabulary with the robot. Findings identified new assets and drawbacks about the robots instruction, and new drawbacks about the robots speech, which were not explored in previous research. Findings suggest design implications, including designing robots with the capacity for more individualization of instruction and adjustable movement and speech features based on learners preferences.

If a Robot Was Teaching, Then Everybody Would Definitely Like School Better: An Analysis of Grade 3-5 Children's Perceptions of Learning STEM Vocabulary with a Social Educational Robot

Capturing real-world 3D spaces as point clouds is efficient and descriptive, but it comes with sensor errors and lacks object parametrization. These limitations render point clouds unsuitable for various real-world applications, such as robot programming, without extensive post-processing (e.g., outlier removal, semantic segmentation). On the other hand, CAD modeling provides high-quality, parametric representations of 3D space with embedded semantic data, but requires manual component creation that is time-consuming and costly. To address these challenges, we propose a novel solution that combines the strengths of both approaches. Our method for 3D workcell sketching from point clouds allows users to refine raw point clouds using an Augmented Reality (AR) interface that leverages their knowledge and the real-world 3D environment. By utilizing a toolbox and an AR-enabled pointing device, users can enhance point cloud accuracy based on the devices position in 3D space. We validate our approach by comparing it with ground truth models, demonstrating that it achieves a mean error within 1cm - significant improvement over standard LiDAR scanner apps.

Precise Workcell Sketching from Point Clouds Using an AR Toolbox

Developments in the field of social robotics open interesting opportunities for applications in healthcare, education, and services. For this, studying engagement in human-robot interaction (HRI) is crucial for improving the quality of interactive experiences. Questionnaires are powerful in describing voluntary behavior; however, engagement is often an implicit non-voluntary behavior that reaches awareness only once initiated. Inspired by research in cognitive psychology, we propose a behavioral feature to quantify engagement in HRI through the measurement of spontaneous movement and spectral wavelet analysis. For this, we conducted an experiment during which participants listened to sad stories narrated by a moving social robot. Throughout the experiment, we tracked the participants' spontaneous and non-voluntary sway movements with a motion capture system. The experiments were conducted with three robotic platforms (Buddy, Pepper, and Nao). Results showed that spontaneous body sway can be modulated by social robots within no-goal-oriented interaction. This opens up the application possibility of our methodology to various modes of HRI such as collaboration.

Exploring Engagement in Human-Robot Interaction through the Quantification of Human Spontaneous Movement

Socially Assistive Robots (SARs) represent a valid support to professional caregivers in providing care to person with need. To improve the quality of the interaction, SARs should be able to automatically assess user engagement. In this work, we addressed this problem by investigating user engagement dynamics during a robot-to-human handover task, considering 3 main components of engagement: affective, cognitive, and behavioral. For this study, we automatically extracted 10 visual features from the camera recordings of 31 participants. Each individual engaged in eight consecutive sessions with a robot manipulator designed with social cues. Our statistical analysis indicates that prolonged interaction with the robot could influence user engagement. Namely, we observed a decrease in positive emotions (affective), a more regulated quantity of motion (behavioral), and a reduced attention on the robot tasks (cognitive). Overall, the results of this study suggests that engagement dynamics can be described by the selected behavioral features, and that the a more predictable robots behavior could negatively influence user engagement.

Investigating User Engagement Dynamics in Robot-To-Human Handovers with a Social Manipulator

HRI research has evolved to take a broader, more inclusive view of how culture influences our interaction with robots. As we delve deeper into cultural integration in HRI, it has become evident that while integrating cultural aspects offers new opportunities, it requires careful consideration due to the heightened sensitivity to the fluid nature of cultural dynamics. Our study focuses on a particular case and examines the role of context and personal preferences in a restaurant setting. We investigate how preferences for cross-cultural greetings performed by a humanoid robot can change based on the restaurant theme and describe what factors influence these preferences by looking at two different groups who participated based on different ethnic greetings. Our study reveals insight into how ethnicity, percentage of life lived in Western countries, personality variations, and implementation of cultural aspects influence the likability of robotic greeting gestures. Our investigations highlight the complexity of creating culturally adaptive robots that demonstrate the cultural norms and gestures that align with the expectations of the respective cultural groups.

Greeting Preferences in a Hospitality Context: A Cross-Cultural Study with a Social Robot

We investigated the impact of a government policy on Artificial Intelligence (AI) technology on people's trust in AI applications in society. Specifically, we sought to examine how trust in AI applications is affected by the government's emphasis on either innovation or regulation, and the risks associated with AI applications. In an online experiment, participants were presented with a government policy that prioritized either the innovation of AI technology for economic growth or the regulation of AI technology for the protection of public interest. Participants then indicated their trust in various AI application cases with varying levels of perceived risk. We found that perceived risk influenced participants trust in the applications, and that participants strongly supported AI policies prioritizing regulation. There were also possible trends suggesting the interactive effect of policies and perceived risk on participants trust in AI applications and the effect of policies on the decrease of participants general trust in AI technology. This work represents a crucial preliminary step in contributing to the ongoing global debate on the AI regulation dilemma.

Balancing Innovation and Regulation: The Impact of Government Policies and Perceived Risk on Public Trust in AI Applications

Social robots interacting with people in public spaces may access and collect their personal information, which raises privacy concerns regarding the disclosure of personal information. This paper aims to investigate factors impacting individuals' intention to disclose personal information to a social robot in public spaces and evaluate the actual disclosure during the interaction with the robot. For this purpose, a model is proposed to predict people's intentions to disclose information to a social robot. We conducted our experiment at a public festival with more than 100 participants using the social robot ARI. The findings reveal the substantial impact of factors including risk beliefs, trusting beliefs, perceived enjoyment, and social influence on the intention to disclose personal information. Moreover, they reveal that although only a small percentage (6.20%) of people had the intention to disclose information to the social robot, most participants (98.00%) finally disclosed their personal information.

How Do People Intend to Disclose Personal Information to a Social Robot in Public Spaces?

Social-emotional learning (SEL) skills are essential for children to develop to provide a foundation for future relational and academic success. Using art as a medium for creation or as a topic to provoke conversation is a well-known method of SEL education. Similarly, social robots have been used to teach SEL competencies like empathy, but the combination of art and social robotics has been minimally explored. In this paper, we present a novel child-robot interaction designed to foster empathy and promote SEL competencies via a conversation about art scaffolded by a social robot. Participants (N=11, age range: 7-11) conversed with a social robot about emotional and neutral art. Analysis of video and speech data demonstrated that this interaction design successfully engaged children in the practice of SEL skills, like emotion recognition and self-awareness, and greater rates of empathetic reasoning were observed when children engaged with the robot about emotional art. This study demonstrated that art-based reflection with a social robot, particularly on emotional art, can foster empathy in children, and interactions with a social robot may help alleviate discomfort when sharing deep or vulnerable emotions.

A HeARTfelt Robot: Social Robot-Driven Deep Emotional Art Reflection with Children

Congestion in urban spaces sometimes leads to serious accidents. To prevent a congestion and serious accident, it is important to properly arrange-building layouts and guide pedestrians safely based on, mathematical modeling and simulation of pedestrian flows. In the real world, however, each pedestrian's behavior is so complicated and often includes spatio-temporally multiscale decision making while most of existing studies mainly focused on a crossing area in front of a station such as Shibuya scramble crossing. In a previous report, a trained model could simulate some pedestrian trajectory, but it also generated undesirable behavior that a pedestrian ignored a traffic light. In this study, we introduce an attention mechanism to solve this problem. The performance of the model is evaluated quantitatively by the prediction error and qualitatively by visualizing the time of a traffic light switching.

Spatio-Temporal Multi-Scale Pedestrian Flow Model by Using Attention Module

Avatar robots are telepresence robots that allow people to project themselves physically in distant places. Teleoperators use such avatars to have remote social interactions. Mobility adds to their interactive capabilities. However, it also presents challenges in social spaces; for instance, teleoperators can mistakenly move their avatars in a socially inappropriate manner. Since these robots have yet to become ubiquitous, there is little real-world data to objectively analyze the presence and frequency of such undesirable movement. Consequently, the different ways such movements manifest are unknown, and their occurrence cannot be detected. To address this problem, we conducted a laboratory experiment with hired participants in a replica artwork exhibition so that we could gather realistic human-avatar interaction data using sensors. Our data analysis helped us identify three types of inadvertently inappropriate movement behavior. We present a definition of such behavior based on these three inappropriate movement types and a labeled dataset containing more than 400 interactions, 18% of which were inappropriate. We also present detection models for each inappropriate movement type trained using our dataset.

Premium content

Downloads

Next from IEEE RO-MAN 2024

If a Robot Was Teaching, Then Everybody Would Definitely Like School Better: An Analysis of Grade 3-5 Children's Perceptions of Learning STEM Vocabulary with a Social Educational Robot

Similar lecture

SHARE: Synthesizing Heterogeneous Autism-support Records into Evidence-based Recommendations