United States

Learning complicated value functions in high dimensional state space by function approximation is a challenging task, partially due to that the max-operator used in temporal difference updates can theoretically cause instability for most linear or non-linear approximation schemes. Mellowmax is a recently proposed differentiable and non-expansion softmax operator that allows a convergent behavior in learning and planning. Unfortunately, the performance bound for the fixed point it converges to remains unclear, and in practice, its parameter is sensitive to various domains and has to be tuned case by case. Finally, the Mellowmax operator may suffer from oversmoothing as it ignores the probability being taken for each action when aggregating them. In this paper we address all the above issues with an enhanced Mellowmax operator, named SM2 (Soft Mellowmax). Particularly, the proposed operator is reliable, easy to implement, and has provable performance guarantee,  while preserving all the advantages of Mellowmax. Furthermore, we show that our SM2 operator can be applied to the challenging multi-agent reinforcement learning scenarios, leading to stable value function approximation and state of the art performance.

AAAI 2021

Stabilizing Q Learning via Soft Mellowmax Operator

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

IEEE RO-MAN 2024

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

technical paper

The purpose of the AAAI conference is to promote research in artificial intelligence (AI) and scientific exchange among AI researchers, practitioners, scientists, and engineers in affiliated disciplines. 

Traffic state estimation (TSE) reconstructs the traffic variables (e.g., density or average velocity) on road segments using partially observed data, which is important for traffic managements. Traditional TSE approaches mainly bifurcate into two categories: model-driven and data-driven, and each of them has shortcomings. To mitigate these limitations, hybrid TSE methods, which combine both model-driven and data-driven, are becoming a promising solution. This paper introduces a hybrid framework, physics-informed deep learning (PIDL), to combine second-order traffic flow models and neural networks to solve the TSE problem. PIDL can encode traffic flow models into deep neural networks to regularize the learning process to achieve improved data efficiency and estimation accuracy. We focus on highway TSE with observed data from loop detectors and probe vehicles, using both density and average velocity as the traffic variables. With numerical examples, we show the use of PIDL to solve a popular second-order traffic flow model, i.e., a Greenshields-based Aw-Rascle-Zhang (ARZ) model, and discover the model parameters. We then evaluate the PIDL-based TSE method using the Next Generation SIMulation (NGSIM) dataset. Experimental results demonstrate the proposed PIDL-based approach to outperform advanced baseline methods in terms of data efficiency and estimation accuracy.

Physics-Informed Deep Learning for Traffic State Estimation: A Hybrid Paradigm Informed by Second-Order Traffic Models

Multi-exit architectures, in which a sequence of intermediate classifiers are introduced at different depths of the feature layers, perform adaptive computation by early exiting ``easy" samples to speed up the inference. In this paper, a novel Harmonized Dense Knowledge Distillation (HDKD) training method for multi-exit architecture is designed to encourage each exit to flexibly learn from all its later exits. In particular, a general dense knowledge distillation training objective is proposed to incorporate all possible beneficial supervision information for multi-exit learning, where a harmonized weighting scheme is designed for the multi-objective optimization problem consisting of multi-exit classification loss and dense distillation loss. A bilevel optimization algorithm is introduced for alternatively updating the weights of multiple objectives and the multi-exit network parameters. Specifically, the loss weighting parameters are optimized with respect to its performance on validation set by gradient descent. Experiments on CIFAR100 and ImageNet show that the HDKD strategy harmoniously improves the performance of the state-of-the-art multi-exit neural networks. Moreover, this method does not require within architecture modifications and can be effectively combined with other previously-proposed training techniques and further boosts the performance.

Harmonized Dense Knowledge Distillation Training for Multi-Exit Architectures

Image captioning is a challenging computer vision task, which aims to generate a natural language description of an image. Most recent researches follow the encoder-decoder framework which depends heavily on the previous generated words for the current prediction. Such methods can not effectively take advantage of the future predicted information to learn complete semantics. In this paper, we propose  Context-Aware Auxiliary Guidance (CAAG) mechanism that can guide the captioning model to perceive global contexts. Upon the captioning model, CAAG performs semantic attention that selectively concentrates on useful information of the global predictions to reproduce the current generation. To validate the adaptability of the method, we apply CAAG to three popular captioners and our proposal achieves competitive performance on the challenging Microsoft COCO image captioning benchmark, e.g. 132.2 CIDEr-D score on Karpathy split and 130.7 CIDEr-D (c40) score on official online evaluation server.

Image Captioning with Context-Aware Auxiliary Guidance

Point-of-interest (POI) recommendation has become an important research topic in these days. The user check-in history used as the input to POI recommendation is very imbalanced and noisy because of sparse and missing check-ins. Although sample reweighting is commonly adopted for addressing this challenge with the input data, its fixed weighting scheme is often inappropriate to deal with different characteristics of users or POIs. Thus, in this paper, we propose PREMERE, an adaptive weighting scheme based on meta-learning. Because meta-data is typically required by meta-learning but is inherently hard to obtain in POI recommendation, we self-generate the meta-data via self-ensembling. Furthermore, the meta-model architecture is extended to deal with the scarcity of check-ins. Thorough experiments show that replacing a weighting scheme with PREMERE boosts the performance of the state-of-the-art recommender algorithms by 2.36â€“26.9% on three benchmark datasets.

PREMERE: Meta-Reweighting via Self-Ensembling for Point-of-Interest Recommendation

Traffic accident forecasting is of great importance to urban public safety, emergency treatment, and construction planning. However, it is very challenging since traffic accidents are affected by multiple factors, and have multi-scale dependencies on both spatial and temporal dimensional features. Meanwhile, traffic accidents are rare events, which leads to the zero-inflated issue. Existing traffic accident forecasting methods cannot deal with all above problems simultaneously. In this paper, we propose a novel model, named GSNet, to learn the spatial-temporal correlations from geographical and semantic aspects for traffic accident risk forecasting. In the model, a Spatial-Temporal Geographical Module is designed to capture the geographical spatial-temporal correlations among regions, while a Spatial-Temporal Semantic Module is proposed to model the semantic spatial-temporal correlations among regions. In addition, a weighted loss function is designed to solve the zero-inflated issue. Extensive experiments on two real-world datasets demonstrate the superiority of GSNet against the state-of-the-art baseline methods.

GSNet: Learning Spatial-Temporal Correlations from Geographical and Semantic Aspects for Traffic Accident Risk Forecasting

With the rapid emergence of video data, image-to-video retrieval has attracted much attention. There are two types of image-to-video retrieval: instance-based and activity-based. The former task aims to retrieve videos containing the same main objects as the query image, while the latter focuses on finding the similar activity. Since dynamic information plays a significant role in the video, we pay attention to the latter task to explore the motion relation between images and videos. In this paper, we propose a Motion-assisted Activity Proposal-based Image-to-Video Retrieval (MAP-IVR) approach to disentangle the video features into motion features and appearance features and obtain appearance features from the images. Then, we perform image-to-video translation to improve the disentanglement quality. The retrieval is performed in both appearance and video feature spaces. Extensive experiments demonstrate that our MAP-IVR approach remarkably outperforms the state-of-the-art approaches on two benchmark activity-based video datasets.

Activity Image-to-Video Retrieval by Disentangling Appearance and Motion

Adapting semantic segmentation models to new domains is an important but challenging problem. Recently enlightening progress has been made, but the performance of existing methods is unsatisfactory on real datasets where the new target domain comprises of heterogeneous sub-domains (e.g. diverse weather characteristics). We point out that carefully reasoning about the multiple modalities in the target domain can improve the robustness of adaptation models. To this end, we propose a condition-guided adaptation framework that is empowered by a special attentive progressive adversarial training (APAT) mechanism and a novel self-training policy. The APAT strategy progressively performs condition-specific alignment and attentive global feature matching. The new self-training scheme exploits the adversarial ambivalences of easy and hard adaptation regions and the correlations among target sub-domains effectively. We evaluate our method (DCAA) on various adaptation scenarios where the target images vary in weather conditions. The comparisons against baselines and the state-of-the-art approaches demonstrate the superiority of DCAA over the competitors.

Exploiting Diverse Characteristics and Adversarial Ambivalence for Domain Adaptive Segmentation

Federated Learning (FL) has attracted increasing attention in recent years. A leading training algorithm in FL is local SGD, which updates the model parameter on each worker and averages model parameters across different workers only once in a while. Although it has fewer communication rounds than the classical parallel SGD, local SGD still has large communication overhead in each communication round for large machine learning models, such as deep neural networks.  To address this issue, we propose a new communication-efficient distributed SGD method,  which can significantly reduce the communication cost by the error-compensated double compression mechanism. Under the non-convex setting, our theoretical results show that our approach has better communication complexity than existing  methods and enjoys the same linear speedup regarding the number of workers as the full-precision local SGD.  Moreover, we  propose a communication-efficient distributed SGD with momentum, which also has better communication complexity than existing methods and enjoys a linear speedup with respect to the number of workers.  At last, extensive experiments are conducted to verify the performance of our proposed two methods. Moreover, we  propose a communication-efficient distributed SGD with momentum to accelerate the convergence, which also has better communication complexity than existing methods and enjoys a linear speedup with respect to the number of workers.  At last, extensive experiments are conducted to verify the performance of our proposed methods.

On the Convergence of Communication-Efficient Local SGD for Federated Learning

As a fundamental problem in algorithmic trading, order execution aims at fulfilling a specific trading order, either liquidation or acquirement, for a given instrument. Towards effective execution strategy, recent years have witnessed the shift from the analytical view with model-based market assumptions to model-free perspective, i.e., reinforcement learning, due to its nature of sequential decision optimization. However, the noisy and yet imperfect market information that can be leveraged by the policy has made it quite challenging to build up sample efficient reinforcement learning methods to achieve effective order execution. In this paper, we propose a novel universal trading policy optimization framework to bridge the gap between the noisy yet imperfect market states and the optimal action sequences for order execution. Particularly, this framework leverages a policy distillation method that can better guide the learning of the common policy towards practically optimal execution by an oracle teacher with perfect information to approximate the optimal trading strategy. The extensive experiments have shown significant improvements of our method over various strong baselines, with reasonable trading actions.

Universal Trading for Order Execution with Oracle Policy Distillation

Causal inference from observational data is receiving wide applications in many fields. However, unidentifiable situations, where causal effects cannot be uniquely computed from observational data, pose critical barriers to applying causal inference to complicated real applications. In this paper, we develop a bounding method for estimating the average causal effect (ACE) under unidentifiable situations due to hidden confounding based on Pearl's structural causal model.
We propose to parameterize the unknown exogenous random variables and structural equations of a causal model using neural networks and implicit generative models. Then, using an adversarial learning framework, we search the parameter space to explicitly traverse causal models that agree with the given observational distribution, and find those that minimize or maximize the ACE to obtain its lower and upper bounds. The proposed method does not make assumption about the type of structural equations and variables. Experiments using both synthetic and real-world datasets are conducted.

Downloads

Next from AAAI 2021

Physics-Informed Deep Learning for Traffic State Estimation: A Hybrid Paradigm Informed by Second-Order Traffic Models

Similar lecture

Robust Gesture-Based Appliance Control Via Operator Identification and Tracking

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2021

Physics-Informed Deep Learning for Traffic State Estimation: A Hybrid Paradigm Informed by Second-Order Traffic Models

Similar lecture

Robust Gesture-Based Appliance Control Via Operator Identification and Tracking

Downloads