United States

The ability to autonomously explore and resolve tasks with minimal human guidance is crucial for the self-development of embodied robots. Although reinforcement learning methods can largely ease human effort, designing reward functions for real-world tasks, especially high-dimensional robotic control, is challenging due to complex relationships among joints and tasks. Recent advancements in the aid of large language models (LLMs) enable automatic reward function design. However, current approaches evaluate reward functions by re-training policies from scratch placing an undue burden on the reward function, expecting it to be effective at every stage of the policy improvement process, from initiation to convergence. We argue for a more practical strategy in robotic autonomy, focusing on refining existing policies with policy-dependent reward functions rather than seeking a universal reward function. To this end, we propose a novel reward-policy co-evolution framework where the reward function and the learned policy benefit from each other&#39;s progressive on-the-fly improvements, resulting in more efficient and higher-performing skill acquisition. 
Specifically, the reward evolution process translates the robot&#39;s previous best reward function, descriptions of tasks and environment into text inputs. These inputs are used to query LLMs to generate a dynamic amount of reward function candidates, ensuring continuous improvement through each evolutionary round. For policy evolution, our method generates new policy populations by hybridizing historically optimal and random policies. Through an improved Bayesian optimization, our approach efficiently and robustly identifies the most capable and plastic reward-policy combination, which then proceeds to the next round of co-evolution.
Despite using less data, our approach demonstrates an average normalized improvement of $95.3$\% across various high-dimensional robotic skill learning tasks.

AAAI 2025

Efficient Language-instructed Skill Acquisition via Reward-Policy Co-Evolution

reinforcement learning

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

AAAI 2026

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The ability to autonomously explore and resolve tasks with minimal human guidance is crucial for the self-development of embodied robots. Although reinforcement learning methods can largely ease human effort, designing reward functions for real-world tasks, especially high-dimensional robotic control, is challenging due to complex relationships among joints and tasks. Recent advancements in the aid of large language models (LLMs) enable automatic reward function design. However, current approaches evaluate reward functions by re-training policies from scratch placing an undue burden on the reward function, expecting it to be effective at every stage of the policy improvement process, from initiation to convergence. We argue for a more practical strategy in robotic autonomy, focusing on refining existing policies with policy-dependent reward functions rather than seeking a universal reward function. To this end, we propose a novel reward-policy co-evolution framework where the reward function and the learned policy benefit from each other's progressive on-the-fly improvements, resulting in more efficient and higher-performing skill acquisition. 
Specifically, the reward evolution process translates the robot's previous best reward function, descriptions of tasks and environment into text inputs. These inputs are used to query LLMs to generate a dynamic amount of reward function candidates, ensuring continuous improvement through each evolutionary round. For policy evolution, our method generates new policy populations by hybridizing historically optimal and random policies. Through an improved Bayesian optimization, our approach efficiently and robustly identifies the most capable and plastic reward-policy combination, which then proceeds to the next round of co-evolution.
Despite using less data, our approach demonstrates an average normalized improvement of $95.3$\% across various high-dimensional robotic skill learning tasks.

technical paper

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



Speaker recognition technology is applied to various tasks, from personal virtual assistants to secure access systems. However, the robustness of these systems against adversarial attacks, particularly to additive perturbations, remains a significant challenge. In this paper, we pioneer applying robustness certification techniques to speaker recognition, originally developed for the image domain. Our work covers this gap by transferring and improving randomized smoothing certification techniques against norm-bounded additive perturbations for classification and few-shot learning tasks to speaker recognition. We demonstrate the effectiveness of these methods on VoxCeleb 1 and 2 datasets for several models. We expect this work to improve the robustness of voice biometrics and accelerate the research of certification methods in the audio domain.

Certification of Speaker Recognition Models to Additive Perturbations

Large Language Models (LLMs) often face challenges when prompted to generate content under specific constraints or conditions. However, in such cases it is often easy to check whether these constraints are satisfied or violated. Recent works have shown that LLMs can benefit from such ``corrective feedback''. Here we claim that this skill of LLMs can be significantly enhanced via training. We introduce an RL framework for teaching models to use such rewards, by simulating interaction sessions, and rewarding the model according to its ability to satisfy the constraints.
We refer to our method as CORGI (Controlled Generation with RL for Guided Interaction), and evaluate it on a variety of controlled generation tasks using unlabeled training data. We find that CORGI consistently outperforms the baseline reinforcement learning method that does not incorporate conversational feedback. Furthermore, CORGI's interactive framework enables meta-learning, allowing the LLM to generalize better to guided interaction in new tasks. Through this dynamic interaction, the model learns to optimize its responses in real time, ensuring improved compliance with predefined requirements. Our experiments demonstrate that combining textual feedback with reinforcement learning results in more effective optimization and enhanced performance across a diverse range of generation tasks. These findings indicate that conversational optimization, when combined with reinforcement learning, significantly improves the effectiveness of LLMs in controlled generation contexts.

Teaching Models to Improve on Tape

Extracting lane topology from perspective views (PV) is crucial for planning and control in autonomous driving. This approach extracts potential drivable trajectories for self-driving vehicles without relying on high-definition (HD) maps. However, the unordered nature and weak long-range perception of the DETR-like framework can result in misaligned segment endpoints and limited topological prediction capabilities. Inspired by the learning of contextual relationships in language models, the connectivity relations in roads can be characterized as explicit topology sequences. In this paper, we introduce $\textbf{Topo2Seq}$, a novel approach for enhancing topology reasoning via topology sequences learning. The core concept of Topo2Seq is a randomized order prompt-to-sequence learning between lane segment decoder and topology sequence decoder. The dual-decoder branches simultaneously learn the lane topology sequences extracted from the Directed Acyclic Graph (DAG) and the lane graph containing geometric information. Randomized order prompt-to-sequence learning extracts unordered key points from the lane graph predicted by the lane segment decoder, which are then fed into the prompt design of the topology sequence decoder to reconstruct an ordered and complete lane graph. In this way, the lane segment decoder learns powerful long-range perception and accurate topological reasoning from the topology sequence decoder. Notably, topology sequence decoder is only introduced during training and does not affect the inference efficiency. Experimental evaluations on the OpenLane-V2 dataset demonstrate the state-of-the-art performance of Topo2Seq in topology reasoning.

Topo2Seq: Enhanced Topology Reasoning via Topology Sequence Learning

Today, deep neural networks are widely used since they can handle a variety of complex tasks. Their generality makes them very powerful tools in modern technology. However, deep neural networks are often overparameterized. The usage of these large models consumes a lot of computation resources. 

In this paper, we introduce a method called **T**ill the **L**ayers **C**ollapse (TLC), which compresses deep neural networks through the lenses of batch normalization layers. By reducing the depth of these networks, our method decreases deep neural networks' computational requirements and overall latency. We validate our method on popular models such as Swin-T, MobileNet-V2, and RoBERTa, across both image classification and natural language processing (NLP) tasks. The source code will be made publicly available upon the acceptance of this paper.

Till the Layers Collapse: Compressing a Deep Neural Network Through the Lenses of Batch Normalization Layers.

We study Consistent Query Answering (CQA) over knowledge bases with existential rules. Specifically, we propose a novel framework for CQA that combines previous approaches, allowing for the simultaneous presence of both open and closed predicates, i.e. predicates interpreted under open- and closed-world assumption, respectively. We establish the data complexity of answering unions of conjunctive queries in such a new framework under the so-called AR semantics and for different classes of existential rules. We also provide new complexity results for the standard (i.e. non-inconsistency tolerant) query answering in the presence of both open and closed predicates. Our results show that, for certain classes of rules, the complexity of CQA matches that of non-inconsistency-tolerant query answering.

Consistent Query Answering over Existential Rules with Open and Closed Predicates

In recent years, with the advancements in brain science, spiking neural networks (SNNs) have garnered significant attention. SNNs can generate spikes that mimic the function of neurons transmission in humans brain, thereby significantly reducing computational costs by the event-driven nature during training. While deep SNNs have shown impressive performance on classification tasks, they still face challenges in more complex tasks such as object detection. In this paper, we propose SpikingYOLOX, extending the structure of the original YOLOX by introducing signed spiking neurons and fast Fourier convolution (FFC). The designed ternary signed spiking neurons could generate three kinds of spikes to obtain more robust features in the deep layer of the backbone. Meanwhile, we integrate FFC with SNN modules to enhance object detection performance, because its global receptive field is beneficial to the object detection task. Extensive experiments demonstrate that the proposed SpikingYOLOX achieves state-of-the-art performance among other SNN-based object detection methods.

SpikingYOLOX: Improved YOLOX Object Detection with Fast Fourier Convolution and Spiking Neural Networks

Recent works have explored the use of counting queries coupled with Description Logic ontologies.
The answer of such a query in a particular model of a knowledge base is either an integer or $\infty$, and its spectrum is the set of its answers over all models.
While it is unclear how to compute and manipulate such a set in general, we identify a class of counting queries whose spectra can be effectively represented.
Further focusing on atomic counting queries, we pinpoint the possible shapes of a spectrum when the ontology is formulated in $ALCIF$: they are essentially the subsets of $\mathbb{N} \cup \{ \infty\}$ closed under addition.
For most sublogics of $ALCIF$, we show that possible spectra enjoy simpler shapes, being $[ m, \infty]$ or variations thereof.
To obtain our results, we refine constructions used for finite model reasoning in those DLs and notably rely on a cycle-reversion technique for the Horn fragment of $ALCIF$.
We also study the data complexity of computing the proposed effective representation and establish the $FP^{NP[log]}$-completeness of this task under several settings.

Spectra of Cardinality Queries over Description Logic Knowledge Bases

LiDAR-based 3D object detection is crucial for autonomous driving. However, due to the quality deterioration of LiDAR point clouds, it suffers from performance degradation in adverse weather conditions. Fusing LiDAR with the weather-robust 4D radar sensor is expected to solve this problem; however, it faces challenges of significant differences in terms of data quality and the degree of degradation in adverse weather. To address these issues, we introduce L4DR, a weather-robust 3D object detection method that effectively achieves LiDAR and 4D Radar fusion. Our L4DR proposes Multi-Modal Encoding (MME) and Foreground-Aware Denoising (FAD) modules to reconcile sensor gaps, which is the first exploration of the complementarity of early fusion between LiDAR and 4D radar. Additionally, we design an Inter-Modal and IntraModal ({IM} 2 ) parallel feature extraction backbone coupled with a Multi-Scale Gated Fusion (MSGF) module to counteract the varying degrees of sensor degradation under adverse weather conditions. Experimental evaluation on a VoD dataset with simulated fog proves that L4DR is more adaptable to changing weather conditions. It delivers a significant performance increase under different fog levels, improving the 3D mAP by up to 20.0% over the traditional LiDAR-only approach. Moreover, the results on the K-Radar dataset validate the consistent performance improvement of L4DR in real-world adverse weather conditions.

L4DR: LiDAR-4DRadar Fusion for Weather-Robust 3D Object Detection

Diffusion-based models have been recently shown to be high-quality data generators. However, its performance severely degrades when training on non-stationary changing data distributions in an online manner, which is because of catastrophic forgetting. In this paper, we propose to enable the diffusion model with a novel Dynamic Expansion Memory Unit (DEMU) approach that adaptively creates new memory buffers to preserve the information deemed critical for the training of the model. Having a selective memory buffering unit is essential for training diffusion networks, which are expensive to train, especially when deployed in resource constrained environments. A Maximum Mean Discrepancy (MMD) based expansion mechanism, that evaluates probabilistic distances between each previously defined memory buffers and the newly given data, as an expansion signal, is employed for ensuring the learning of a diversity of information. A novel memory compaction approach is proposed to automatically remove statistically overlapping memory units through a graph relationship evaluation, is employed for preventing the limitless expansion of DEMU. In addition, we propose a new model expansion mechanism to automatically add new diffusion models as experts in a mixture system, which enhances the multi-domain image generation performance. Comprehensive results show that the proposed approach performs better than the state-of-the-art.

Lifelong Scalable Generative System via Online Maximum Mean Discrepancy

DNN-based watermarking methods have rapidly advanced, with the ``Encoder-Noise Layer-Decoder'' (END) framework being the most widely used. To ensure end-to-end training, the noise layer in the framework must be differentiable. However, real-world distortions are often non-differentiable, leading to challenges in end-to-end training. 
Existing solutions only treat the distortion perturbation as additive noise, which does not fully integrate the effect of distortion in training.
To better incorporate non-differentiable distortions into training, we propose a novel dual-decoder architecture ($\textbf{END$^{2}$}$). Unlike conventional END architecture, our method employs two structurally identical decoders: the Teacher Decoder, processing pure watermarked images, and the Student Decoder, handling distortion-perturbed images. 
The gradient is backpropagated only through the Teacher Decoder branch to optimize the encoder thus bypassing the problem of non-differentiability. To ensure resistance to arbitrary distortions, we enforce alignment of the two decoders' feature representations by maximizing the cosine similarity between their intermediate vectors on a hypersphere. 
Extensive experiments demonstrate that our scheme outperforms state-of-the-art algorithms under various non-differentiable distortions. Moreover, even without the differentiability constraint, our method surpasses baselines with a differentiable noise layer. Our approach is effective and easily implementable across all END architectures, enhancing practicality and generalizability.

Downloads

Next from AAAI 2025

Certification of Speaker Recognition Models to Additive Perturbations

Similar lecture

Reward Model Evaluation via Automatically-Ranked Policy Alignment

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2025

Certification of Speaker Recognition Models to Additive Perturbations

Similar lecture

Reward Model Evaluation via Automatically-Ranked Policy Alignment

Downloads