United States

As large language models (LLMs) continue to advance in capability and influence, ensuring their security and preventing harmful outputs has become crucial. A promising approach to address these concerns involves training models to automatically generate adversarial prompts for red teaming. However, the evolving subtlety of vulnerabilities in LLMs challenges the effectiveness of current adversarial methods, which struggle to generate diverse, complex prompts and dynamically explore the weaknesses of these models. To tackle these challenges, we introduce the **S**elf-**E**volving **A**dversarial **S**afety (**SEAS**) optimization framework, which includes both a SEAS dataset and a SEAS pipeline. The SEAS dataset comprises complex adversarial prompts, while the SEAS pipeline operates through three stages: Initialization, Attack, and Adversarial Optimization. This framework generates a diverse range of adversarial prompts and dynamically explores the model&#39;s vulnerabilities to enhance its security. Our contributions include a novel adversarial framework, a comprehensive safety dataset, and empirical evidence demonstrating the effectiveness of SEAS. After three iterations, the Target model achieves a security level comparable to GPT-4.

AAAI 2025

SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models

safety and robustness

snlp

As large language models (LLMs) continue to advance in capability and influence, ensuring their security and preventing harmful outputs has become crucial. A promising approach to address these concerns involves training models to automatically generate adversarial prompts for red teaming. However, the evolving subtlety of vulnerabilities in LLMs challenges the effectiveness of current adversarial methods, which struggle to generate diverse, complex prompts and dynamically explore the weaknesses of these models. To tackle these challenges, we introduce the **S**elf-**E**volving **A**dversarial **S**afety (**SEAS**) optimization framework, which includes both a SEAS dataset and a SEAS pipeline. The SEAS dataset comprises complex adversarial prompts, while the SEAS pipeline operates through three stages: Initialization, Attack, and Adversarial Optimization. This framework generates a diverse range of adversarial prompts and dynamically explores the model's vulnerabilities to enhance its security. Our contributions include a novel adversarial framework, a comprehensive safety dataset, and empirical evidence demonstrating the effectiveness of SEAS. After three iterations, the Target model achieves a security level comparable to GPT-4.

poster

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



Formal XAI is an emerging field that focuses on providing explanations
with mathematical guarantees for the decisions made by machine
learning models. Recent work in this area has centered on the
computation of ``sufficient reasons''. Given a model $\mathcal{M}$
and an input instance $\mathbf{x}$, a sufficient reason for the decision $\mathcal{M}(\mathbf{x})$ is a
subset $S$ of the features of $\mathbf{x}$ such that for any instance $\mathbf{z}$
that has the same values as $\mathbf{x}$ for every feature in $S$, it holds that $\mathcal{M}(\mathbf{x}) = \mathcal{M}(\mathbf{z})$. Intuitively, this means
that the features in $S$ are sufficient to fully justify the classification of $\mathbf{x}$ by $\mathcal{M}$.
For sufficient reasons to be useful in practice, they should be as
small as possible, and a natural way to reduce the size of sufficient
reasons is to consider a probabilistic relaxation; the probability of $\mathcal{M}(\mathbf{x}) = \mathcal{M}(\mathbf{z})$ must
be at least some value $\delta \in (0,1]$, where $\mathbf{z}$ is a random instance compatible with $\mathbf{x}$.  Computing small $\delta$-sufficient reasons ($\delta$-SRs) is known to be a theoretically hard problem; even over decision trees — traditionally deemed simple and interpretable models — strong inapproximability results make the efficient computation of small $\delta$-SRs unlikely.
We propose the notion of $(\delta, \epsilon)$-SR, a simple relaxation of $\delta$-SRs, and show that while this relaxation is hard to compute over decision trees (as a simple consequence of the previous inapproximability results), it can be computed efficiently over linear models.

Probabilistic Explanations for Linear Models

Existing causal learning algorithms focus on micro-level causal discovery, confronting significant challenges in identifying the influence of macro systems, composed of micro-level variables, on other variables. This difficulty arises because the causal relationships in macro systems are often mediated through micro-level causal interactions, which can lead to erroneous causal discovery or omission when dispersed. To address this issue, we propose the Emergence-inspired Multi-granularity Casual learning (EMCausal) method. Inspired by emergent phenomena of aggregating micro-level nodes into macro-level entities, EMCausal introduces a progressive mapping encoder to simulate the process, thereby capturing the causal relationships driven by these macro entities. Next, it introduces a causal consistency constraint to collaboratively reconstruct micro-level nodes using macro-level representations, enabling the learning of a multi-granular causal structure. 
Experimental results on both synthetic and real datasets demonstrate that EMCausal can identify causal graphs under the influence of causal emergence, outperforming competitive baselines in terms of accuracy and robustness.

Emergence-Inspired Multi-Granularity Causal Learning

Although recent years have witnessed significant advancements in medical image segmentation, the pervasive issue of domain shift among medical images from diverse centres hinders the effective deployment of pre-trained models. Many Test-time Adaptation (TTA) methods have been proposed to address this issue by fine-tuning pre-trained models with test data during inference. These methods, however, often suffer from less-satisfactory optimization due to suboptimal optimization direction (dictated by the gradient) and fixed step-size (predicated on the learning rate). In this paper, we propose the Gradient alignment-based Test-time adaptation (GraTa) method to improve both the gradient direction and learning rate in the optimization procedure. Unlike conventional TTA methods, which primarily optimize the pseudo gradient derived from a self-supervised objective, our method incorporates an auxiliary gradient with the pseudo one to facilitate gradient alignment. Such gradient alignment enables the model to excavate the similarities between different gradients and correct the gradient direction to approximate the empirical gradient related to the current segmentation task. Additionally, we design a dynamic learning rate based on the cosine similarity between the pseudo and auxiliary gradients, thereby empowering the adaptive fine-tuning of pre-trained models on diverse test data. Extensive experiments establish the effectiveness of the proposed gradient alignment and dynamic learning rate and substantiate the superiority of our GraTa method over other state-of-the-art TTA methods on a benchmark medical image segmentation task. The code and weights of pre-trained source models will be available.

Gradient Alignment Improves Test-Time Adaptation for Medical Image Segmentation

Low-rank adaptation (LoRA) is an efficient strategy for adapting latent diffusion models (LDMs) on a private dataset to generate specific images by minimizing the adaptation loss. However, the LoRA-adapted LDMs are vulnerable to membership inference (MI) attacks that can judge whether a particular data point belongs to the private dataset, thus leading to the privacy leakage. To defend against MI attacks, we first propose a straightforward solution: Membership-Privacy-preserving LoRA (MP-LoRA). MP-LoRA is formulated as a min-max optimization problem where a proxy attack model is trained by maximizing its MI gain while the LDM is adapted by minimizing the sum of the adaptation loss and the MI gain of the proxy attack model. However, we empirically find that MP-LoRA has the issue of unstable optimization, and theoretically analyze that the potential reason is the unconstrained local smoothness, which impedes the privacy-preserving adaptation. To mitigate this issue, we further propose a Stable Membership-Privacy-preserving LoRA (SMP-LoRA) that adapts the LDM by minimizing the ratio of the adaptation loss to the MI gain. Besides, we theoretically prove that the local smoothness of SMP-LoRA can be constrained by the gradient norm, leading to improved convergence. Our experimental results corroborate that SMP-LoRA can indeed defend against MI attacks and generate high-quality images. Our Code is available at https://anonymous.4open.science/r/StablePrivateLoRA-7030/README.md.

Privacy-Preserving Low-Rank Adaptation against Membership Inference Attacks for Latent Diffusion Models

Offline black-box optimization aims to identify the optimal solution of a black-box objective function under the guidance of a surrogate model constructed solely from a pre-collected dataset. It is commonly used in industrial scenarios, which often involve constraints, i.e., constrained offline optimization (COO). Offline optimization has progressed in addressing the out-of-distribution (OOD) issue caused by its inherent inability to interact with the objective function. However, there is not enough research in addressing more difficult scenarios, which must simultaneously address OOD issues and constrained issues to find stable, high-quality (i.e., high-scoring and feasible) solutions. To bridge this gap, this paper proposes a method called constrained offline optimization via risk evaluation and management (COOREM), which is capable of consistently surpassing the offline dataset under the condition of satisfying constraints. Specifically, COOREM employs a dual-energy model to separately evaluate OOD risk and constrained risk. This separation strategy aims to distinguish and address two difficulty cases: the infeasible but not OOD solutions and the feasible but OOD solutions. Moreover, COOREM effectively manages OOD risk and constrained risk, ensuring the identification of high-quality solutions. Extensive experiments on real-world tasks, e.g., space missions, process synthesis and design problems, showcase COOREM's effectiveness in managing both OOD risk and constrained risk. Furthermore, our findings indicate that COOREM could outperform online methods that need to access the objective function in certain space missions.

Constrained Offline Black-Box Optimization via Risk Evaluation and Management

We investigate the sample-memory-pass trade-offs for pure exploration in multi-pass streaming multi-armed bandits (MABs) with the \emph{a priori} knowledge of the optimality gap $\Delta_{[2]}$. 
Here, and throughout, the optimality gap $\Delta_{[i]}$ is defined as the mean reward gap between the best and the $i$-th best arms. A recent line of results by Jin, Huang, Tang, and Xiao [ICML'21] and Assadi and Wang [COLT'24] have shown that if there is no known $\Delta_{[2]}$, a pass complexity of $\tilde{\Theta}(\log(1/\Delta_{[2]}))$ is necessary and sufficient to obtain the \emph{worst-case optimal} $O(n/\Delta^2_{[2]})$ sample complexity with a single-arm memory. However, our understanding of multi-pass algorithms with known $\Delta_{[2]}$ is still limited. Here, the key open problem is how many passes are required to achieve the complexity, i.e., $O( \sum_{i=2}^{n}1/\Delta^2_{[i]} \cdot \log{n})$ arm pulls, with a sublinear memory size.
	
In this work, we show that the ``right answer'' for the question is $\tilde{\Theta}(\log{n})$ passes.  We first present a lower bound, showing that any algorithm that finds the best arm with slightly sublinear memory -- a memory of $o({n}/{\text{polylog}({n})})$ arms -- and $O(\sum_{i=2}^{n}{1}/{\Delta^2_{[i]}}\cdot \log{n})$ arm pulls has to make $\Omega(\frac{\log{n}}{\log\log{n}})$ passes over the stream. We then show a nearly-matching algorithm that assuming the knowledge of $\Delta_{[2]}$, finds the best arm with $O( \sum_{i=2}^{n}1/\Delta^2_{[i]} \cdot \log{n})$ arm pulls and a \emph{single arm} memory.

Nearly Tight Bounds for Exploration in Streaming Multi-Armed Bandits with Known Optimality Gap

Solving tabular math word problems (TMWPs) has become one critical role to evaluate the mathematical reasoning ability of large language models (LLMs), where large-scale TMWP samples are commonly required for LLM fine-tuning. Since the collection of high-quality TMWP dataset is costly and time-consuming, recent research has concentrated on automatic TMWP generation. However, current generated samples usually suffer from issues of either correctness or diversity. In this paper, we propose a TEmplate-driven LLM-paraphrased (TeLL) framework for generating high-quality TMWP samples with diverse descriptions and correct question-answers. To this end, we first extract templates from existing real samples to generate question-answer pairs, ensuring the correctness. Then, we adopt an LLM to extend templates and paraphrase problems, obtaining diverse TMWP samples. Furthermore, we find the reasoning annotation is important for solving TMWPs. Therefore, we propose to enrich each solution with illustrative reasoning steps. Through the proposed framework, we construct a high-quality dataset TabMWP-TeLL by adhering to the question types in the TabMWP dataset, and we conduct extensive experiments on a variety of LLMs to demonstrate the effectiveness of TabMWP-TeLL in improving TMWP solving performance.

Template-Driven LLM-Paraphrased Framework for Tabular Math Word Problem Generation

Visual language models like Contrastive Language-Image Pretraining (CLIP) have shown impressive performance in analyzing natural images with language information. However, these models often encounter challenges when applied to specialized domains such as remote sensing due to the limited availability of image-text pairs for training. To tackle this issue, we introduce DiffCLIP, a novel framework that extends CLIP to effectively convey comprehensive language-driven semantic information for accurate classification of high-dimensional multimodal remote sensing images. DiffCLIP is a few-shot learning method that leverages unlabeled images for pretraining. It employs unsupervised mask diffusion learning to capture the distribution of diverse modalities without requiring labels. The modality-shared image encoder maps multimodal data into a unified subspace, extracting shared features with consistent parameters across modalities. A well-trained image encoder further enhances learning by aligning visual representations with class-label text information from CLIP. By integrating these approaches, DiffCLIP significantly boosts CLIP performance using a minimal number of image-text pairs. We evaluate DiffCLIP on widely used high-dimensional multimodal datasets, demonstrating its effectiveness in addressing few-shot annotated classification tasks. DiffCLIP achieves an overall accuracy improvement of 10.65\% across three remote sensing datasets compared with CLIP, while utilizing only 2-shot image-text pairs.

DiffCLIP: Few-shot Language-driven Multimodal Classifier

Feature matching between image pairs is a fundamental problem in computer vision that drives many applications, such as SLAM. 
Recently, semi-dense matching approaches have achieved substantial performance enhancements and established a widely-accepted coarse-to-fine paradigm. 
However, the majority of existing methods focus on improving coarse feature representation rather than the fine-matching module.
Prior fine-matching techniques, which rely on point-to-patch matching probability expectation or direct regression, often lack precision and do not guarantee the continuity of feature points across sequential images.
To address this limitation, this paper concentrates on enhancing the fine-matching module in the semi-dense matching framework.
We employ a lightweight and efficient homography estimation network to generate the perspective mapping between patches obtained from coarse matching. 
This patch-to-patch approach achieves the overall alignment of two patches, resulting in a higher sub-pixel accuracy by incorporating additional constraints.
By leveraging the homography estimation between patches, we can achieve a dense matching result with low computational cost. 
Extensive experiments demonstrate that our method achieves higher accuracy compared to previous semi-dense matchers.
Meanwhile, our dense matching results exhibit similar end-point-error accuracy compared to previous dense matchers while maintaining semi-dense efficiency.

HomoMatcher: Achieving Dense Feature Matching with Semi-Dense Efficiency by Homography Estimation

Recommender systems used in online platforms can drive users to
consume content continuously in an attempt to maximize satisfaction. Such engagement is invariably broken due to more pressing
work, alternate pursuits, distractions or fatigue. Recommender systems need to ensure the continuity of experience when the user
joins back. Session-based recommender systems typically create
different sessions based on a fixed time interval (𝜃 ), often resulting
in creation of a separate session when the user gets off the platform
temporarily. When the user joins back, session-based recommender
systems are likely to recommend content different than what they
would have in case the earlier session had continued. This may
cause dissatisfaction given that there is a difference in the predicted
world model of the user, i.e. the expectation from the last session,
and the observed one, i.e. the recommendations.
To handle this problem, we propose the creation of content-
driven sessions instead of time-driven sessions. In this, a session
continues until a single item category dominates in the user-item
interactions. A new session is created when a different item cate-
gory begins to dominate. The proposed content-driven method also
solves the ever-standing problem of deciding the optimal value of
time threshold for defining the time-based session. We report
that the proposed method outperforms existing SOTA method-
ologies set by time-based sessions by a large margin in terms of
recommendation performance on multiple datasets.

SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models

Downloads

Next from AAAI 2025

Probabilistic Explanations for Linear Models

SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2025

Probabilistic Explanations for Linear Models

Downloads