United States

Are Federated Learning (FL) systems free from backdoor poisoning with the arsenal of various defense strategies deployed? This is an intriguing problem with significant practical implications regarding the utility of FL services. Despite the recent flourish of poisoning-resilient FL methods, our study shows that carefully tuning the collusion between malicious participants can minimize the trigger-induced bias of the poisoned local model from the poison-free one, which plays the key role in delivering stealthy backdoor attacks and circumventing a wide spectrum of state-of-the-art defense methods in FL. In our work, we instantiate the attack strategy by proposing a distributed backdoor attack method, namely Cerberus Poisoning (CerP). It jointly tunes the backdoor trigger and controls the poisoned model changes on each malicious participant to achieve a stealthy yet successful backdoor attack against a wide spectrum of defensive mechanisms of federated learning techniques. Our extensive study on 3 large-scale benchmark datasets and 13 mainstream defensive mechanisms confirms that Cerberus Poisoning raises a significantly severe threat to the integrity and security of federated learning practices, regardless of the flourish of robust Federated Learning methods.

AAAI 2023

Poisoning with Cerberus: Stealthy and Colluded Backdoor Attack against Federated Learning

backdoor attacks

defense

federated learning

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

poster

Previous remote sensing recognition approaches predomi-
nantly perform well on the training-testing dataset. Howev-
er, due to large style discrepancies not only among mul-
tidomain datasets but also within a single domain, they suf-
fer from obvious performance degradation when applied to
unseen domains. In this paper, we propose a style-content
metric learning framework to address the generalizable re-
mote sensing object recognition issue. Specifically, we first-
ly design an inter-class dispersion metric to encourage the
model to make decision based on content rather than the
style, which is achieved by dispersing predictions gener-
ated from the contents of both positive sample and nega-
tive sample and the style of input image. Secondly, we pro-
pose an intra-class compactness metric to force the model
to be less style-biased by compacting classifier’s prediction-
s from the content of input image and the styles of posi-
tive sample and negative sample. Lastly, we design an intra-
class interaction metric to improve model’s recognition ac-
curacy by pulling in classifier’s predictions obtained from
the input image and positive sample. Extensive experiments
on four datasets show that our style-content metric learn-
ing achieves superior generalization performance against the
state-of-the-art competitors. Code and model are available at:
https://github.com/wdzhao123/TSCM.

Style-Content Metric Learning for Multidomain Remote Sensing Object Recognition

Multi-view clustering has gained broad attention owing to its capacity to exploit complementary information across multiple data views. Although existing methods demonstrate delightful clustering performance, most of them are of high time complexity and cannot handle large-scale data. Matrix factorization-based models are a representative of solving this problem. However, they assume that the views share a dimension-fixed consensus coefficient matrix and view-specific base matrices, limiting their representability. Moreover, a series of large-scale algorithms that bear one or more hyperparameters are impractical in real-world applications. To address the two issues, we propose an auto-weighted multi-view clustering (AWMVC) algorithm. Specifically, AWMVC first learns coefficient matrices from corresponding base matrices of different dimensions, then fuses them to obtain an optimal consensus matrix. By mapping original features into distinctive low-dimensional spaces, we can attain more comprehensive knowledge, thus obtaining better clustering results. Moreover, we design a six-step alternative optimization algorithm proven to be convergent theoretically. Also, AWMVC shows excellent performance on various benchmark datasets compared with existing ones. The code of AWMVC is publicly available at https://github.com/wanxinhang/AAAI-2023-AWMVC.

Auto-weighted Multi-view Clustering for Large-scale Data

Although multi-view clustering (MVC) has achieved remarkable performance by integrating the complementary information of views, it is inefficient when facing scalable data. Proverbially, anchor strategy can mitigate such a challenge a certain extent. However, the unsupervised dynamic strategy usually cannot obtain the optimal anchors for MVC. The main reasons are that it does not consider the fairness of different views and lacks the priori supervised guidance. To completely solve these problems, we first propose the priori anchor graph regularization (PAGG) for scalable multi-view bipartite graph clustering, dubbed as SMGC method. Specifically, SMGC learns a few representative consensus anchors to simulate the numerous view data well, and constructs a bipartite graph to bridge the affinities between the anchors and original data points. In order to largely improve the quality of anchors, PAGG predefines prior anchor labels to constrain the anchors with discriminative cluster structure and fair view allocation, such that a better bipartite graph can be obtained for fast clustering. Experimentally, abundant of experiments are accomplished on six scalable benchmark datasets, and the experimental results fully demonstrate the effectiveness and efficiency of our SMGC.

Priori Anchor Labels Supervised Scalable Multi-view Bipartite Graph Clustering

We consider the fair division problem of indivisible items. It is well-known that an envy-free allocation may not exist, and a relaxed version of envy-freeness, envy-freeness up to one item (EF1), has been widely considered. In an EF1 allocation, an agent may envy others' allocated shares, but only up to one item. In many applications, we may wish to specify a subset of prioritized agents where strict envy-freeness needs to be guaranteed from these agents to the remaining agents, while ensuring the whole allocation is still EF1. Prioritized agents may be those agents who are envious in a previous EF1 allocation, those agents who belong to underrepresented groups, etc. Motivated by this, we propose a new fairness notion named envy-freeness with prioritized agents EFprior, and study the existence and the algorithmic aspects for the problem of computing an EFprior allocation. With additive valuations, the simple round-robin algorithm is able to compute an EFprior allocation. In this paper, we mainly focus on general valuations. In particular, we present a polynomial-time algorithm that outputs an EFprior allocation with most of the items allocated. When all the items need to be allocated, we also present polynomial-time algorithms for some well-motivated special cases.

Fair Division with Prioritized Agents

Existing episodic reinforcement algorithms assume that the  length of an episode is fixed across time and known a priori. In this paper, we consider a general framework of  episodic reinforcement learning when the length of each episode is drawn from a distribution. We first establish that this problem is equivalent to online reinforcement learning with general discounting where the learner is trying to optimize the expected discounted sum of rewards over an infinite horizon, but where the discounting function is not necessarily geometric. We show that minimizing regret with this new general discounting is equivalent to minimizing regret with uncertain episode lengths. We then design a reinforcement learning algorithm that minimizes regret with general discounting but acts for the setting with uncertain episode lengths. We instantiate  our general bound for different types of discounting, including geometric and polynomial discounting. We also show that we can obtain similar regret bounds even when the uncertainty over the episode lengths is unknown, by estimating the unknown distribution over time. Finally, we compare our learning algorithms with existing value-iteration based episodic RL algorithms on a grid-world environment.

Online Reinforcement Learning with Uncertain Episode Lengths

In offline multi-agent reinforcement learning (MARL), the agents estimate policies from a given dataset. We study reward-poisoning attack on offline MARL, where an exogenous attacker can modify the reward vectors in the dataset before the agents see the data. Similar to poisoning attacks on a single RL agent, the attacker wants to guide the agents to learn a nefarious target policy while minimizing the lp norm of reward modification. Unlike attacks on single-agent RL, the attacker takes advantage of the strategic interactions between the agents to install the target policy as a Markov Perfect Dominant Strategy Equilibrium (MPDSE) and can achieve lower reward modification costs compared to separate single-agent attacks. We then show that the attack works on various MARL agents including uncertainty-aware learners, and we exhibit linear programs to efficiently solve the offline MARL attack problem. Our work thus paves the way for studying defense in offline MARL in the future.

Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learning

Given a possibly false claim sentence, how can we automatically correct it with minimal editing? Existing methods either require a large number of pairs of false and corrected claims for supervised training or do not handle well errors spanning over multiple tokens within an utterance. In this paper, we propose VENCE, a novel method for factual error correction (FEC) with minimal edits. VENCE formulates the FEC problem as iterative sampling editing actions with respect to a target density function. We carefully design the target function with predicted truthfulness scores from an offline trained fact verification model. VENCE samples the most probable editing positions based on back-calculated gradients of the truthfulness score concerning input tokens and the editing actions using a distantly-supervised language model (T5). Experiments on a public dataset show that VENCE improves the well-adopted SARI metric by 5.3 (or a relative improvement of 11.8%) over the previous best distantly-supervised methods.

Converge to the Truth: Factual Error Correction via Iterative Constrained Editing

Question answering (QA) models for reading comprehension tend to learn shortcut solutions rather than the solutions intended by QA datasets.
QA models that have learned shortcut solutions can achieve human-level performance in shortcut examples where shortcuts are valid, but these same behaviors degrade generalization potential on anti-shortcut examples where shortcuts are invalid.
Various methods have been proposed to mitigate this problem, but they do not fully take the characteristics of shortcuts themselves into account.
We assume that the learnability of shortcuts, i.e., how easy it is to learn a shortcut, is useful to mitigate the problem.
Thus, we first examine the learnability of the representative shortcuts on extractive and multiple-choice QA datasets.
Behavioral tests using biased training sets reveal that shortcuts that exploit answer positions and word-label correlations are preferentially learned for extractive and multiple-choice QA, respectively.
We find that the more learnable a shortcut is, the flatter and deeper the loss landscape is around the shortcut solution in the parameter space.
We also find that the availability of the preferred shortcuts tends to make the task easier to perform from an information-theoretic viewpoint.
Lastly, we experimentally show that the learnability of shortcuts can be utilized to construct an effective QA training set; the more learnable a shortcut is, the smaller the proportion of anti-shortcut examples required to achieve comparable performance on shortcut and anti-shortcut examples.
We claim that the learnability of shortcuts should be considered when designing mitigation methods.

Which Shortcut Solution Do Question Answering Models Prefer to Learn?

Learning descriptive 3D features is crucial for understanding 3D scenes with diverse objects and complex structures. However, it is usually unknown whether important geometric attributes and scene context obtain enough emphasis in an end-to-end trained 3D scene understanding network. To guide 3D feature learning toward important geometric attributes and scene context, we explore the help of textual scene descriptions. Given some free-form descriptions paired with 3D scenes, we extract the knowledge regarding the object relationships and object attributes. We then inject the knowledge to 3D feature learning through three classification-based auxiliary tasks. This language-assisted training can be combined with modern object detection and instance segmentation methods to promote 3D semantic scene understanding, especially in a label-deficient regime. Moreover, the 3D feature learned with language assistance is better aligned with the language features, which can benefit various 3D-language multimodal tasks. Experiments on several benchmarks of 3D-only and 3D-language tasks demonstrate the effectiveness of our language-assisted 3D feature learning.  Code is available at https://github.com/Asterisci/Language-Assisted-3D.

Language-Assisted 3D Feature Learning for Semantic Scene Understanding

Data uncertainty is commonly observed in the images for face
recognition (FR). However, deep learning algorithms often make
predictions with high confidence even for uncertain or irrelevant
inputs. Intuitively, FR algorithms can benefit from both the estimation of uncertainty and the detection of out-of-distribution (OOD)
samples. Taking a probabilistic view of the current classification
model, the temperature scalar is exactly the scale of uncertainty
noise implicitly added in the softmax function. Meanwhile, the
uncertainty of images in a dataset should follow a prior distribution. Based on the observation, a unified framework for uncertainty
modeling and FR, Random Temperature Scaling (RTS), is proposed
to learn a reliable FR algorithm. The benefits of RTS are two-fold.
(1) In the training phase, it can adjust the learning strength of clean
and noisy samples for stability and accuracy. (2) In the test phase, it
can provide a score of confidence to detect uncertain, low-quality
and even OOD samples, without training on extra labels. Extensive
experiments on FR benchmarks demonstrate that the magnitude of
variance in RTS, which serves as an OOD detection metric, is closely
related to the uncertainty of the input image. RTS can achieve top
performance on both the FR and OOD detection tasks. Moreover,
the model trained with RTS can perform robustly on datasets with
noise. The proposed module is light-weight and only adds negligible
computation cost to the training and test processes.

Poisoning with Cerberus: Stealthy and Colluded Backdoor Attack against Federated Learning

Downloads

Next from AAAI 2023

Style-Content Metric Learning for Multidomain Remote Sensing Object Recognition

Similar lecture

DeFL: Defending Against Model Poisoning Attacks in Federated Learning via Critical Learning Periods Awareness