United States

The great challenge of handwritten mathematical expression recognition (HMER) is the complex structures of the expressions, which are directly related to the symbol spatial positions. Existing HMER methods typically employ attention mechanisms in the decoder of their models to implicitly perceive the symbol positions, or employ symbol counting and tree-based strategies to model the symbol spatial relation. However, these methods still cannot effectively capture the structural information of formulas, thus negatively impacting the symbol decoding in HMER. To deal with this problem and enhance the HMER performance, this paper proposes a novel auxiliary task, namely predicting the symbol spatial distribution map of handwritten expression images. On such basis, this paper designs a symbol spatial-aware network (SSAN) for this task, which is jointly optimized with the HMER model. Specifically, considering the similarity of the symbol spatial positions between the handwritten mathematical expression images and their corresponding printed templates, we obtain the symbol spatial distribution map by first generating printed templates from LaTeX ground-truth for handwritten formula images and then replacing the connected components of printed templates with 2D Gaussian distribution maps of the same size. Meanwhile, due to the loose alignment of the symbol spatial positions between handwritten and printed formula images, and misclassification of similar symbols, we further propose a coarse-to-fine alignment strategy and an attention-guided symbol masking strategy in SSAN to tackle these issues. Extensive experiments demonstrate that SSAN significantly improves the recognition performance of the HMER models, and the proposed auxiliary tasks are more effective in enhancing HEMR performance than existing auxiliary tasks. Code is available at https://anonymous.4open.science/r/SSAN-5223.

AAAI 2025

SSAN: A Symbol Spatial-Aware Network for Handwritten Mathematical Expression Recognition

transfer domain adaptation

multi-task learning

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

technical paper

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



Given an edge-colored graph, the goal of the Proportionally Fair Matching problem is to find a *maximum weight matching* while ensuring proportional representation of each color. The colors may correspond to demographic groups or other protected traits where we seek to ensure roughly equal representation from each group. It is known that, assuming ETH, it is impossible to approximate the problem with $\ell$ colors in time $2^{o(\ell)} n^{\mathcal{O}(1)}$ (i.e., subexponential in $\ell$). Further, even determining the existence of a non-empty matching satisfying proportionality is NP-Hard.
  
To overcome this hardness, we relax the stringent proportional fairness constraints to a probabilistic notion. We introduce a notion we call $\eta$-ProbablyApproximatelyFair, where we ensure proportionality to within a constant factor violation with high probability. We propose and analyze simple, fast algorithms with constant-factor approximation guarantees that satisfy $\eta$-ProbablyApproximateFairness for the Proportionally Fair Matching problem.

Proportionally Fair Matching via Randomized Rounding

Natural language is the most intuitive means for humans to interact with robots, making task planning based on natural language commands a longstanding area of research. Large language models (LLMs) have significantly improved task planning by enhancing understanding of language and common sense. However, current methods still face several challenges: they lack a deep understanding of physical environments, their performance relies heavily on prompt examples, LLMs are oversized and not customized for specific tasks, and the planning costs remain high. 
To overcome these issues, we introduce the GNN-Transformer Task Planner (GTTP), designed to predict task-level actions by leveraging the semantic environment and incorporating historical state data. The GTTP architecture is scalable through the use of GNN layers, while transformer layers facilitate understanding task progression. In addition, our model uses a text encoder to embed environments, allowing it to be trained on simulated datasets and applied directly in real-world scenarios. We also propose an automated data generation method that includes semantic augmentation, planning verification, and instruction generation via LLM. This method enables the collection of 14$k$ instruction-annotated tasks in the VirtualHome environment with minimal human effort. The model has been validated across diverse scenes containing up to 715 objects, achieving significantly higher success rates compared to baseline models. It has also been successfully deployed on a physical mobile manipulator, demonstrating its practical applicability and effectiveness. Our datasets and code will be made publicly available.

GNN-Transformer Task Planning Enhanced with Semantic-Driven Data Augmentation

In this paper, we focus on estimating the causal effect of an intervention over time on a dynamical system. 
To that end, we formally define causal interventions and their effects over time on discrete-time stochastic processes (DSPs). 
Then, we show under which conditions the equilibrium states of a DSP, both before and after a causal intervention, can be captured by a structural causal model (SCM). 
With such an equivalence at hand, we provide an explicit mapping from vector autoregressive models (VARs), broadly applied in econometrics, to linear, but potentially cyclic and/or affected by unmeasured confounders, SCMs. The resulting \emph{causal VAR framework} allows us to perform \emph{causal inference over time} from observational time series data.  
Our experiments on synthetic and real-world datasets show that the proposed framework achieves strong performance in terms of observational forecasting while enabling accurate estimation of the causal effect of interventions on dynamical systems. 
We demonstrate, through a case study, the potential practical questions that can be addressed using the proposed causal VAR framework.

A Practical Approach to Causal Inference over Time

Emotion Recognition in Conversations (ERC) refers to the task of automatically identifying the emotion of each utterance in conversations. Unfortunately, because the emotion of an utterance is influenced by the contextual information of the conversation, to label each utterance in ERC, the annotators are required to repeatedly read the whole conversation, making the fine-grained labeling process very costly. In this work, we suggest an efficient coarse-grained labeling pattern for ERC, where it asks annotators to assign a set of emotions for each conversation. The empirical results of our preliminary evaluations indicate the accuracy and efficiency of this conversation-level labeling pattern. Accordingly, we suggest that inducing ERC predictors with conversation-level emotion sets in a weakly-supervised learning manner can be a potential candidate for ERC, and we formally refer to this new weakly-supervised learning task as Dataless ERC (DERC). To validate its potential, we propose a simple-yet-flexible DERC framework, namely DERC with Progressive Learning (DERC-PL). We jointly update pseudo-utterance-level emotions and the ERC predictor in a self-training manner, and in particular, progressively update the ERC predictor from training subsets with lower noise densities to the ones with higher noise densities. We specify several versions of DERC-PL by applying various off-the-shelf ERC methods. Empirical results indicate that DERC-PL can be on par with existing weakly-supervised learning baselines and supervised learning ERC methods.

Utterance-level Emotion Recognition in Conversation with Conversation-level Supervision

Emotion recognition in conversation (ERC) has been promoted with diverse approaches in the recent years. However, many studies have pointed out that emotion shift and confusing labels make it difficult for models to distinguish between different emotions. Existing ERC models suffer from these problems when the emotions are forced to be mapped into single label. In this paper, we propose a multi-label classification framework for emotion recognition in conversation (ML-ERC).
Specifically, we introduce weighted supervised contrastive learning tailored for multi-label, which can also extend previous ERC models. 
Experimental results on three benchmark datasets show that the proposed approach effectively handles multiple emotions. Our model achieves improved results for identifying emotions in the most challenging settings in emotion shift and confusing labels.

Beyond Single Emotion: Multi-label Approach to Conversational Emotion Recognition

In continuous domains, reinforcement learning (RL) policies are often based on Gaussian distributions due to their natural shape and mathematical advantages. However, many continuous control tasks impose physical constraints that limit actions, leading to a bias in Gaussian policies, where the unbounded nature of their support causes a tendency toward boundary actions. This "boundary action bias" negatively impacts the performance of widely-used algorithms like Proximal Policy Optimization (PPO). Despite the prevalence of this bias, it has been overlooked in existing research and applications. In this paper, we aim to address this issue by conducting a thorough investigation into the nature and effects of boundary action bias. Through theoretical analysis and extensive experiments, we uncover the underlying dynamics of this bias in policy training. Our findings reveal that boundary actions have dual effects on performance, highlighting the importance of balancing the policy distribution. To address this challenge, we propose a truncated Gaussian policy that retains most of the favorable properties of Gaussian distributions while mitigating the bias. Additionally, we introduce a scale-adjustment technique to achieve a balanced distribution with respect to boundary actions. Experimental results on various continuous control tasks in MuJoCo demonstrate the superiority of our proposed method over conventional Gaussian policies and other approaches designed to counteract this bias.

Truncated Gaussian Policy for Debiased Continuous Control

User data spread across multiple modalities has popularized multi-modal recommender systems (MMRS). They recommend diverse content such as products, social media posts, TikTok reels, etc., based on a user-item interaction graph. With rising data privacy demands, recent methods propose unlearning private $\textbf{user data}$ from uni-modal recommender systems (RS). However, methods for unlearning $\textbf{item data}$ related to outdated user preferences, revoked licenses, and legally requested removals are still largely unexplored.

Previous RS unlearning methods are unsuitable for MMRS due to the incompatibility of their matrix-based representation with the multi-modal user-item interaction graph. Moreover, their data partitioning step degrades performance on each shard due to poor data heterogeneity and requires costly performance aggregation across shards.

This paper introduces DynMMRec, the first approach known to us for unlearning in MMRS and unlearning item data. Given a trained RS model, DynMMRec employs a novel Reverse Bayesian Personalized Ranking (BPR) objective to enable the model to forget marked data. The reverse BPR attenuates the impact of user-item interactions within the $\textit{forget}$ set, while the forward BPR reinforces the significance of user-item interactions within the $\textit{retain}$ set.
Our experiments demonstrate that DynMMRec outperforms baseline methods across various unlearning requests when evaluated on benchmark MMRS datasets. DynMMRec achieves recall performance improvements of up to $\mathbf{49.85}$% compared to baseline methods and is up to $\mathbf{1.3\times}$ faster than the Gold model, which is trained on $\textit{retain}$ set from scratch. DynMMRec offers significant advantages, including superiority in removing target interactions, preserving retained interactions, and zero overhead costs compared to previous methods. The code will be released after review.

Multi-Modal Recommendation Unlearning for Legal, Licensing, and Modality Constraints

Information Retrieval (IR) systems used in search and recommendation platforms frequently employ Learning-to-Rank (LTR) models to rank items in response to user queries. These models heavily rely on features derived from user interactions, such as clicks and engagement data. This dependence introduces cold start issues for items lacking user engagement and poses challenges in adapting to non-stationary shifts in user behavior over time. We address both challenges holistically as an online learning problem and propose BayesCNS, a Bayesian approach designed to handle cold start and non-stationary distribution shifts in search systems at scale. BayesCNS achieves this by estimating prior distributions for user-item interactions, which are continuously updated with new user interactions gathered online. This online learning procedure is guided by a ranker model, enabling efficient exploration of relevant items using contextual information provided by the ranker. We successfully deployed BayesCNS in a large-scale search system and demonstrated its efficacy through comprehensive offline and online experiments. Notably, an online A/B experiment showed a 10.60\% increase in new item interactions and a 1.05\% improvement in overall success metrics over the existing production baseline.

BayesCNS: A Unified Bayesian Approach to Address Cold Start and Non-Stationarity in Search Systems at Scale

Recent advancements in text-to-speech and speech conversion technologies have enabled the creation of highly convincing synthetic speeches. While these innovations offer numerous practical benefits, they also cause significant security challenges when maliciously misused. Therefore, there is an urgent need to detect these synthetic speeches. Phoneme features provide a powerful speech representation for deepfake detection. However, previous phoneme-based detection approaches typically focused on specific phonemes, overlooking temporal inconsistencies across the entire phoneme sequence. In this paper, we develop a new mechanism for detecting speech deepfakes by identifying the inconsistencies of phoneme-level speech features. We design an adaptive phoneme pooling technique that extracts sample-specific phoneme-level features from frame-level speech data. By applying this technique to features extracted by pre-trained audio models on previously unseen deepfake datasets, we demonstrate that deepfake samples often exhibit phoneme-level inconsistencies when compared to genuine speech. To further enhance detection accuracy, we propose a deepfake detector that uses a graph attention network to model the temporal dependencies of phoneme-level features. Additionally, we introduce a random phoneme substitution augmentation technique to increase feature diversity during training. Extensive experiments on four benchmark datasets demonstrate the superior performance of our method over existing state-of-the-art detection methods.

Phoneme-Level Feature Discrepancies: A Key to Detecting Sophisticated Speech Deepfakes

To build a responsible data economy and protect data ownerhip, it is crucial to enable learning models from separate, heterogeneous data sources without centralization. For example, federated learning (FL) aims to train models across massive remote devices or isolated organizations, while keeping user data local. However, federated learning can face critical practical issues such as scalability, noisy samples, biased learning systems or procedures, and privacy leakage. At the intersection between optimization, trustworthy (fair, robust, and private) ML, and learning in heterogeneous environments, my research aims to support scalable and responsible data sharing to collectively build intelligent models.

Downloads

Next from AAAI 2025

Proportionally Fair Matching via Randomized Rounding

Similar lecture

Symbol tuning improves in-context learning in language models

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2025

Proportionally Fair Matching via Randomized Rounding

Similar lecture

Symbol tuning improves in-context learning in language models

Downloads