Singapore

The design of Large Language Models (LLMs) has long been hampered by a fundamental conflict within their core attention mechanism: its remarkable expressivity is built upon a computational complexity of $O(H \cdot N^2)$ that grows quadratically with the context size ($N$) and linearly with the number of heads ($H$). This standard implementation harbors significant computational redundancy, as all heads independently compute attention over the same sequence space. Existing sparse methods, meanwhile, often trade information integrity for computational efficiency. To resolve this efficiency-performance trade-off, we propose SPAttention, whose core contribution is the introduction of a new paradigm we term Principled Structural Sparsity. SPAttention does not merely drop connections but instead reorganizes the computational task by partitioning the total attention workload into balanced, non-overlapping distance bands, assigning each head a unique segment. This approach transforms the multi-head attention mechanism from $H$ independent $O(N^2)$ computations into a single, collaborative $O(N^2)$ computation, fundamentally reducing complexity by a factor of $H$. The structured inductive bias compels functional specialization among heads, enabling a more efficient allocation of computational resources from redundant modeling to distinct dependencies across the entire sequence span. Extensive empirical validation on the OLMoE-1B-7B and 0.25B-1.75B model series demonstrates that while delivering an approximately two-fold increase in training throughput, its performance is on par with standard dense attention, even surpassing it on select key metrics, while consistently outperforming representative sparse attention methods including Longformer, Reformer, and BigBird across all evaluation metrics. Our work demonstrates that thoughtfully designed structural sparsity can serve as an effective inductive bias that simultaneously improves both computational efficiency and model performance, opening a new avenue for the architectural design of next-generation, high-performance LLMs.

AAAI 2026

Making Every Head Count: Sparse Attention Without the Speed-Performance Trade-off

sparse

attention

### Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

Message from the General Chair: 
*It is my great pleasure and honor to welcome you to the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025), held in beautiful Vienna, Austria, from July 27 to August 1, 2025. ACL2025continues our field’s tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.* 
[Read more](https://drive.google.com/file/d/1GI_hvOpjswAuYdUTromfeDiPpCcqidwg/view?usp=sharing)

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2025". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you need to log in with the **email address you registered with**. 

ACL 2025

Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

technical paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

Multilingual Alignment is an effective and representative paradigm to enhance LLMs' multilingual capabilities, which transfers the capabilities from the high-resource languages to the low-resource languages. Meanwhile, some research on language-specific neurons provides a new perspective to analyze and understand LLMs' mechanisms. However, we find that there are many neurons that are shared by multiple but not all languages and cannot be correctly classified. In this work, we propose a ternary classification methodology that categorizes neurons into three types, including language-specific neurons, language-related neurons, and language-agnostic neurons. And we propose a corresponding identification algorithm to distinguish these different types of neurons. Furthermore, based on the distributional characteristics of different types of neurons, we divide the LLMs' internal process for multilingual inference into four parts: (1) multilingual understanding, (2) shared semantic space reasoning, (3) multilingual output space transformation, and (4) vocabulary space outputting. Additionally, we systematically analyze the models before and after alignment with a focus on different types of neurons. We also analyze the phenomenon of ''Spontaneous Multilingual Alignment''. Overall, our work conducts a comprehensive investigation based on different types of neurons, providing empirical results and valuable insights to better understand multilingual alignment and multilingual capabilities of LLMs.

How Does Alignment Enhance LLMs’ Multilingual Capabilities? A Language Neurons Perspective

Value decomposition is a central approach in multi-agent reinforcement learning (MARL), enabling centralized training with decentralized execution by factorizing the global value function into local values. To ensure individual-global-max (IGM) consistency, existing methods either enforce monotonicity constraints, which limit expressive power, or adopt softer surrogates at the cost of algorithmic complexity. In this work, we present a dynamical systems analysis of non-monotonic value decomposition, modeling learning dynamics as continuous-time gradient flow. We prove that, under approximately greedy exploration, all zero-loss equilibria violating IGM consistency are unstable saddle points, while only IGM-consistent solutions are stable attractors of the learning dynamics. Extensive experiments on both synthetic matrix games and challenging MARL benchmarks demonstrate that unconstrained, non-monotonic factorization reliably recovers IGM-optimal solutions and consistently outperforms monotonic baselines. Additionally, we investigate the influence of temporal-difference targets and exploration strategies, providing actionable insights for the design of future value-based MARL algorithms.

Beyond Monotonicity: Revisiting Factorization Principles in Multi-Agent Q-Learning

Financial time series forecasting is both highly significant and challenging. Previous approaches typically standardized time series data before feeding it into forecasting models, which inherently leads to a loss of important information during the encoding process. Moreover, past time series models generally require fixed numbers of variables or lookback window lengths, which further limits the scalability of time series forecasting. In addition, interpretability and uncertainty in forecasting remain underexplored, as these factors directly impact the reliability and practical value of predictions. To address these issues, we first constructed a diverse financial image-text dataset (FVLDB) and developed the Uncertainty-adjusted Group Relative Policy Optimization (UARPO) method to enable the model not only output predictions but also analyze the uncertainty of those predictions. We then launched FinZero, a multi-modal pre-trained model finetuned by UARPO to perform reasoning, prediction, and analytical understanding on the FVLDB financial time series. Extensive experiments validate that FinZero exhibits strong adaptability and scalability. After fine-tuning with UARPO, FinZero achieves an approximate 13.48% improvement in prediction accuracy over GPT-4o in the high-confidence group, demonstrating the effectiveness of reinforcement learning fine-tuning of multi-modal large model for financial time series forecasting tasks. The FVLDB dataset is available at anonymous URL: https://drive.google.com/file/d/1nYUkB9Qf3O3B8byTXvVXg9jGmVL3vxT-/view?usp=share_link.

FinZero: Launching Multi-modal Financial Time Series Forecasting with Large Reasoning Model

We present CAIA, a benchmark exposing a critical blind spot in AI evaluation: the inability of state-of-the-art models to operate in adversarial, high-stakes environments where misinformation is weaponized and errors are irreversible. While existing benchmarks measure task completion in controlled settings, real-world deployment demands resilience against active deception. Using cryptocurrency markets as a natural laboratory, where $30 billion was lost to exploits in 2024, we evaluate 17 leading models on 178 time-anchored tasks requiring agents to distinguish truth from manipulation, navigate fragmented information landscapes, and make irreversible financial decisions under adversarial pressure. Our results reveal a fundamental capability gap: without tools, even frontier models achieve only 12-28% accuracy on tasks junior analysts routinely handle. Tool augmentation improves performance but plateaus at 67.4% (GPT-5) versus 80% human baseline, despite unlimited access to professional resources. Most critically, we uncover a systematic tool selection catastrophe: models preferentially choose unreliable web search (55.5% of invocations) over authoritative blockchain data, falling for SEO-optimized misinformation and social media manipulation. This behavior persists even when correct answers are directly accessible through specialized tools, suggesting foundational limitations rather than knowledge gaps. The implications extend beyond cryptocurrency to any domain where adversaries actively exploit AI weaknesses, e.g. cybersecurity, content moderation, etc. Our finding that Pass@k metrics mask dangerous trial-and-error behavior challenges fundamental assumptions about autonomous deployment. We release CAIA with contamination controls and continuous updates, establishing adversarial robustness as a necessary condition for trustworthy AI autonomy. The benchmark reveals that current models, despite impressive reasoning scores, remain fundamentally unprepared for environments where intelligence must survive active opposition.

When Hallucination Costs Millions: Benchmarking AI Agents in High-Stakes Adversarial Financial Markets

Large Language Models (LLMs) have shown remarkable capabilities in document processing, but their inability to provide visual grounding without OCR dependencies poses significant challenges in business-critical applications. Current solutions either require model fine-tuning or rely on external OCR services, introducing additional costs, latency, and limitations in handling derived information. This paper presents ViG-LLM, a novel framework that enables closed-box LLMs to generate localization information through a multi-agent system combining U-Net-based layout deconstruction with viewport identification tasks. Evaluated on the FATURA and CORD dataset, our framework achieves perfect accuracy over spatial reasoning tuned LLM like Amazon Nova Pro, while demonstrating superior template-specific consistency. The framework maintains robust performance across LLM architectures while reducing operational costs by 60% compared to Textract-based solutions. In real-world document processing applications, the framework helps retain the high reasoning capabilities of the system in document information extraction tasks while improving explainability, reliability and human interaction for information verification. Through human-in-the-loop learning and closed-box prompt alignment techniques, ViG-LLM provides a robust, adaptable solution for visual grounding tasks in document processing workflows.

ViG-LLM: Enhancing Visual Grounding Capabilities in Closed-Box LLMs for Document Information Extraction without OCR Dependencies

Making use of swarm methods in financial market modeling of liquidity, and techniques from financial analysis in swarm analysis, holds the potential to advance both research areas. In swarm research, the use of game theory methods holds the promise of explaining observed phenomena of collective utility adherence with rational self-interested swarm participants. In financial markets, a better understanding of how independent financial agents may self-organize for the betterment and stability of the marketplace would be a boon for market design researchers. This paper unifies Liquidity Games, where trader payoffs depend on aggregate liquidity within a trade, with Rational Swarms, where decentralized agents use difference rewards to align self-interested learning with global objectives. We offer a theoretical frameworks where we define a swarm of traders whose collective objective is market liquidity provision while maintaining agent independence. Using difference rewards within a Markov team games framework, we show that individual liquidity-maximizing behaviors contribute to overall market liquidity without requiring coordination or collusion. This Financial Swarm model provides a framework for modeling rational, independent agents where they achieve both individual profitability and collective market efficiency in bilateral asset markets.

Multiagent Reinforcement Learning for Liquidity in Bond Markets

As AI becomes increasingly powerful and ubiquitous, it is disrupting skills and displacing workers. NSF’s National AI Institute for Adult Learning and Online Education (AI-ALOE) posits that AI can be part of the solution to the growing problem if we can use AI for reskilling, upskilling, and workforce development at scale. The long-term vision of AI-ALOE is to develop and use AI technologies to enhance the proficiency of online education for all adult learners, using in-person education as a benchmark. The day-to-day mission of AI-ALOE is to conduct responsible research into AI that is grounded in theories of human cognition and learning and derived from the scientific process of learning engineering. I will describe ongoing research at AI-ALOE.

AI for Reskilling, Upskilling, and Workforce Development

Small Data: A New Paradigm for the Next Generation of AI

Current algorithms for steering LLM behavior are often implemented for specific use cases and tasks. To help provide a more general purpose approach to steering model behavior, IBM has recently developed two toolkits: AI Steerability 360 (AISteer360) and In-Context Explainability 360 (ICX360). This hands-on lab will provide a comprehensive walkthrough of these toolkits.

Participants will first be guided through a conceptual overview of how to steer model behavior across four model control surfaces: input, structural, state, and output steering. Through a series of interactive coding sessions, attendees will implement steering methods on a running example focused on steering a model to produce less toxic outputs. The lab will demonstrate how to construct steering controls for fine-grained model intervention, use cases for specific model tasks, and benchmarks for comparing steering methods on a given use case.

Closing the loop, participants will learn how the ICX360 toolkit can be used to understand why a given (steered) model produced a given output, and how to use these insights to guide refined controls. The session will build progressively from concept to implementation, ensuring participants understand how to create an end-to-end steering workflow.

Prerequisites: Participants should be comfortable with Python programming and have some prior exposure to LLMs and natural language processing (prompting, fine-tuning, etc.). Experience using the Hugging Face transformers package will be beneficial but not required. Participants must bring their own laptop with a Python IDE installed.

Remote compute access will be facilitated through Hugging Face and Google Colab. All setup instructions and access details will be communicated to enrolled participants in advance via the lab’s website.

Learning to Steer Large Language Models

Agentic AI is poised to redefine intelligent decision-making in Industry 4.0 by enabling autonomous systems to perceive, reason, and act over multimodal data sources such as sensor streams, structured knowledge bases, and unstructured maintenance logs. Despite the rapid progress of large language models and multimodal learning, building deployable and trustworthy agentic systems remains challenging due to fragmented data silos, integration complexity, lack of explainability, and limited evaluation protocols.
This tutorial offers a comprehensive, hands-on walkthrough of the full lifecycle of multimodal agentic AI systems—from inception to productization. Participants will begin by learning foundational agent architectures and reasoning strategies, including Plan-Execute, ReAct, Reflexion, RAFA, and hybrid symbolic-neural approaches. Building on these concepts, the tutorial features two interactive lab sessions: (i) addressing integration challenges in smart manufacturing using an open-source multi-agent environment, and (ii) benchmarking agent performance, reasoning, and explainability in an enterprise-scale industrial simulation.
Along the way, participants will engage with state-of-the-art tools and workflows for data integration, trace alignment, and failure diagnosis. They will explore evaluation techniques that combine human feedback, sensor-grounded truth, and emerging LLM-as-a-judge methods to assess reliability and trustworthiness. Additional emphasis will be placed on governance and monitoring frameworks, as well as visualizations for traceability and introspection, enabling participants to design explainable and auditable deployments.
The tutorial leverages two open-source environments AssetOpsBench, a simulation platform with 141 curated industrial scenarios, and SmartPilot, a smart manufacturing multi-agent system for multimodal data integration and reasoning, providing participants with realistic data and experimental workflows. Together, these environments demonstrate how to evaluate agents under diverse operating conditions, such as anomaly detection, fault progression, and post-maintenance recovery.

Downloads

Next from AAAI 2026

How Does Alignment Enhance LLMs’ Multilingual Capabilities? A Language Neurons Perspective

Similar lecture

DAM: Dynamic Attention Mask for Long-Context Large Language Model Inference Acceleration