Austria

We investigate the linguistic abilities of multimodal large language models in reference resolution tasks featuring simple but abstract visual stimuli such as color patches and color grids. Although the task may not seem challenging for today&#39;s language models, being straightforward for human dyads, we consider it a highly relevant probe of the pragmatic capabilities of MLLMs. Indeed, our results and analyses suggest that basic pragmatic capabilities such as context-dependent interpretation of color descriptions still constitute major challenges for state-of-the-art MLLMs.

ACL 2025

Are Multimodal Large Language Models Pragmatically Competent Listeners in Simple Reference Resolution Tasks?

reference games

mllms

pragmatics

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

Registration Required

ACL 2026

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

We investigate the linguistic abilities of multimodal large language models in reference resolution tasks featuring simple but abstract visual stimuli such as color patches and color grids. Although the task may not seem challenging for today's language models, being straightforward for human dyads, we consider it a highly relevant probe of the pragmatic capabilities of MLLMs. Indeed, our results and analyses suggest that basic pragmatic capabilities such as context-dependent interpretation of color descriptions still constitute major challenges for state-of-the-art MLLMs.

poster

### Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

Message from the General Chair: 
*It is my great pleasure and honor to welcome you to the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025), held in beautiful Vienna, Austria, from July 27 to August 1, 2025. ACL2025continues our field’s tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.* 
[Read more](https://drive.google.com/file/d/1GI_hvOpjswAuYdUTromfeDiPpCcqidwg/view?usp=sharing)

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2025". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you need to log in with the **email address you registered with**. 

Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

Large language models (LLMs) are arguably the most predictive models of human cognition available. Despite their impressive human-alignment, LLMs are often labeled as "just next-token predictors" that purportedly fall short of genuine cognition. We argue that these deflationary claims need further justification. Drawing on prominent cognitive and artificial intelligence research, we critically evaluate two forms of "Justaism" that dismiss LLM cognition by labeling LLMs as "just" simplistic entities without specifying or substantiating the critical capacities they supposedly lack. Our analysis highlights the need for a more measured discussion of LLM cognition, aiming to better inform future research and the development of artificial intelligence.

A rebuttal of two common deflationary stances against LLM cognition

Information Extraction (IE) systems are traditionally domain-specific, requiring costly
adaptation that involves expert schema design,
data annotation, and model training. While
Large Language Models have shown promise
in zero-shot IE, performance degrades significantly in unseen domains where label definitions differ. This paper introduces GUIDEX,
a novel method that automatically defines
domain-specific schemas, infers guidelines,
and generates synthetically labeled instances,
allowing for better out-of-domain generalization. Fine-tuning Llama 3.1 with GUIDEX
sets a new state-of-the-art across seven zeroshot Named Entity Recognition benchmarks.
Models trained with GUIDEX gain up to 7 F1
points over previous methods without humanlabeled data, and nearly 2 F1 points higher
when combined with it. Models trained on
GUIDEX demonstrate enhanced comprehension of complex, domain-specific annotation
schemas. Code, models, and synthetic datasets
are available at neilus03.github.io/guidex.com

GUIDEX: Guided Synthetic Data Generation for Zero-Shot Information Extraction

Emotions have been shown to play a role in argument convincingness, yet this aspect is underexplored in the natural language processing (NLP) community. Unlike prior studies that use static analyses, focus on a single text domain or language, or treat emotion as just one of many factors, we introduce a dynamic framework inspired by manipulation checks commonly used in psychology and social science; leveraging LLM-based manipulation checks, this framework examines the extent to which perceived emotional intensity influences perceived convincingness. Through human evaluation of arguments across different languages, text domains, and topics, we find that in over half of cases, judgments of convincingness remain unchanged despite variations in perceived emotional intensity; when emotions do have an impact, they more often enhance rather than weaken convincingness. We further analyze how 11 LLMs behave in the same scenario, finding that while LLMs generally mirror human patterns, they struggle to capture nuanced emotional effects in individual judgments.

Do Emotions Really Affect Argument Convincingness? A Dynamic Approach with LLM-based Manipulation Checks

Process Reward Models (PRMs) have demonstrated promising results in mathematical reasoning, but existing process annotation approaches, whether through human annotations or Monte Carlo simulations, remain computationally expensive. In this paper, we introduce Step COmpression for Process Estimation (SCOPE), a novel compression-based approach that significantly reduces annotation costs. We first translate natural language reasoning steps into code and normalize them through Abstract Syntax Tree, then merge equivalent steps to construct a prefix tree. Unlike simulation-based methods that waste numerous samples on estimation, SCOPE leverages a compression-based prefix tree where each root-to-leaf path serves as a training sample, reducing the complexity from O(NMK) to O(N) We construct a large-scale dataset containing 509K samples with only 5% of the computational resources required by previous methods. Empirical results demonstrate that PRMs trained on our dataset consistently outperform existing automated annotation approaches on both Best-of-N strategy and ProcessBench.

SCOPE: Compress Mathematical Reasoning Steps for Efficient Automated Process Annotation

Citation context analysis (CCA) is a field of research studying the role and purpose of citation in scientific discourse. While most of the efforts in CCA have been focused on elaborate characterization schemata to assign function or intent labels to individual citations, the citation context as the basis for such a classification has received rather limited attention. This relative neglect, however, has led to the precedence of vague definitions and restricting assumptions, limiting the citation context in its expressiveness. It is a common practice, for example, to restrict the context to the citing sentence. While this might be enough to cover mentions and background citations, more influential ones are often thoroughly discussed, extending beyond a one-sentence context window. To address this concern, we analyze the semantic structure of citation contexts in terms of their elemental dimensions and distribution throughout the citing text. To evaluate this approach, we construct and publish the FineCite Corpus containing 1,056 manually annotated fine-grained citation contexts. Our experiments on established CCA benchmarks demonstrate the effectiveness of our finer-grained context definition, showing improvement compared to state-of-the-art approaches. We will release our code and dataset to the public upon acceptance.

FINECITE: A Novel Approach on Fine-Grained Citation Context Analysis

Real-world applications of KBQA require models to handle unanswerable questions with a limited volume of in-domain labeled training data. We propose the novel task of few-shot transfer for KBQA with unanswerable questions and contribute two new datasets for performance evaluation. We present FUn-FuSIC -- a novel solution for our task that extends FuSIC-KBQA, the state-of-the-art few-shot transfer model for answerable-only KBQA. We first note that FuSIC-KBQA's iterative repair makes a strong assumption that all questions are answerable. As a remedy, we propose Feedback for Unanswerability (FUn), which uses iterative repair using feedback from a suite of strong and weak verifiers, and an adaptation of self-consistency for unanswerabilty to better assess the answerability of a question. Our experiments show that FUn-FuSIC significantly outperforms suitable adaptations of multiple LLM-based and supervised SoTA models on our task, while establishing a new SoTA for answerable few-shot transfer as well.

An Iterative Repair Approach for Few-shot Transfer in KBQA with Unanswerability

Small large language models (sLLMs) offer the advantage of being lightweight and efficient, which makes them suitable for resource-constrained environments. However, sLLMs often struggle to maintain topic consistency in task-oriented dialogue systems, which is critical for scenarios such as service chatbots. Specifically, it is important to ensure that the model denies off-topic or malicious inputs and adheres to its intended functionality so as to prevent potential misuse and uphold reliability. Towards this, existing activation engineering approaches have been proposed to manipulate internal activations during inference. While these methods are effective in certain scenarios, our preliminary experiments reveal their limitations in ensuring topic adherence. Therefore, to address this, we propose a novel approach termed \textbf{En}tropy-scaled \textbf{S}teering vectors for \textbf{To}pic \textbf{M}aintenance (EnSToM). EnSToM dynamically adjusts the steering intensity based on input uncertainty, which allows the model to handle off-topic distractors effectively while preserving on-topic accuracy. Our experiments demonstrate that EnSToM achieves significant performance gain with a relatively small data size compared to fine-tuning approaches. By improving topic adherence without compromising efficiency, our approach provides a robust solution for enhancing sLLM-based dialogue systems\footnote{We will release our source code upon acceptance}.

EnSToM: Enhancing Dialogue Systems with Entropy-Scaled Steering Vectors for Topic Maintenance

Programming with a coding assistant is a fundamentally interactive process, yet existing static benchmarks fail to capture key features of model-user collaboration. We introduce an interactive evaluation pipeline to examine how LLMs incorporate different types of feedback in a collaborative setting, in which we obfuscate the input of static coding benchmarks so that the code model must interact with a simulated user. Across 10 models and 3 datasets, the relative rankings of models often permute greatly between static and interactive settings, despite models being fairly robust to feedback that contains errors. We also observe that similarly effective feedback types differ in terms of how models respond to higher- vs. lower-quality feedback. Moreover, feedback type impacts the degree to which the models make aesthetic or behavioral edits to their output. Our work aims to "re-evaluate" model coding capabilities through an interactive lens toward bridging the gap between existing evaluations and real-world usage.

When Benchmarks Talk: Re-Evaluating Code LLMs with Interactive Feedback

We introduce "Generative Fusion Decoding'' (GFD), a novel shallow fusion framework designed to integrate large language models (LLMs) into cross-modal text recognition systems, such as automatic speech recognition (ASR) and optical character recognition (OCR). GFD is methodologically orthogonal to prior work and operates across mismatched token spaces of different models by calculating likelihood at the byte level, enabling seamless fusion during the decoding process. This plug-and-play framework is compatible with various auto-regressive models and does not require re-training for feature alignment or two-pass strategies. GFD offers two key benefits with LLM interaction: effective domain adaptation using in-context learning, and maintaining contextual semantics over long-form audio. These advantages have been shown to effectively reduce error rates for ASR in both English and Chinese. Additionally, GFD's extensibility goes beyond prior work as exhibited in significant improvement on instructed speech recognition tasks and OCR, which have not been reported before. Finally, we provide a time complexity analysis demonstrating the efficiency of GFD, highlighting its potential in real-time error correction. Overall, GFD parallely integrates LLMs into text recognition systems, enhancing accuracy and domain adaptativeness while maintaining decoding efficiency.

Let’s Fuse Step by Step: A Generative Fusion Decoding Algorithm with LLMs for Robust and Instruction-Aware ASR and OCR

The conversational capabilities of Large Language Models (LLMs) suggest that they may be able to perform as automated talk therapists. It is crucial to know if these systems would be effective and adhere to known standards. We present a counsellor chatbot that focuses on motivating tobacco smokers to quit smoking. It uses a state-of-the-art LLM and a widely applied therapeutic approach called Motivational Interviewing (MI), and was evolved in collaboration with clinician-scientists with expertise in MI. We also describe and validate an automated assessment of both the chatbot's adherence to MI and client responses. The chatbot was tested on 106 participants, and their confidence that they could succeed in quitting smoking was measured before the conversation and one week later. Participants' confidence increased by an average of 1.7 on a 0-10 scale. The automated assessment of the chatbot showed adherence to MI standards in 98% of utterances, higher than human counsellors. The chatbot scored well on a participant-reported metric of perceived empathy but lower than typical human counsellors. Furthermore, participants' language indicated a good level of motivation to change, a key goal in MI. These results suggest that the automation of talk therapy with a modern LLM has promise.

Downloads

Next from ACL 2025

A rebuttal of two common deflationary stances against LLM cognition

Similar lecture

Reference Attack: A New Cross-Modal Jailbreaking Attack against Multimodal Large Language Models