United States

Chain-of-Thought (CoT) prompting is the dominant strategy for eliciting step-by-step reasoning in LLMs, yet its effect on model behavior remains poorly understood. We present a controlled study of when reasoning augmentation helps versus hurts in code generation — a task requiring deductive reasoning under formal constraints — spanning three architectures: a 2×2 design with Qwen2.5-Coder-1.5B and DeepSeek-Coder-1.3B (base and instruction-tuned variants) on HumanEval, MBPP, and LiveCodeBench, plus CodeLlama-7B. Our key finding is that instruction tuning reverses CoT&#39;s effect on the same base architecture: CoT significantly improves Qwen base (+13.4%, $p&lt;0.001$) but significantly degrades Qwen instruct ($-15.2\%$, $p&lt;0.001$). DeepSeek remains insensitive regardless of training regime (all $p&gt;0.2$). To explain why, we apply layer-wise linear probing and find that all models encode prompt type (Direct vs. CoT) by Layer 1–4 ($&gt;$90% accuracy) — universally, whether CoT helps or hurts. This reveals a key mechanistic insight: representation does not determine interpretation. The same internal signal for &quot;this is a reasoning prompt&quot; drives opposite behavioral outcomes depending on training regime. Building on this, we develop a probe-guided style router that selects from 12 prompt styles per problem via a single forward pass (84ms). The router is statistically indistinguishable from the best fixed style in 7/8 settings (McNemar&#39;s, all $p_{\text{Best}}&gt;0.1$) and significantly outperforms CoT where CoT is most harmful ($p=0.012$, $h=+0.40$). Our findings argue for model-aware prompt strategies informed by internal representations rather than one-size-fits-all reasoning augmentation.

ACL 2026

Think Less, Code Better: Probing When Chain-of-Thought Hurts and How to Route Around It

## Welcome!
"I am excited to welcome you to this year’s edition of the Conference on Empirical Methods in Natural Language Processing! Importantly, it marks the 30th edition of EMNLP. With over 8,000 submissions, more than 3,000 accepted papers, and thousands of attendees, we have come a long way from that first
workshop, which had 14 accepted papers. As the field looks ahead, Suzhou is the fitting location for celebrating this milestone: rooted in a long literary tradition, yet modern and forward-looking, and home to a large share of the NLP community." 

*Message from the General Chair, Dirk Hovy*

EMNLP 2025

Celebrate 30 Years of EMNLP! 
EMNLP 2025 will be held in Suzhou, China from November 5th to November 9th, 2025.

Chain-of-Thought (CoT) prompting is the dominant strategy for eliciting step-by-step reasoning in LLMs, yet its effect on model behavior remains poorly understood. We present a controlled study of when reasoning augmentation helps versus hurts in code generation — a task requiring deductive reasoning under formal constraints — spanning three architectures: a 2×2 design with Qwen2.5-Coder-1.5B and DeepSeek-Coder-1.3B (base and instruction-tuned variants) on HumanEval, MBPP, and LiveCodeBench, plus CodeLlama-7B. Our key finding is that instruction tuning reverses CoT's effect on the same base architecture: CoT significantly improves Qwen base (+13.4%, $p<0.001$) but significantly degrades Qwen instruct ($-15.2\%$, $p<0.001$). DeepSeek remains insensitive regardless of training regime (all $p>0.2$). To explain why, we apply layer-wise linear probing and find that all models encode prompt type (Direct vs. CoT) by Layer 1–4 ($>$90% accuracy) — universally, whether CoT helps or hurts. This reveals a key mechanistic insight: representation does not determine interpretation. The same internal signal for "this is a reasoning prompt" drives opposite behavioral outcomes depending on training regime. Building on this, we develop a probe-guided style router that selects from 12 prompt styles per problem via a single forward pass (84ms). The router is statistically indistinguishable from the best fixed style in 7/8 settings (McNemar's, all $p_{\text{Best}}>0.1$) and significantly outperforms CoT where CoT is most harmful ($p=0.012$, $h=+0.40$). Our findings argue for model-aware prompt strategies informed by internal representations rather than one-size-fits-all reasoning augmentation.

technical paper

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/acl2026

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

Garden-path sentences offer a controlled probe of incremental sentence processing because they require a reader to revise an initially plausible parse when a later region disambiguates the structure. We present an architecture-aware comparison of garden-path recovery in causal and masked language models using 100 ambiguous--control pairs (200 sentences) spanning three classic constructions: NP/Z, NP/S, and MV/RR. Causal models are evaluated with left-to-right word surprisal, whereas masked models are evaluated with pseudo-surprisal derived from masked language model scoring. Beyond the disambiguating region, we analyze cumulative excess surprisal, area-under-curve recovery summaries, and layer-wise hidden-state divergence between ambiguous and control counterparts. Across the headline-valid model set, causal models show larger disruption than masked models overall, with the clearest family-level difference on NP/Z constructions. GPT-2-medium yields the strongest NP/Z spike, while DeBERTa-v3-base shows little evidence of disruption on the same construction. Hidden-state analyses further reveal a qualitative family difference: excluding the embedding layer, masked models peak in very shallow layers, whereas GPT-2 and GPT-2-medium peak substantially deeper. These results suggest that decoder-only models exhibit stronger online disruption at the point of syntactic revision, while bidirectional encoders appear comparatively buffered at the disambiguator, consistent with their access to right context. More broadly, the findings argue that garden-path evaluation should emphasize recovery dynamics, not merely end-state plausibility or task accuracy.

Garden Path Recovery in Causal and Masked Language Models

Counterspeech offers a way to tackle harmful content online without restricting freedom of expression. This work explores counterspeech generation using small language models (SLMs) as lightweight and cost-effective alternatives to large language models. We evaluate SLMs ranging from 100 million to 3 billion parameters using simple prompting strategies as well as fine-tuning, combining automatic and robust human evaluations. Our findings demonstrate that small language models can generate relevant, coherent, and high-quality counterspeech, suggesting their potential suitability for efficient and responsible deployments.

Counterspeech Generation using Small Language Models

Shortcut learning remains a major obstacle to robust NLP systems: models can achieve high in-distribution accuracy by relying on surface cues that fail under distribution shift. We study whether shortcut reliance can be diagnosed and mitigated in small instruction-tuned language models using a simple representation-level quantity. We fine-tune Gemma 3 1B Instruct and Llama 3.2 1B on two synthetic sentiment shortcuts in SST-2 and one natural shortcut in MNLI based on lexical overlap. During training, we fit linear probes for the task label and the shortcut attribute at every layer and define CDRE as the absolute cosine similarity between the two probe directions. Across settings, increasing shortcut prevalence produces a sharp rise in the robustness gap between shortcut-aligned and shortcut-free test sets, and higher deep-layer CDRE tracks this degradation. At a 99\% shortcut ratio, Llama's clean accuracy on capitalization-biased SST-2 drops from 93.2\% at 0\% bias to 49.0\%, while Gemma drops from 91.8\% to 60.2\%. A CDRE-regularized objective substantially improves robustness for capitalization and lexical-overlap shortcuts, but offers little benefit for a speaker-prefix shortcut whose learned directions are already nearly orthogonal. These results show that probe-derived representation entanglement provides a reliable signal of harmful shortcut reliance and offers a practical criterion for determining when shortcut mitigation is likely to be effective.

Measuring and Mitigating Shortcut Reliance in Language Models with Probe-Based Representation Entanglement

Automatic Speech Recognition (ASR) has achieved strong performance for high-resource languages, but dense intra-sentential code-switched speech in African low-resource settings remains underexplored. Existing multilingual and pretrained ASR systems improve general recognition accuracy, yet they remain weak at switch regions, are sensitive to language imbalance during adaptation, and are typically evaluated with metrics that obscure switching-specific errors. This thesis proposes a self-adaptive and epistemic uncertainty-guided framework for African low-resource code-switched ASR, using Hausa--English (Engausa) and Hausa--Yor\`ub\'a as case studies. The work investigates three linked questions: (1) how to design a linguistically informed code-switched corpus with explicit switch-region annotation and labeled/unlabeled partitions for adaptive learning, (2) whether epistemic uncertainty is systematically elevated around switch regions and can guide pseudo-label selection in semi-supervised training, and (3) whether switch-aware adaptation with auxiliary language identification and boundary supervision can reduce recognition errors without increasing catastrophic forgetting. The long-term goal is to develop scalable and data-efficient ASR systems that model code-switching as a structured linguistic phenomenon rather than as noise in multilingual African speech.

Thesis Proposal: Self-Adaptive and Epistemic Uncertainty-Guided ASR of Dense Intra-Sentential Code-Switched Speech for African Low-Resource Languages

Low-resource agglutinative languages, characterized by rich morphological inflection and severe vocabulary sparsity in corpora, have long posed numerous challenges in the field of representation learning. Word-level representations preserve semantic integrity but struggle to handle sparse surface forms, whereas morpheme-level representations, though easier to learn, often lack holistic semantic information. Existing multi-granularity methods are typically modeled at the word and phrase levels, with very limited application to low-resource agglutinative languages. Focusing on the morphemes of agglutinative languages, this paper proposes MAGNet, a morphology-aware gated multi-granularity pre-training framework. At the morpheme granularity, this framework leverages morphological knowledge and integrates morpheme segmentation with morphological tagging to construct fine-grained representations. It further introduces a morphology-aware masked language modeling objective to facilitate the model in learning functional morphological regularities. Meanwhile, at the word granularity, a word-level encoder is employed to capture contextual semantics and maintain its semantic coherence.Finally, a gated fusion mechanism dynamically fuses representations of different granularities according to the context. Experiments conducted on two low-resource agglutinative languages, Mongolian and Turkish, for the tasks of dependency parsing and named entity recognition (NER) demonstrate that our method achieves consistent performance improvements over strong baseline models. Ablation studies further validate the complementary roles of morphological tagging and whole-word modeling in efficient representation learning.

Morphology-Aware Multi-Granularity Representation Learning for Agglutinative Languages

Most languages lack labeled evaluation benchmarks for large language models (LLMs). Creating such benchmarks requires native speakers, domain expertise, and answer annotation - resources unavailable for the vast majority of languages. We investigate whether a model's internal processing signals - such as generation entropy and tokenizer statistics - correlate with its actual accuracy on a language, with the long-term goal of estimating language competence without labeled data. Our key observation is that for languages a model does not know, both tokenizer segmentation and generation entropy become highly variable across questions, whereas for known languages they remain consistent. We call this the inconsistency hypothesis and test it on 11 instruction-tuned LLMs (1B-70B parameters) across 14 language-script varieties (12 Turkic plus English and Russian controls). Standard deviation outperforms the mean as an accuracy predictor in 21 of 25 greedy-pass family $\times$ measure combinations (binomial $p < 0.001$, though the measures are not fully independent); sampling-based measures show no such pattern (4 of 20). A three-feature proxy combining a tokenizer coverage ratio chars\_std / fertility), entropy standard deviation, and the model's EN/RU benchmark accuracy achieves leave-one-language-out $R^2 = 0.75$ and Kendall $\tau = 0.72$ on 154 model-language pairs, with the tokenizer ratio contributing most of the language-level signal ($\Delta R^2 = 0.34$). Removing the EN/RU anchor isolates an anchor-free component at $R^2 = 0.49$, which exceeds the model-mean baseline and captures language coverage but not model capability.

Processing Inconsistency Predicts Language Competence: LLM Evaluation Without Answer Labels on Turkic Languages

Machine unlearning aims to remove the influence of specific training data from a model while preserving reliable behavior on the remaining data, making reliable prediction and uncertainty estimation essential for evaluation. Calibration is commonly used as a proxy for reliability in language models, but low calibration error does not necessarily imply reliable decision rules, as models may rely on spurious correlations while remaining well calibrated. We investigate this gap in generative language models using the multiple-choice question-answering evaluation protocol on the TOFU benchmark, measuring probabilistic reliability with calibration metrics (ECE, MCE, Brier) and decision-rule reliability via attribution-based shortcut detection with Integrated Gradients and Local Mutual Information. We find that fine-tuned models achieve low calibration error (ECE ≈ 0.04) compared to pretrained models (ECE > 0.5), and models after unlearning retain similarly low calibration despite reduced accuracy on the forget split, while attribution analysis shows increased reliance on correlation-based tokens. These results demonstrate that good calibration can coexist with shortcut-based decision rules after unlearning, extending the reliability paradox to the machine unlearning setting.

Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models

Automated ICD coding is a critical task for standardizing clinical information from electronic health records (EHRs) and supporting downstream healthcare administration.
However, existing automated ICD coding systems face several fundamental challenges. First, the majority of existing research focuses on English ICD tasks, with limited attention to Chinese-language clinical contexts due to the scarcity of publicly available Chinese ICD datasets. 
Second, most approaches primarily target disease coding, overlooking procedure coding as well as the multi-stage workflows followed in real-world clinical practice. 
Moreover, many recent methods rely heavily on closed-source large language models or substantial computational resources, which limits their scalability and deployability in clinical environments.
To address these gaps, this paper proposes \textit{JointCoder}, which includes a real-world Chinese ICD coding dataset and a multi-agent framework that reformulates automated ICD coding as a joint disease-procedure coding task. JointCoder explicitly models real-world clinical coding workflows through stage-wise agent collaboration.
All agents are instantiated using locally deployed 1.7B-parameter models, enabling scalable and privacy-preserving deployment.
Extensive experiments on real-world Chinese ICD coding datasets demonstrate JointCoder’s superiority over state-of-the-art baselines across all evaluation metrics. Our Github repository is available at https://github.com/KJOrigin/JointCoder. Our demo website is available at http://159.89.222.214:9028/. The demo video can be accessed at https://yunpan.qlu.edu.cn:443/link/CE86D849DD24EA9EE3D68AF2FD90D539.

JointCoder: Exploring Automated ICD Coding on Real-World Chinese EHRs with a Multi-Agent Framework

Mobile input method editors (IMEs) are the primary interface for text input, yet they remain constrained to manual typing and struggle to produce personalized text. While lightweight large language models (LLMs) make on-device auxiliary generation feasible, enabling deeply personalized, privacy-preserving, and real-time generative IMEs poses fundamental challenges. To this end, we present HUOZIIME, a personalized on-device IME powered by LLM. We endow HUOZIIME with initial human-like prediction ability by post-training a base LLM on synthesized personalization data. Notably, a hierarchical memory mechanism is designed to continually capture and leverage user-specific input history. Furthermore, we perform systemic optimizations tailored to on-device LLM-based IME deployment, ensuring efficient and responsive operation under mobile constraints. Experiments demonstrate efficient on-device execution and high-fidelity memory-driven personalization. Code and package are available at https://github.com/Shan-HIT/HuoziIME.

HuoziIME: An On-Device LLM-Enhanced Input Method for Deep Personalization

We present olmOCR, a fully open OCR system developed through iterative public releases and community feedback. The system combines a 7B vision-language model trained in two stages: supervised finetuning on 260K diverse PDF pages, followed by reinforcement learning with visual unit tests over synthetic documents. Visual unit tests are binary checks of structural fidelity, including tables and equations, and serve both as an interpretable evaluation framework and as direct optimization targets. We also introduce olmOCR-Bench, a benchmark of 1.4K challenging PDFs evaluated via visual unit tests, on which olmOCR achieves state-of-the-art performance among open systems and proprietary APIs at a fraction of the cost. We have deployed olmOCR at scale to 100M+ PDFs to curate pretraining data for Olmo 3. We share lessons from our open development process and release all models, data, and code across two major releases.

Downloads

Next from ACL 2026

Garden Path Recovery in Causal and Masked Language Models

Similar lecture

Parrot: A Training Pipeline Enhances Both Program CoT and Natural Language CoT for Reasoning