Morocco

We investigate to what extent Multi-modal Large Language Models (MLLMs) can accurately identify the orientation of input images rotated 0В°, 90В°, 180В°, and 270В°. This task demands robust visual reasoning capabilities to detect rotational cues and contextualize spatial relationships within images regardless of their orientation. To evaluate MLLMs on these abilities, we introduce RotBench -- a 350-image manually-filtered benchmark comprised of lifestyle, portrait, and landscape images. Despite the relatively simple nature of this task, we show several state-of-the-art open and proprietary MLLMs do not reliably identify rotation in input images. Providing models with additional information -- including captions, segmentation maps, and more -- or using chain-of-thought prompting offers only small and inconsistent improvements. Our results indicate most models are able to reliably identify right-side up (0В°) images, while certain models are able to identify up-side down (180В°) images. None can reliably distinguish between 90В° and 270В°. Simultaneously showing the image rotated in different orientations leads to moderate performance gains in reasoning models, while a modified setup using voting improves performance in weaker models. We further show fine-tuning does not improve models&#39; ability to distinguish 90В° and 270В°, despite substantially improving identification of 180В° images. Together, these results reveal a significant gap between MLLMs&#39; spatial reasoning capabilities and human perception in identifying rotation.

EACL 2026

RotBench: Evaluating Multi-modal Large Language Models on Identifying Image Rotation

rotation understanding

vision question answering

spatial reasoning

multimodality

## Welcome to EMNLP 2024! 
We are excited to welcome you to one of the most prominent conferences in the field of Natural Language Processing. This year, EMNLP 2024 is being held in a hybrid format,
offering both virtual and in-person participation in beautiful Miami. Due to a record-breaking number of submissions, we've expanded the total number of accepted papers to accommodate more cutting-edge research from around the globe.
### [Conference Handbook](https://drive.google.com/file/d/1WPROgxjLAC96AJL7Ugy0tEnYm7dkrbHt/view?usp=sharing)

You are required to register for this event. **Please register [here](https://2024.emnlp.org/registration/).** The EMNLP 2024 event page on Underline will be open to public one week prior to the event.

Please register!

EMNLP 2024

EMNLP 2024 will take place in Miami, Florida from Nov 12th to Nov 16th, 2024, at the Hyatt Regency Miami Hote and on Underline for remote participants.

We investigate to what extent Multi-modal Large Language Models (MLLMs) can accurately identify the orientation of input images rotated 0В°, 90В°, 180В°, and 270В°. This task demands robust visual reasoning capabilities to detect rotational cues and contextualize spatial relationships within images regardless of their orientation. To evaluate MLLMs on these abilities, we introduce RotBench -- a 350-image manually-filtered benchmark comprised of lifestyle, portrait, and landscape images. Despite the relatively simple nature of this task, we show several state-of-the-art open and proprietary MLLMs do not reliably identify rotation in input images. Providing models with additional information -- including captions, segmentation maps, and more -- or using chain-of-thought prompting offers only small and inconsistent improvements. Our results indicate most models are able to reliably identify right-side up (0В°) images, while certain models are able to identify up-side down (180В°) images. None can reliably distinguish between 90В° and 270В°. Simultaneously showing the image rotated in different orientations leads to moderate performance gains in reasoning models, while a modified setup using voting improves performance in weaker models. We further show fine-tuning does not improve models' ability to distinguish 90В° and 270В°, despite substantially improving identification of 180В° images. Together, these results reveal a significant gap between MLLMs' spatial reasoning capabilities and human perception in identifying rotation.

technical paper

#### *Message from the General Chair, Aline Villavicencio*
I’m delighted and honoured to welcome you to the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026), taking place in the beautiful city of Rabat, in Morocco, in March 24-29, 2026. EACL is the flagship European conference of the Association and EACL 2026 proudly continues our field’s tradition of excellence in scholarship, innovation, and inclusivity. I am deeply grateful to the many volunteers whose dedication, generosity, and tireless efforts have made this conference possible.
For the first time EACL is being hosted in the African continent. This is an important milestone for our community, and we are grateful to our Moroccan hosts for enabling this historic moment by bringing this edition of EACL to Rabat. We are also delighted that the Second Arabic NLP School is co-located with EACL. We hope attendees enjoy this wonderful opportunity to strengthen ties with the Computational Linguistics communities across the African continent. *[Read full message](https://drive.google.com/file/d/14NlmHvwM6fPJuMmOvVh7K0vtQbEyv3SZ/view?usp=sharing)* 

<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://underline.io/events/522/reception','_blank')">Go to Workshops and Tutorials Program</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to EACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.eacl.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/eacl2026

Registration Required

Welcome to the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL). EACL 2026 will be held in Rabat, Morocco, from March 24–29, 2026. 

As Large Language Models (LLMs) continue to evolve, ensuring their safety across multiple languages has become a critical concern. While LLMs demonstrate impressive capabilities in English, their safety mechanisms may not generalize effectively to other languages, leading to disparities in toxicity detection, bias mitigation, and harm prevention. This systematic review examines the multilingual safety of LLMs by synthesizing findings from recent studies that evaluate their robustness across diverse linguistic and cultural contexts beyond English language. Our review explores the methodologies used to assess multilingual safety, identifies challenges such as dataset availability and evaluation biases. Based on our analysis we highlight gaps in multilingual safety research and provide recommendations for future work. This review aims to contribute to the development of fair and effective safety mechanisms for LLMs across all languages. We provide the extracted data in an interactive Streamlit dashboard, enabling transparent access to the raw data and allowing for continuous updates.

Safety of Large Language Models Beyond English: A Systematic Literature Review of Risks, Biases, and Safeguards

With the advent of large language models (LLMs), it has become common practice for users to draft text and utilize LLMs to enhance its quality through paraphrasing. However, this process can sometimes result in the loss or distortion of the original intended meaning. Due to the human-like quality of LLM-generated text, traditional detection methods often fail, particularly when text is paraphrased to closely mimic original content. In response to these challenges, we propose a novel approach named SearchLLM, designed to identify LLM-paraphrased text by leveraging search engine capabilities to locate potential original text sources. By analyzing similarities between the input and regenerated versions of candidate sources, SearchLLM effectively distinguishes LLM-paraphrased content. SearchLLM is designed as a proxy layer, allowing seamless integration with existing detectors to enhance their performance. Experimental results across various LLMs demonstrate that SearchLLM consistently enhances the accuracy of recent detectors in detecting LLM-paraphrased text that closely mimics original content. Furthermore, SearchLLM also helps the detectors prevent paraphrasing attacks.

SearchLLM: Detecting LLM Paraphrased Text by Measuring the Similarity with Regeneration of the Candidate Source via Search Engine

We introduce the Korean Canonical Legal Benchmark (KCL), a benchmark designed to assess language models’ legal reasoning capabilities independently of domain-specific knowledge. KCL provides question-level supporting precedents, enabling a more faithful disentanglement of reasoning ability from parameterized knowledge. KCL consists of two components: (1) KCL-MCQA, multiple-choice problems of 283 questions with 1,103 aligned precedents, and (2) KCL-Essay, open-ended generation problems of 169 questions with 550 aligned precedents and 2,739 instance-level rubrics for automated evaluation. Our systematic evaluation of 30+ models shows large remaining gaps, particularly in KCL-Essay, and that reasoning-specialized models consistently outperform their general-purpose counterparts. We release all resources, including the benchmark dataset and evaluation code, at https: //github.com/lbox-kr/kcl

Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs' Legal Reasoning Capabilities

Multiple choice evaluation is widely used for benchmarking large language models, yet near ceiling accuracy in low option settings can be sustained by shortcut strategies that obscure true competence. Therefore, we propose a massive option evaluation protocol that scales the candidate set to one hundred options and sharply reduces the impact of chance performance. We apply this framework to a Korean orthography error detection task where models must pick the single incorrect sentence from a large candidate set. With fixed targets and repeated resampling and shuffling, we obtain stable estimates while separating content driven failures from positional artifacts. Across experiments, results indicate that strong performance in low option settings can overstate model competence. This apparent advantage often weakens under dense interference at high N, revealing gaps that conventional benchmarks tend to obscure. We identify two failure modes, semantic confusion and position bias toward early options under uncertainty. To isolate the effect of context length, we run padding controlled and length matched tests, which suggest that the main bottleneck is candidate ranking rather than context length. Together, these findings support massive option evaluation as a general framework for stress testing model reliability under extreme distractor density, beyond what low option benchmarks can reveal.

Pushing the Boundaries of Multiple Choice Evaluation to One Hundred Options

Retrieval-Augmented Generation (RAG) systems face efficiency bottlenecks in prefill due to attention mechanism, and traditional KV cache only accelerates decoding. In this context, reusing document-level KV cache computed for retrieved documents in previous sessions during the prefill stage appears to be a natural way to amortize computation, but it raises serious correctness challenges due to position and context misalignment across queries and sessions. This research proposes a multi-document KV cache reuse framework for multi-document RAG workloads across queries and sessions to resolve position misalignment and context misalignment, preserving accuracy while eliminating document-specific quadratic complexity in prefill. Theoretical analysis will establish conditions under which multi-document KV cache reuse remains stable and close to full recomputation, providing principled guarantees for both efficiency and accuracy. These results will enable deployment in existing RAG pipelines without architectural changes or model retraining. Crucially, to ensure robustness in real-world deployments, validation will extend beyond standard benchmarks to include noise-robustness tests and domain-specific workloads (e.g., legal). The research aims to empirically confirm these guarantees and demonstrate that substantial prefill speedups can be achieved without materially degrading task-level performance.

Thesis Proposal: Efficient KV Cache Reuse for Multi-Document Retrieval-Augmented Generation

This paper introduces Kahaani, a multimodal, co-creative storytelling system that leverages Generative Artificial Intelligence, designed for children to address the challenge of sustaining engagement to foster educational narrative experiences. Here we define co-creative as a collaborative creative process in which both the child and Kahaani contribute to the generation of the story. The system combines Large Language Model (LLM), Text-to-Speech (TTS), Text-to-Music (TTM), and Text-to-Video (TTV) generation to produce a rich, immersive, and accessible storytelling experience. The system grounds the co-creation process in two classical storytelling framework, Freytag’s Pyramid and Propp’s Narrative Functions. The main goals of Kahaani are: (1) to help children improve their English skills, (2) to teach important life lessons through story morals, and (3) to help them understand how stories are structured, all in a fun and engaging way. We present evaluations for each AI component used, along with a user study involving three parent–child pairs to assess the overall experience and educational value of the system.

Kahaani: A Multimodal Co-Creative Storytelling System

This study examines how credibility, trust, and bias interact within complex communication systems that shape public understanding of scientific information. It addresses two questions: 1. What are the primary factors that influence the public’s comprehension of scientific findings? 2. How do the factors influencing public understanding of climate change science interact within a complex system? A scoping literature review synthesized disparate communication models from media studies, science communication, psychology, and information science to identify a shared set of system variables. The identified variables were organized into source-, message-, channel-, and receiver-related factors and used to develop a causal loop diagram showing how credibility, trust, and information processing co-evolve through reinforcing and balancing feedback. The resulting diagram illustrates two major loops: one centered on trust in information sources, which can foster social cohesion or accelerate truth decay, and another linking individual trust dynamics to broader patterns of polarization and unity. By clarifying how well-established constructs interact to produce dynamic communication outcomes, the framework is useful for scholars developing integrative theory and for policymakers and practitioners designing interventions in misinformation-prone environments. The CLD also provides a foundation for future system dynamics modeling to examine how interventions in transparency, media literacy, or platform governance may influence public trust over time.

Communication as a Complex System: Modeling the Feedback Dynamics of Trust and Credibility

We study object hallucination in Multimodal Large Language Models (MLLMs) and improve visual contrastive decoding (VCD) by constructing an object-aligned auxiliary view. We leverage object-centric attention in self-supervised Vision Transformers. In particular, we remove the most salient visual evidence to construct an auxiliary view that disrupts unsupported tokens and produces a stronger contrast signal. Our method is prompt-agnostic, model-agnostic, and can be seamlessly plugged into the existing VCD pipeline with little computation overhead, i.e., a single cacheable forward pass. Empirically, our method demonstrates consistent gains on two popular object hallucination benchmarks across two MLLMs.

Mask What Matters: Mitigating Object Hallucinations in Multimodal Large Language Models with Object-Aligned Visual Contrastive Decoding

Lightweight Large Language Models (LLMs) have demonstrated notable safety alignment deficits, particularly outside of English. These challenges are especially acute for Traditional Chinese (TC), stemming from distinct linguistic characteristics and a scarcity of dedicated safety resources. To address this, we introduce the Prompt Assortment for Traditional Chinese Hazards (PATCH) dataset, the first large-scale adversarial dataset tailored for TC safety evaluation, aligned with standard threat taxonomies. Using PATCH, we evaluated Llama Guard, RoBERTa, and Longformer architectures with full fine-tuning, Low-Rank Adaptation (LoRA), and Chat-Vector methods. Our findings demonstrate that parameter-efficient LoRA achieves classification performance (F1 > 0.99) comparable to full fine-tuning, providing an effective and efficient method for developing TC safety classifiers. We also find initial evidence suggesting targeted LoRA tuning may offer cross-lingual safety benefits.

PATCH Dataset: Empowering Traditional Chinese Safety Classifiers for Lightweight LLM

Fine-tuning large language models can introduce misaligned behaviors, but when and how distinct misaligned behaviors emerge during training remains poorly understood. We investigate the temporal dynamics of misalignment by analyzing model behavior at every checkpoint throughout fine-tuning. Our results reveal that hallucination serves as a primary, early-onset failure mode across domains. We fine-tuned Qwen2.5-7B-Instruct on medical and math datasets with varying degrees of misalignment and evaluated hallucination and malicious behavior at each checkpoint. In the medical domain, we observe a distinct temporal lag: hallucination scores spike early in training, preceding the emergence of malicious traits by approximately 18-22 checkpoints. In contrast, the math domain exhibits rising hallucinations without the corresponding emergence of malicious behavior, suggesting that while hallucination is a pervasive early signal, the solidification of "evil" traits is highly domain-dependent. Critically, because hallucination provides an earlier and more reliable signal than malicious traits in medical fine-tuning, monitoring it could enable targeted interventions in domains where harmful behaviors subsequently emerge. Our work demonstrates that misalignment evolves gradually with detectable precursor signals, and that temporal relationships between misalignment types vary significantly by domain.

Downloads

Next from EACL 2026

Safety of Large Language Models Beyond English: A Systematic Literature Review of Risks, Biases, and Safeguards

Similar lecture

The Instinctive Bias: Spurious Images lead to Hallucination in MLLMs