United States

Reliable automatic evaluation is essential for benchmarking Machine Translation (MT) systems. However, their reliability is often underexplored in morphologically complex, non-English-centric contexts. Current metrics frequently exhibit &quot;semantic blindness&quot; toward subtle morphological shifts in morphologically rich languages like Finnish. This study proposes a framework that utilizes a controlled perturbation suite to systematically isolate and quantify the metric sensitivity to morphosemantic errors in Swedish to Finnish translation. It combines authentic MT outputs with a tiered adversarial corpus, subjecting reference translations. It will quantify discriminative ability and ordinal monotonicity by leveraging a diverse evaluation suite.

ACL 2026

Thesis Proposal: Sensitivity of MT Evaluation Metrics to Morphosemantic Errors: A Case Study on Swedish–Finnish Translation

## Welcome to EMNLP 2022!
I am delighted to welcome you to EMNLP 2022! I believe this conference has been complicated beyond any precedent. Over the past year, it’s been thrilling to see the organization team approach each new puzzle with creativity and enthusiasm. We hope that those participating in Abu Dhabi as well as those joining remotely will leave the conference feeling newly inspired by the program and newly connected to our ever-growing community. Following EMNLP 2021 and major NLP conferences since, EMNLP 2022 is “hybrid,” serving both virtual and in-person participants.

Our key innovations for EMNLP 2022 include:

* EMNLP 2022 is “hybrid” in a second sense, as well: we allowed both direct and rolling review paper submissions, building on the pilot experiment of EMNLP 2021, which considered a small number of ARR submissions. 
* Familiar from NAACL but new to EMNLP, we’ve added an industry track.
* During the conference, “portals” will link virtual poster sessions to in-person conference participants during poster sessions each day.
* The first ACL-family conference in the United Arab Emirates.

 *Message from Noah A. Smith, University of Washington and Allen Institute for AI, Seattle, Washington, USA* 
***EMNLP 2022 General Chair***
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/9eec7d4a287ee18c278b08229290aa83.png)](https://drive.google.com/file/d/1OlPv6QBeo62VVTughj2jkiLeyHd1WnUt/view)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/a3db7a768409f05192210d98601edb25.png)](https://emnlp2022.rocket.chat/)

To access this site you need to register. Please register [here](https://2022.emnlp.org/registration/).

Register here

EMNLP 2022

Welcome!
EMNLP 2022 will take place in Abu Dhabi from December 7th to December 11th, 2022. And it will be held in hybrid mode, both online and offline.

Reliable automatic evaluation is essential for benchmarking Machine Translation (MT) systems. However, their reliability is often underexplored in morphologically complex, non-English-centric contexts. Current metrics frequently exhibit "semantic blindness" toward subtle morphological shifts in morphologically rich languages like Finnish. This study proposes a framework that utilizes a controlled perturbation suite to systematically isolate and quantify the metric sensitivity to morphosemantic errors in Swedish to Finnish translation. It combines authentic MT outputs with a tiered adversarial corpus, subjecting reference translations. It will quantify discriminative ability and ordinal monotonicity by leveraging a diverse evaluation suite.

technical paper

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/acl2026

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

We present **Copyright Detective**, the first interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs. The system treats copyright infringement versus compliance as an **evidence discovery** process rather than a static classification task due to the complex nature of copyright law. It integrates multiple detection paradigms, including content recall testing, paraphrase-level similarity analysis, persuasive jailbreak probing, and unlearning verification, within a unified and extensible framework. Through interactive prompting, response collection, and iterative workflows, our system enables systematic auditing of verbatim memorization and paraphrase-level leakage, supporting responsible deployment and transparent evaluation of LLM copyright risks even with black-box access. In our experiments with GPT-4o-mini, we demonstrate that specific persuasive strategy "Pathos" shift the leakage distribution from about 0.1 (ROUGE-L) to 0.7. Our live system is hosted on [Streamlit](https://copyright-detective.streamlit.app), and the source code will be made publicly available at [GitHub](https://github.com/changhu73/Copyright-Detective) upon acceptance. In addition, a [demo video](https://youtu.be/10gmwV3q_gg) is also included as supplementary material.

Copyright Detective : A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks

We demonstrate Hindsight, a working memory system for AI agents that organizes long-term memory into four logical networks and exposes three core operations. The world, experience, observation, and opinion networks separate objective facts from subjective beliefs, giving developers clear visibility into what an agent knows versus what it believes. The retain, recall, and reflect operations handle memory ingestion, retrieval, and reasoning respectively. The retrieval layer combines vector search, keyword matching, graph traversal, and temporal filtering in a parallel pipeline backed by PostgreSQL with pgvector. Unlike existing memory systems such as MemGPT, Zep, and Mem0, Hindsight is the only system that jointly provides fact-belief separation, temporal entity graphs, evolving opinions with confidence scores, and configurable behavioral profiles. On the LongMemEval and LoCoMo benchmarks, Hindsight with a 20B open-source model achieves 83.6% and 83.2% accuracy, outperforming full-context GPT-4o and all prior memory systems; with Gemini-3 Pro, accuracy reaches 91.4% on LongMemEval. Our interactive demonstration lets users build memory graphs through multi-session conversations, inspect how memories are classified across networks, and observe how agent opinions form and change over time. The system is $\textbf{open-source under the MIT license}$ and available as a Python package (pip install hindsight-all) and Docker image, with $\textbf{1.9K GitHub stars}$ and 191 forks till date.A video demo is at https://youtu.be/4M2wS-yEmVA.

Hindsight: Structured Agent Memory that Retains, Recalls, and Reflects

Code sandboxes have emerged as a critical infrastructure for advancing the coding capabilities of large language models, providing verifiable feedback for both RL training and evaluation. However, existing systems fail to provide accurate verification and efficiency under high-concurrency workloads. We present ScaleBox, a high-fidelity and scalable system designed to address these limitations in large-scale code training. ScaleBox introduces automated special-judge generation and management, fine-grained parallel execution across test cases with seamless multi-node coordination, and a configuration-driven evaluation suite for reproducible benchmarking. A series of experiments demonstrates that ScaleBox significantly enhances code verification accuracy and efficiency. Our further RLVR experiments show that ScaleBox substantially improves both performance on LiveCodeBench and training stability, significantly outperforming heuristic-matching baselines. By providing a reliable and high-throughput infrastructure, ScaleBox facilitates more effective research and development in large-scale code training.

ScaleBox: Enabling High-Fidelity and Scalable Code Verification for Large Language Models

Checklists have emerged as a popular approach for interpretable and fine-grained evaluation, particularly with LLM-as-a-Judge. 
Beyond evaluation, these structured criteria can serve as signals for model alignment, reinforcement learning, and self-correction. To support these endeavors, we present $\texttt{AutoChecklist}$, an open-source library that unifies checklist-based evaluation into composable pipelines. At its core is a taxonomy of five checklist generation abstractions, each encoding a distinct strategy for deriving evaluation criteria. A modular $\textit{Generator $\rightarrow$ Refiner $\rightarrow$ Scorer}$ pipeline connects any generator with a unified scorer, and new configurations can be registered via prompt templates alone. The library ships with eight built-in pipelines implementing published approaches and supports multiple LLM providers (OpenAI, OpenRouter, vLLM). Beyond the Python API, the library includes a CLI for off-the-shelf evaluation and a web interface for interactive exploration. Validation experiments confirm that these checklist methods significantly align with human preferences and quality ratings, and a case study on ICLR peer review rebuttals demonstrates flexible domain adaptation. $\texttt{AutoChecklist}$ is publicly available at https://github.com/ChicagoHAI/AutoChecklist.

AutoChecklist: Composable Pipelines for Checklist Generation and Scoring with LLM-as-a-Judge

Over the past years, Automated Essay Scoring (AES) systems have gained increasing attention as scalable and consistent solutions for assessing proficiency of student writing. Despite recent progress, support for Arabic AES remains limited due to linguistic complexity and scarcity of large publicly-available annotated datasets. In this work, we present Qayyem, a Web-based platform designed to support Arabic AES by providing an integrated workflow for assignment creation, batch essay upload, scoring configuration, and per-trait essay evaluation. Qayyem abstracts the technical complexity of interacting with scoring server APIs, allowing instructors to access advanced scoring services through a user-friendly interface. The platform deploys a number of state-of-the-art Arabic essay scoring models with different effectiveness and efficiency figures.

Qayyem: A Real-time Platform for Scoring Proficiency of Arabic Essays

The growing demand for Mental Health (MH) services highlights the need for scalable computational tools, yet progress in computational psychology is hindered by scarce sensitive data, complex assessment procedures, and high technical barriers. While language is a well-established marker of different MH conditions, existing NLP solutions are often fragmented, closed-source, or difficult to use, limiting their adoption in interdisciplinary research.
We present TONY, an open-source, python TOolkit for NLP in clinical psYchology. TONY bridges traditional psycholinguistic analysis and modern NLP by combining interpretable lexical features with state-of-the-art lightweight transformer models within a unified and easy-to-use framework. This hybrid approach enables robust and transparent text analysis without relying on large-scale models or closed-source software.
TONY is designed for researchers and practitioners working at the intersection of NLP and MH, facilitating collaboration across disciplines. Compared to the few existing systems, TONY offers a more comprehensive and exhaustive solution, reducing the barrier to entry through a unified, modular, and reproducible pipeline that integrates classical and neural approaches in a single open framework. The toolkit is released under an open-source license and is evaluated through multiple MH–related datasets, demonstrating its flexibility and effectiveness in low-resource settings

TONY: an open-source TOolkit for Nlp in psYchology

General-purpose VLMs remain unreliable for biomedical research because valid answers in scientific papers depend on evidence split across figures, tables, charts, captions, and referring text.
Existing post-training pipelines are bottlenecked by costly expert annotation and by synthetic data that drops this evidence structure.
We present Ryze, a fully automated system that converts raw biomedical papers into an \emph{evidence-enriched} training set and a domain-specialized VLM\@.
Ryze synthesizes QA pairs with complete supporting evidence (visual element, caption, extracted structure, and referring paragraphs), reduces layout and OCR errors via chart/table-aware extraction and LLM-based cleansing, and applies a two-stage post-training strategy combining supervised fine-tuning with reinforcement learning.
Starting from Qwen3-VL-8B, Ryze produces BioVLM-8B at under \$200, achieving 48.0\% weighted accuracy on LAB-Bench---outperforming the base model by +12.6\% and surpassing GPT-5.2 by +3.8\%.
We release Ryze as open source together with the trained BioVLM-8B model.

Ryze: Evidence-Enriched Data Synthesis from Biomedical Papers

Preparing graduate students for effective professional communication remains a central goal of higher education, yet consistently assessing the quality of presentation slide decks - particularly in fast-growing AI/ML programs - poses significant challenges.
We introduce SlideGuard, an evaluation agent that assesses slide decks against a comprehensive framework of expert-defined criteria using a visual language model.
The criteria, developed in collaboration with domain experts, span visual design, narrative coherence, and argumentative structure.
SlideGuard delivers explicit, interpretable justifications for its scoring decisions, and its content-hash-based caching enables efficient re-evaluation after incremental edits, reducing the time educators spend on slide deck evaluation and accelerating feedback delivery to students.
We evaluate the approach on a dataset of 150 annotated slide decks and show that it detects the majority of expert-identified issues, with stronger results on structural and visual criteria and known limitations on subjective dimensions such as research quality.
SlideGuard is released under the Apache~2.0 license and is available on GitHub,\footnote{\url{https://github.com/Industrial-AI-Research-Lab/SlideGuard}} including all criterion prompts, configuration files, and evaluation scripts to facilitate replication.

SlideGuard: AI-Driven Evaluation of Graduate Student Presentation Materials

Interactive articles help readers engage with complex ideas through exploration, yet creating them remains costly, requiring both domain expertise and web development skills. Recent LLM-based agents can automate content creation, but naively applying them yields uncontrollable and unverifiable outputs. We present Vividoc, a human-agent collaborative system that generates interactive educational documents from a single topic input. Vividoc introduces a multi-agent pipeline (Planner, Executor, Evaluator) and the Document Specification (DocSpec), a human-readable intermediate representation that decomposes each interactive visualization into State, Render, Transition, and Constraint components. The DocSpec enables educators to review and refine generation plans before code is produced, bridging the gap between pedagogical intent and executable output. We collect a dataset of 101 real-world interactive documents across 11 domains and conduct a user study showing that ViviDoc produces documents comparable in quality to human-authored ones. Our demo is available at \url{https://vividoc.vercel.app/} and a video demonstration at \url{https://www.youtube.com/watch?v=rJrnPJLyHUI}.

Demonstrating ViviDoc: Generating Interactive Documents through Human-Agent Collaboration

With the rise in capabilities of large language models (LLMs) and their deployment in real-world tasks, evaluating LLM alignment with human preferences has become an important challenge. Current benchmarks average preferences across all users to compute aggregate ratings, overlooking individual user preferences when establishing model rankings. Since users have varying preferences in different contexts, we call for personalized LLM benchmarks that rank models according to individual needs. We compute personalized model rankings using ELO ratings and Bradley-Terry coefficients for 115 active Chatbot Arena users and analyze how user query characteristics (topics and writing style) relate to LLM ranking variations. We demonstrate that individual rankings of LLM models diverge dramatically from aggregate LLM rankings, with Bradley-Terry correlations averaging only $\rho = 0.04$ (57\% of users show near-zero or negative correlation) and ELO ratings showing moderate correlation ($\rho = 0.43$). Through topic modeling and style analysis, we find users exhibit substantial heterogeneity in topical interests and communication styles, influencing their model preferences. We further show that a compact combination of topic and style features provides a useful feature space for predicting user-specific model rankings. Our results provide strong quantitative evidence that aggregate benchmarks fail to capture individual preferences for most users, and highlight the importance of developing personalized benchmarks that rank LLM models according to individual user preferences.

Next from ACL 2026

Copyright Detective : A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks

Similar lecture

Robust MT Evaluation with Sentence-level Multilingual Augmentation