United States

Document understanding in real-world applications often requires processing heterogeneous, multi-page document packets containing multiple documents stitched together. Despite recent advances in visual document understanding, the fundamental task of document packet splitting, which involves separating a document packet into individual units, remains largely unaddressed. We present the first comprehensive benchmark dataset, $\textit{DocSplit}$, along with novel evaluation metrics for assessing the document packet splitting capabilities of large language models. $\textit{DocSplit}$ comprises five datasets of varying complexity, covering diverse document types, layouts, and multimodal settings. We formalize the $\textit{DocSplit}$ task, which requires models to identify document boundaries, classify document types, and maintain correct page ordering within a document packet. The benchmark addresses real-world challenges, including out-of-order pages, interleaved documents, and documents lacking clear demarcations. We conduct extensive experiments evaluating multimodal LLMs on our datasets, revealing significant performance gaps in current models&#39; ability to handle complex document splitting tasks. The $\textit{DocSplit}$ benchmark datasets and proposed novel evaluation metrics provide a systematic framework for advancing document understanding capabilities essential for legal, financial, healthcare, and other document-intensive domains. We release the datasets and evaluation code to facilitate future research in document packet processing.

ACL 2026

DocSplit: A Comprehensive Benchmark Dataset and Evaluation Approach for Document Packet Recognition and Splitting

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

Document understanding in real-world applications often requires processing heterogeneous, multi-page document packets containing multiple documents stitched together. Despite recent advances in visual document understanding, the fundamental task of document packet splitting, which involves separating a document packet into individual units, remains largely unaddressed. We present the first comprehensive benchmark dataset, $\textit{DocSplit}$, along with novel evaluation metrics for assessing the document packet splitting capabilities of large language models. $\textit{DocSplit}$ comprises five datasets of varying complexity, covering diverse document types, layouts, and multimodal settings. We formalize the $\textit{DocSplit}$ task, which requires models to identify document boundaries, classify document types, and maintain correct page ordering within a document packet. The benchmark addresses real-world challenges, including out-of-order pages, interleaved documents, and documents lacking clear demarcations. We conduct extensive experiments evaluating multimodal LLMs on our datasets, revealing significant performance gaps in current models' ability to handle complex document splitting tasks. The $\textit{DocSplit}$ benchmark datasets and proposed novel evaluation metrics provide a systematic framework for advancing document understanding capabilities essential for legal, financial, healthcare, and other document-intensive domains. We release the datasets and evaluation code to facilitate future research in document packet processing.

poster

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

As the paradigm of AI shifts from text-based LLMs to Speech Language Models (SLMs), there is a growing demand for full-duplex systems capable of real-time, natural human-computer interaction.
However, the development of such models is constrained by the scarcity of high-quality, multi-speaker conversational data, as existing large-scale resources are predominantly single-speaker or limited in volume.
Addressing the complex dynamics of natural dialogue, such as overlapping and back-channeling remains a challenge, with standard processing pipelines suffering from diarization errors and ASR hallucinations.
To bridge this gap, we present a robust and scalable open-source data processing pipeline designed for full-duplex model.
Our code and project page are publicly available at https://anonymous-2001-j.github.io/sommelier.github.io/.

Sommelier: Scalable Open Multi-turn Audio Pre-processing for Full-duplex Speech Language Models

Consider an organization whose users send requests in natural language to an AI system that fulfills them by carrying out specific tasks. In this paper, we consider the problem of ensuring such user requests comply with a list of diverse policies determined by the organization with the purpose of guaranteeing the safe and reliable use of the AI system. We propose, to the best of our knowledge, the first benchmark consisting of annotated user requests of diverse compliance with respect to a list of policies. Our benchmark is related to industrial applications in the technology sector. We then use our benchmark to evaluate the performance of various LLM models on policy compliance assessment under different solution methods. We analyze the differences on performance metrics across the models and solution methods, showcasing the challenging nature of our problem.

Policy Compliance of User Requests in Natural Language for AI Systems

Rigorous content moderation is crucial for online advertising but leads to millions of daily rejections. This scale renders manual rectification infeasible, particularly for video advertisements.
However, existing safety-driven methods often suffer from aggressive over-editing, which compromises the advertiser's original semantic intent merely to satisfy compliance.
In this work, we target the rectification of textual violations in video ads, covering both speech transcripts and on-screen text. We propose $\mathcal{R}^3$, a novel framework designed to harmonize compliance with original semantic intent preservation.
Our approach integrates three key innovations: (1) an experience-driven data synthesis framework that bootstraps high-quality supervision via group-**R**elative compliance experience extractor; 
(2) a curriculum **R**einforcement learning strategy with hierarchical rewards designed to enforce compliance while maximizing semantic consistency;
and 
(3) a comprehensive video **R**ectification framework seamlessly integrating text recognition, rewriting, and re-rendering for industrial deployment. Extensive experiments on industrial datasets and online A/B testing demonstrate that $\mathcal{R}^3$ significantly outperforms state-of-the-art baselines, achieving an optimal trade-off between violation rectification and intent preservation.

$\mathcal{R}^3$: Advertisement Compliance $\mathcal{R}$ectification via Group-$\mathcal{R}$elative Experience Extractor and Curriculum $\mathcal{R}$einforcement

The rapid growth of online grocery shopping demands recommendation models that capture cyclical purchasing behavior and diverse user queries. At a large-scale online grocery e-commerce platform, traditional item-level models face scalability and accuracy limitations, motivating category-level recommendation as a more structured and business-aligned solution. We propose GrocLM, a fine-tuned language model for grocery category recommendation in a real-world production setting. Built with a two-stage LoRA-based fine-tuning strategy, GrocLM models repurchasing patterns and implicit category relationships from historical purchase and query data. To ensure discrete and valid outputs, we introduce a trie tree-based constrained decoding mechanism. Extensive experiments on proprietary production data and public datasets demonstrate that GrocLM significantly outperforms strong baselines and surpasses the deployed production model in live testing, improving recommendation precision, category diversity, and scalability.

GrocLM: Grocery Category Recommendation in E-Commerce with Large Language Models

Deep Research (DR) systems autonomously retrieve and synthesize information from web sources, however, industrial DR applications face a critical gap: effective integration of internal tools with web search. We introduce DeepResearch Retail, an evaluation framework grounded in real-world e-commerce data for assessing Deep Research with tools (DR+Tools) in realistic commercial settings. The framework evaluates both factual faithfulness and multidimensional response quality when reasoning over heterogeneous web and internal data sources.
We further present Hybrid-ReAct, a multi-agent architecture that demonstrates how collaborative reasoning and tool use can produce evidence-grounded answers. Experimental results validate our framework's utility, showing improvements in agent's performance when leveraging web-page information and multi-agent specialization.

DeepResearch Retail: Benchmarking Tool-Augmented Deep Research in the E-Commerce Domain

While Large Language Models excel at reasoning and language understanding, they struggle with multi-step operational workflows requiring precise procedural adherence, which is fundamental for industrial automation. Existing SOP-guided agents assume well-defined procedures and structured APIs, failing to address enterprise realities like incomplete SOPs, dynamic web interfaces, and unpredictable document formats. We present Agent-Ops, an end-to-end multi-agent framework automating Standard Operating Procedures in e-commerce. Agent-Ops contributes: (1) SOP Groomer, a human-AI framework transforming ambiguous documentation into automation-ready specifications, improving accuracy by 13.2%, (2) WebAgent, achieving 91.3% task completion and 86.5% execution consistency through demonstration-based learning, and (3) a Document Verification Agent performing multi-lingual validation across tax invoices, certificates, and supply chain documents with 94.2% accuracy. Deployed across seven SOP categories in three geographic regions, Agent-Ops achieves 85-97% end-to-end accuracy while reducing case resolution from 30 to 5 minutes (83% reduction). Production deployment with over 1000 Account Managers validates that LLM-based agents achieve enterprise-grade reliability when augmented with robust web automation, comprehensive document understanding, and systematic SOP refinement.

Agent-Ops: A Multi-Agent Orchestration Framework for End-to-End SOP Automation in E-Commerce Operations

While coronary imaging is widely used for anatomical assessment, CCTA reports play a distinct last-mile role in clinical care. Rather
than serving as an intermediate signal, CCTA provides an assessment of coronary disease severity (known as the CAD-RADS score) to
guide patient management. However, real-world clinical text exhibits substantial heterogeneity in terminology and structure, leading
to inconsistent interpretation by automated systems, even for clinically similar cases. Recent work leverages a direct application of
LLMs for automated CAD-RADS scoring, but is limited by small, non-public, and homogeneous clinical data. We introduce CCTA-RADS, the largest publicly available dataset of 940 real-world CCTA reports from a major cardiovascular center, each annotated with CAD-RADS scores. Our analysis reveals that direct approaches, including state-of-the-art LLMs (GPT-4o, GPT-o3) and fine-tuned BERT models underperform on diverse real-world clinical data. To address these limitations, we propose a two-stage pipeline that decouples structuring from classification: an LLM-based parser normalizes heterogeneous reports into structured format, followed by fine-tuned BERT classification. This approach substantially improves the F1-score by 6%-13% compared with direct methods. We deploy our system as an interactive web interface that allows clinicians to upload CCTA reports for automated CAD-RADS assessment with SHAP and LIME explainability visualizations.

Learning from Textual Radiology Reports: A Benchmark Dataset for Coronary CT Angiography

Retrieval-Augmented Generation (RAG) sys-
tems degrade sharply under extreme noise,
where relevant evidence is sparse and easily
pruned by static retrieval decisions. Existing
approaches fixed top-k retrieval, hierarchical
chunking, cross-encoder reranking, or policy-
based iterative control- either rely on rigid
heuristics or incur substantial computational
overhead, and often fail to recover context-
dependent evidence without introducing redun-
dancy or latency.
We introduce NEST (Nested Evidence Survival
for Retrieval), a lightweight, training-free RAG
framework that improves factual grounding by
explicitly separating recall amplification from
precision selection. NEST first maximizes re-
call through Nested Evidence Survival, eval-
uating candidates under nested retrieval con-
texts to rescue evidence that would otherwise
be pruned by static chunking. It then applies
a survival-consistent Mean Reciprocal Rank
(MRR) selection mechanism to retain evidence
that remains salient across retrieval scopes, re-
moving redundancy without harming recall.
Evaluated on WebQuestions, HotpotQA (dis-
tractor setting), and a proprietary InternalQA
benchmark with 50M Common Crawl distrac-
tors, NEST consistently outperforms strong
adaptive RAG baselines, including DeepRAG,
improving EM by up to +2.4 pp and F1 by +2.1
pp, while increasing retrieval recall by +6.8
pp. These gains are achieved with only 12–18
ms additional latency. Ablation studies confirm
that Nested Evidence Survival drives recall im-
provements, while MRR-based selection con-
verts these gains into precision, demonstrating
that recall-first retrieval with principled prun-
ing can outperform iterative control and model
scaling in retrieval-augmented generation.

NEST: Nested Evidence Survival for Retrieval

Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by incorporating external knowledge, yet traditional single-round retrieval struggles with complex multi-step reasoning.
Agentic RAG addresses this by enabling LLMs to dynamically decide when and what to retrieve, but current RL-based training methods suffer from sparse outcome rewards that discard intermediate signals and low sample efficiency where failed samples contribute nothing.
We propose Search-P1, a framework that introduces path-centric reward shaping for agentic RAG training, comprising two key components: (1) Path-Centric Reward, which evaluates the structural quality of reasoning trajectories through order-agnostic step coverage and soft scoring that extracts learning signals even from failed samples, and (2) Dual-Track Path Scoring with offline-generated reference planners that assesses paths from both self-consistency and reference-alignment perspectives.
Experiments on multiple QA benchmarks demonstrate that Search-P1 achieves significant improvements over Search-R1 and other strong baselines, with an average accuracy gain of 7.7 points.

Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training

Modern e-commerce assistants must go beyond simple product search to support inspiration, comparison, and tool-grounded fact-checking across non-linear shopping journeys. However, distilling these complex behaviors into efficient, deployable models is bottle-necked by a lack of post-training data: trajectories must cover diverse agentic workflows with high fidelity, yet the desired outputs are open-ended without a single ground truth. We propose a closed-loop Multi-Agent Simulation Framework to synthesize diverse, faithful, and policy-aligned shopping trajectories. The system orchestrates a journey-aware, stateful user simulator to drive exploration, a shopping agent that manages both tools and UI elements, and a critic agent that provides rubric-driven feedback to iteratively refine the data. On a domain-specific benchmark, this synthetic data enables a small model to significantly outperform same-size baselines and surpass a large-model baseline, achieving near-zero tool-calling errors with 8$\times$ higher inference throughput.

Downloads

Next from ACL 2026

Sommelier: Scalable Open Multi-turn Audio Pre-processing for Full-duplex Speech Language Models

Similar lecture

A Multi-Modal Multilingual Benchmark for Document Image Classification