United States

This paper presents our system for SemEval-2026 Task~11 Subtask~1 on content-independent syllogistic reasoning. The task evaluates whether language models can determine the formal validity of logical arguments independently of their semantic plausibility. To reduce content-driven biases, we propose a data augmentation strategy that progressively abstracts lexical semantics by replacing content words with symbolic placeholders and pseudo-words while preserving logical structure. Experiments based on fine-tuning microsoft/deberta-large-mnli show that abstraction-based augmentation reduces Content Effect and improves accuracy, leading to competitive performance on the official leaderboard. However, we observe substantial sensitivity to random initialization, suggesting that evaluation outcomes are partly influenced by stochastic factors. To better understand these effects, we conduct a layer-wise probing analysis using a Minimum Description Length framework, showing that the proposed approach decreases the accessibility of plausibility information in later transformer layers, indicating a shift toward more structure-oriented reasoning.

ACL 2026

Team ewelinaksiez at SemEval-2026 Task 11: Reducing Content Bias in Syllogistic Reasoning via Semantic Abstraction

semantic abstraction

syllogistic reasoning

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. <br>
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) <br><br><br>
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>
<br><br><br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/acl2026

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

workshop paper

This paper describes the system developed by the LATE-iimas team for Task 10 of SemEval-2026: Psycomark, specifically for Subtask 2, which involves conspiracy detection. Our approach was based on fine-tuning the popular pre-trained language model DeBERTa-v3-Large. To address the challenges inherent in the provided dataset, such as class imbalance and the linguistic ambiguity of the "Can't tell" label, we implemented a 5-Fold Stratified Cross-Validation technique combined with a Weighted Cross-Entropy Loss function. The final system, which operates using an ensemble of the resulting models, achieved a Weighted F1-Score of 0.75, placing it in the top 10 of the ranking.

LATE-iimas at SemEval-2026 Task 10: Conspiracy Detection via DeBERTa-v3 Ensemble and Weighted Loss Optimization

We describe our SemEval-2026 Task 8 Subtask A system, which focuses on evaluating and improving the retrieval aspect of multi-turn Retrieval-Augmented Generation (RAG) conversations. We implement a training-free fusion approach that combines three distinct query representations to retrieve documents independently. The results from these three views are pooled and reranked using a MonoT5 cross-encoder. Our findings demonstrate that this fusion approach consistently outperforms single-strategy baselines, revealing that optimal retrieval strategies vary significantly at the query level, and establishing multi-query fusion as a baseline for multi-turn RAG systems.

GUIR at SemEval-2026 Task 8: Training-Free Multi-Query Fusion for Robust Conversational Retrieval

The generation of source code via Artificial Intelligence has become a prevalent practice in both academia and industry, posing significant challenges to academic integrity and authorship attribution. In this work, we address SemEval-2026 Task 13: Detecting Machine-Generated Code by evaluating the effectiveness of four distinct methodologies: Graph Neural Networks (GNNs), Pre-trained Language Models (PLMs), Large Language Models (LLMs), and Stylometric Feature Engineering using XGBoost. Our approach focuses on three specific scenarios: Subtask A (Binary Detection), Subtask B (Multi-Class Authorship), and Subtask C (Hybrid Code Detection). While our models achieved high performance during the validation phase, the transition to the final test set revealed substantial challenges in generalization, likely due to the increased diversity of programming languages and generators in the unseen data. This work serves as a foundational first step, identifying critical gaps in model robustness and highlighting the need for more sophisticated methodologies to bridge the performance gap in complex, real-world environments.

LATE-IIMAS at Semeval-2026 Task 13: Evaluating GNNs, PLMs, LLMs, and Stylometry for Automatic Code Identification

This paper presents IIMAS-RAG, our system for SemEval-2026 Task 8 on evaluating multi-turn retrieval-augmented generation. Our approach combines LLM-based query rewriting, hybrid sparse-dense retrieval with SPLADE and Voyage-3-large fused via Reciprocal Rank Fusion, and answerability-conditioned generation with GPT-4.1. The system ranked 4th out of 38 teams in Subtask A (Retrieval) and 13th out of 29 teams in Subtask C (Full RAG). Our results show that query rewriting is the most impactful retrieval component, while generation remains challenging in low-context and partially answerable scenarios.

IIMAS-RAG at SemEval-2026 Task 8: Hybrid Sparse-Dense Retrieval and Answerability-Conditioned Generation for Multi-Turn RAG

We describe NLP-CEIA-UFG, our system for
SemEval-2026 Task 8, which evaluates multi-
turn retrieval-augmented generation (RAG)
over heterogeneous document corpora. Our
pipeline centers on a three-iteration dynamic
retrieval loop in which two gpt-oss-120b-
powered modules—an Iterative Query Genera-
tor and a Notes Builder—interact at each step
to diversify queries and accumulate structured
notes on information gaps. After the loop, an
Answerability Classifier routes the query to one
of three generation paths (Complete Answer,
Partial Answer, or Clarification Request). Hy-
brid BM25 and dense retrieval is fused via Re-
ciprocal Rank Fusion and refined by the Jina
listwise reranker. The retrieval pipeline is com-
piled under DSPy and optimized with GEPA.
We achieve nDCG@5 of 0.4502 (rank 17/38,
Subtask A) and HM = 0.3774 (rank 24/29, Sub-
task C). Post-hoc analysis identifies an over-
conservative Answerability Classifier as the
primary bottleneck: 75.5% of all responses
were flagged as IDK by the evaluator, includ-
ing 69.8% of ANSWERABLE questions, while
the retrieval and generation components per-
form well when the classifier routes correctly.
Our code is available at https://github.
com/GuiiCorreia/SemEval-2026.

NLP-CEIA-UFG at SemEval-2026 Task 8: Iterative Retrieval with Notes-Guided Query Refinement for Multi-Turn RAG

We describe the participation of team TFB in SemEval-2026 Task 4 on narrative similarity. We explore ColBERT-inspired sentence-level late interaction to capture event reordering, compare fine-tuning with synthetic data at multiple difficulty tiers, finding that distribution proximity to the target data matters more than volume and evaluate chain-of-thought prompting. We complement our approaches with a human annotation study (Krippendorff's alpha=0.32) confirming the task's inherent difficulty, an analysis of synthetic data distribution shift explaining why fine-tuning on out-of-distribution data hurts the model's performance. Despite our tests, we didn't surpass results of sentence-t5-xxl on Track B and Qwen2.5-7B on Track A. We finally decided to submit these two models for the task.

TFB at SemEval-2026 Task 4: Diagnosing Model Failures in Narrative Understanding

We investigate contextual embedding manipulation for Word Sense Disambiguation (WSD)
as part of SemEval-2026 Task 5. We propose four approaches built on BERT-like pretrained
models, experimenting with the informativeness of similarity calculations and classification
methods. We introduce scratch-trained cross-attention mechanisms inspired by GLiNER to compute similarity between definition or synonym representations and the full context. Our best performance achieved 57% accuracy with a Spearman correlation of 0.20. Our results suggest that finetuning strategy and trainng curriculum matter more than pretrained model choice for this novel task, and we identify several directions for future improvement. View our code base at: https://github.com/
heliosraz/SemEval52026

ZYC at SemEval-2026 Task 5: Application of BERT-based Contextual Embeddings Similarity for WSD

This report describes our participation in SemEval-2026 Task 8 on multi-turn retrieval and question answering. The task evaluates conversational systems across four domains (finance, cloud documentation, government, Wikipedia), and includes unanswerable queries where the available collection does not contain sufficient evidence to produce a complete response. We propose a multi-turn retrieval-augmented generation pipeline that combines learned sparse retrieval with LLM–based reranking and generation. Using sparse retrieval as the primary retrieval method, we leverage its strong generalization across domains. In addition, we make use of the long-context capabilities of LLMs for conversational query rewriting, pointwise and listwise reranking, and generating the final response, each conditioned on the full conversational history. This multi-step design enables effective integration of conversational context throughout retrieval and generation, improving robustness across domains.

uva-irlab-conv at SemEval-2026 Task 8: Multi-Turn RAG with Learned Sparse Retrieval and Listwise Reranking

We present NarSiL (Narrative Similarity Learners), our system for SemEval-2026 Task 4 Track A on Narrative Story Similarity. NarSiL employs a two-stage architecture: a Mixture-of-Experts (MoE) initial classifier that also leverages supermajority voting across three large language models (Gemma-3-12B, GPT-3.5-turbo-instruct, and Gemini-2.5-Flash) over multiple runs, followed by a structured three-pathway fallback for ambiguous cases. The three pathways correspond directly to the task's three core similarity components, abstract theme, narrative outcome, and course of action. Each path yields a similarity score corresponding to its respective component, and the scores are then combined through a weighted aggregation step. NarSiL achieves 64.25% accuracy on the official test set. An improved score of 70.25% is obtained by considering only the supermajority voting of GPT, followed by the previously described fallback.

NarSiL at SemEval-2026 Task 4: A Multi-Expert, Multi-Pathway System for Narrative Story Similarity

This paper presents a multi-paradigm approach to the first two subtasks of SemEval-2026 Task 11. For the first subtask, we explore two complementary strategies: a Llama-3 8B PEFT Majority Vote Ensemble, trained with bias-targeted augmented data, and a hybrid approach that separates LLM processing from logical reasoning, converting sentences into canonical logical forms for deterministic analysis. The hybrid approach is further extended to the second subtask. Official results placed us 17th in the first subtask and 15th in the second. Post-evaluation analysis indicates that our best model achieved perfect accuracy on the first subtask and revealed several errors in the ground truth data. After identifying certain implementation issues in the second subtask approach, the F1 retrieval score increased to over 98%, which would place us within the top 5 on the leaderboard.

Downloads

Next from ACL 2026

LATE-iimas at SemEval-2026 Task 10: Conspiracy Detection via DeBERTa-v3 Ensemble and Weighted Loss Optimization

Similar lecture

SemEval-2026 Task 11: Disentangling Content and Formal Reasoning in Large Language Models

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

LATE-iimas at SemEval-2026 Task 10: Conspiracy Detection via DeBERTa-v3 Ensemble and Weighted Loss Optimization

Similar lecture

SemEval-2026 Task 11: Disentangling Content and Formal Reasoning in Large Language Models

Downloads