Austria

Question Answering over large tables is challenging due to the difficulty of reasoning required in linking information from different parts of a table, such as heading and metadata to the values in the table. We investigate using Large Language Models (LLM) for tabular reasoning, where, given a pair of a table and a question from the DataBench benchmark, the models generate answers. We experiment with three techniques that enable symbolic reasoning through code execution: (1) a direct code prompting (DCP) approach that uses Python; (2) Multi-Step Code (MSC) prompting using SQL and ReAct prompting; and (3) a combination of multi-step reasoning (MSR), few-shot (FS) learning, and Python tools. We also conduct an analysis exploring the impact of answer types, data size, and multi-column dependencies on LLMs&#39; answer generation performance, including an assessment of the models&#39; limitations and the underlying challenges of tabular reasoning in LLMs.


ACL 2025

CSIRO LT at SemEval-2025 Task 8: Answering Questions over Tabular Data using LLMs

### Welcome!
The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. Our Virtual Poster Sessions will take place online Thursday, August 22, 2024.

You are required to register for this event. **Please register [here](https://2024.aclweb.org/registration). **

If you have already registered, please check your inbox for an email from Underline granting you access to ACL 2024 content.

Please register!

ACL 2024

The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. More information will be announced soon.

Question Answering over large tables is challenging due to the difficulty of reasoning required in linking information from different parts of a table, such as heading and metadata to the values in the table. We investigate using Large Language Models (LLM) for tabular reasoning, where, given a pair of a table and a question from the DataBench benchmark, the models generate answers. We experiment with three techniques that enable symbolic reasoning through code execution: (1) a direct code prompting (DCP) approach that uses Python; (2) Multi-Step Code (MSC) prompting using SQL and ReAct prompting; and (3) a combination of multi-step reasoning (MSR), few-shot (FS) learning, and Python tools. We also conduct an analysis exploring the impact of answer types, data size, and multi-column dependencies on LLMs' answer generation performance, including an assessment of the models' limitations and the underlying challenges of tabular reasoning in LLMs.


workshop paper

### Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

Message from the General Chair: 
*It is my great pleasure and honor to welcome you to the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025), held in beautiful Vienna, Austria, from July 27 to August 1, 2025. ACL2025continues our field’s tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.* 
[Read more](https://drive.google.com/file/d/1GI_hvOpjswAuYdUTromfeDiPpCcqidwg/view?usp=sharing)

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2025". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you need to log in with the **email address you registered with**. 

Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

We present our approach to SemEval-2025 Task 6 (PromiseEval), which focuses on verifying promises in corporate ESG reports across four subtasks: identification, evidence assessment, clarity evaluation, and timing \cite{chen2025semeval}. We explore three models: (1) ESG-BERT \cite{mukherjee2021esgbert} with task-specific heads, (2) an enhanced version with subtask-specific linguistic features, and (3) a unified model combining attention-based pooling, document metadata, and multi-objective learning. On the English ML-Promise dataset, our unified model achieves a score of 0.5268, outperforming the baseline (0.5227), demonstrating the benefits of linguistic features, attention mechanisms, and multi-objective training despite challenges like class imbalance and limited data.

CLaC at SemEval-2025 Task 6: A Multi-Architecture Approach for Corporate Environmental Promise Verification

This paper presents our participation in the SemEval-2025 task 6: multinational, multilingual, multi-industry promise verification. The SemEval-2025 Task 6 aims to extract Promise Identification, Supporting Evidence, Clarity of the Promise-Evidence Pair, and Timing for Verification from the promises made to businesses and governments. Using these data to verify whether companies and governments have fulfilled their promises. In this task, we focus on the English dataset. Our model introduces regularization dropout based on the BERT-base model to focus on the stability of non-targe classes, improve the robustness of the model, and ultimately improve the indicators. Our approach obtained competitive results in task.

YNU-HPCC at SemEval-2025 Task 6: Using BERT Model with R-drop for Promise Verification

We investigated various machine learning approaches for the Promise Verification task, comparing zero-shot and six-shot GPT-4o, SVM with Multilingual E5 embeddings, and fine-tuned DistilBERT. While six-shot GPT-4o achieved the best performance, it was resource-intensive. SVM proved to be an economical and effective alternative, especially with data augmentation. DistilBERT offered a scalable solution balancing efficiency and accuracy. Our findings highlight the trade-offs between performance, cost, and efficiency in model selection for this task.

CSCU at SemEval-2025 Task 6: Enhancing Promise Verification with Paraphrase and Synthesis Augmentation: Effects on Model Performance

Hallucinations pose a significant challenge for large language models when answering knowledge-intensive queries. As LLMs become more widely adopted, it is crucial not only to detect if hallucinations occur but also to pinpoint exactly where in the LLM output they occur. SemEval 2025 Task 3, Mu-SHROOM: Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes, is a recent effort in this direction. This paper describes the UCSC system submission to the shared Mu-SHROOM task. We introduce a framework that first retrieves relevant context, next identifies false content from the answer, and finally maps them back to spans in the LLM output. The process is further enhanced by automatically optimizing prompts. Our system achieves the highest overall performance, ranking #1 in average position across all languages. We release our code and experiment results.

UCSC at SemEval-2025 Task 3: Context, Models and Prompt Optimization for Automated Hallucination Detection in LLM Output

A two-stage role classification model based on DeBERTa is proposed for the Entity Framework task in SemEval 2025 Task 10. The task is confronted with challenges such as multi-labeling, multi-category, and category imbalance, particularly in the semantic overlap and data sparsity of fine-grained roles. Existing methods primarily rely on rules, traditional machine learning, or deep learning, but the accurate classification of fine-grained roles is difficult to achieve. To address this, the proposed model integrates the deep semantic representation of the DeBERTa pre-trained language model through two sub-models: main role classification and sub-role classification, and utilizes Focal Loss to optimize the category imbalance issue. Experimental results indicate that the model achieves an accuracy of 75.32\% in predicting the main role, while the exact matching rate for the sub-role is 8.94\%. This is mainly limited by the strict matching standard and semantic overlap of fine-grained roles in the multi-label task. Compared to the baseline's sub-role exact matching rate of 3.83\%, the proposed model significantly improves this metric. The model ultimately ranked 23rd on the leaderboard. The code of this paper is available at: https://github.com/jiyuaner/YNU-HPCC-at-SemEval-2025-Task10.

YNU-HPCC at SemEval-2025 Task 10: A Two-Stage Approach to Solving Multi-Label and Multi-Class Role Classification Based on DeBERTa

This paper describes our system submitted to SemEval-2025 Task 4, which introduces the Synthetic Token Alternative Training (STAT) algorithm for efficient unlearning in large language models (LLMs). The proposed method aims to enable pretrained models to selectively forget designated data (the forget set) while preserving performance on the remaining data (the retain set).
The STAT framework adopts a dual-stage process. In the first stage, pseudo tokens are generated through random sampling and applied to the forget set, facilitating more effective targeted unlearning. In the second stage, the model undergoes gradient-based optimization using an alternative training scheme that alternates between pseudo-token-augmented samples from the forget set and unmodified samples from the retain set. This design promotes stable unlearning of the specified data while accelerating convergence and preserving the model’s general performance.
Our system achieved 3rd place in the 7B model track (OLMo-7B) and 7th place in the 1B model track (OLMo-1B), demonstrating substantial improvements over the official baselines, exhibiting superior stability in knowledge retention and more effective targeted forgetting compared to existing approaches.

YNU at SemEval-2025 Task 4: Synthetic Token Alternative Training for LLM Unlearning

This study reports the YNU-HPCC team’s par ticipation in Subtask A of SemEval-2025 Task 1 on multimodal idiomatic representation. The task requires ranking candidate images based on their semantic relevance to a target idiom within a given sentence, challenging models to disambiguate idiomatic semantics, and align ing them with abstract visual concepts across English and Portuguese. Using AltCLIP-m18 as the base model, our approach enhances its zero-shot capabilities with LoRA fine-tuning and combines ListMLE ranking optimization with Focal Loss to handle hard samples. Ex perimental results on the primary test set show significant improvements over the base model, with Top-1 Accuracy/DCG scores of 0.53/2.94 for English and 0.77/3.31 for Portuguese. The code is publicly available at https://github. com/1579364808/Semeval_2025_task1.

YNU-HPCC at SemEval-2025 Task 1: Enhancing Multimodal Idiomaticity Representation via LoRA and Hybrid Loss Optimization

This paper introduces a hierarchical classification framework, termed the Tree-guided Stagewise Classifier (TGSC), which adopts a Chain-of-Thought (CoT) reasoning paradigm to address multi-label and multi-class classification challenges in multilingual news article analysis, as part of SemEval-2025 Task 10.
Our approach leverages the zero-shot capabilities of Large Language Models (LLMs) through a structured hierarchical reasoning process. The classification proceeds step-by-step through the hierarchy: beginning at the root node, the model iteratively traverses category branches, making decisions at each level, and ultimately identifying the appropriate leaf-level category.
To enhance classification accuracy, we design a prompt engineering strategy that embeds hierarchical structural constraints to better guide the reasoning process. Experimental results demonstrate the effectiveness of TGSC, showing competitive performance across multiple languages in both Subtask 1 and Subtask 2.

YNUzwt at SemEval-2025 Task 10: Tree-guided Stagewise Classifier for Entity Framing and Narrative Classification

The proliferation of multilingual misinformation demands robust systems for cross-lingual fact-checked claim retrieval. This work addresses SemEval-2025 Shared Task 7, which challenges participants to retrieve fact-checks for social media posts across 14 languages, even when posts and fact-checks are in different languages. We propose a hybrid retrieval pipeline that integrates both sparse lexical matching techniques and dense semantic retrieval methods. Our approach implements the dynamic fusion of these complementary retrieval strategies and employs curriculum-trained rerankers to optimize retrieval performance. Our system achieves 67.2% cross-lingual and 86.01% monolingual accuracy on the Shared Task MultiClaim dataset.

FactDebug at SemEval-2025 Task 7: Hybrid Retrieval Pipeline for Identifying Previously Fact-Checked Claims Across Multiple Languages

This poster presents our submission to SemEval-2025 Task 1, which addresses the novel challenge of idiomatic image retrieval, ranking images based on their alignment with the idiomatic or literal meaning of a phrase in context. Idioms often defy literal interpretation, making them difficult for NLP systems, especially in multimodal settings. We propose a modular Mixture of Experts framework that separates the processing of idiomatic and literal expressions while leveraging both textual and visual cues. The system uses a BERT-based classifier to detect idiomaticity and activates specialized expert modules that integrate CLIP embeddings and contrastive learning for improved image relevance scoring. Trained on the AdMIRe and AStitchInLanguageModels datasets, our model achieves strong performance, with 65% Rank-1 accuracy on the test set and 50% on the extended test set. Through comprehensive evaluations and ablation studies, we demonstrate the efficacy of our architecture and its potential for advancing multimodal idiom understanding.

Downloads

Next from ACL 2025

CLaC at SemEval-2025 Task 6: A Multi-Architecture Approach for Corporate Environmental Promise Verification

Similar lecture

Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMs

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2025

CLaC at SemEval-2025 Task 6: A Multi-Architecture Approach for Corporate Environmental Promise Verification

Similar lecture

Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMs

Downloads