United States

The rapid spread of misinformation on social media underscores the need for effective tools to analyze public discourse. In this thesis proposal, we focus on the concept of truthfulness stance and propose a truthfulness stance detection framework that leverages large language models (LLMs) with retrieval-augmented generation (RAG) to enhance the contextual understanding of tweets in relation to claims. The framework is evaluated on a newly developed dataset and established benchmark datasets.
Experimental results demonstrate that our approach outperforms state-of-the-art methods, significantly improving the Macro-F1 score. To highlight the practical impact of truthfulness stance detection in mitigating misinformation, we showcase several real-world applications and potential future directions.

NAACL 2025

Thesis Proposal: Understanding Misinformation on Social Media Through Truthfulness Stance

social sensing

large language model

computational social science

### Welcome to 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics

Welcome to the 2025 meeting of the Nations of the Americas Chapter of the Association for Computational Linguistics! I am proud to help organize the first NAACL conference to carry the new name of our organization, one that emphasizes inclusion for all of the Americas. I am also pleased to welcome you to Albuquerque, New Mexico, a state whose unique blend of cultural influences will make for an excellent backdrop for NAACL 2025, especially with this year’s special theme on NLP in a Multicultural World. 
**[Continue reading...](https://drive.google.com/file/d/1jX-qGhqVSZZCIrAnJaz798pflu5Irrdn/view?usp=sharing)**

*- Colin Cherry, Google, NAACL 2025 General Chair* 

[![](https://assets.underline.io/markdown_image/1/image/b087f8a4dc5816d6e1a6514e59c59ac3.png)](https://drive.google.com/file/d/1T96GzPqObXrMln2BMByCSXTSizjTg69P/view?usp=sharing)

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

To access NAACL 2025 event page you are required to register. Please follow [**this link**](https://2025.naacl.org/registration/registration/) to register. Access will depend on your registration type.

Please register!

Welcome to 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics

poster

We introduce CLEAR (Contrasting Textual Feedback with Experts and Amateurs for Reasoning), a novel approach to language model reasoning that leverages the strengths of a larger (expert) model and smaller (amateur) model. The expert and amateur models each provide feedback on a model's initial output and are contrasted with each other into refined feedback. This feedback is subsequently applied to iteratively improve CLEAR's responses. Across several challenging tasks, CLEAR outperforms other state-of-the-art reasoning methods, notably achieving a state-of-the-art score on the GSM8K benchmark (97.2%).

CLEAR: Contrasting Textual Feedback with Experts and Amateurs for Reasoning

In multi-turn dialogues, large language models face a critical challenge of ensuring coherence while adapting to user-specific information.. This study introduces the persona knowledge gap, the discrepancy between a model’s internal understanding and the knowledge required for coherent, personalized conversations. While prior research has recognized these gaps, computational methods for their identification and resolution remain underexplored. We propose Conversation Preference Elicitation and Recommendation (CPER), a novel framework that dynamically detects and resolves persona knowledge gaps using intrinsic uncertainty quantification and feedback-driven refinement. CPER consists of three key modules: a Contextual Understanding Module for preference extraction, a Dynamic Feedback Module for measuring uncertainty and refining persona alignment, and a Persona-Driven Response Generation module for adapting responses based on accumulated user context. We evaluate CPER on two real-world datasets: CCPE-M for preferential movie recommendations and ESConv for mental health support. Using A/B testing, human evaluators preferred CPER's responses 42% more often than baseline models in CCPE-M and 27% more often in ESConv. A qualitative human evaluation confirms that CPER's responses are preferred for maintaining contextual relevance and coherence, particularly in longer (12+ turn) conversations.

(CPER) From Guessing to Asking: An Approach to Resolving Persona Knowledge Gap in LLMs during Multi-Turn Conversations

The widespread integration of large language models (LLMs) across diverse domains, from entertainment to healthcare and banking, has underscored the pressing need to address the risks associated with artificial intelligence (AI). Indeed, as LLMs have grown in scale, new challenges have emerged, including privacy breaches, dissemination of misinformation, issues related to robustness and alignment, and concerns regarding the societal biases ingrained in these models. In response, there has been a surge in research focused on responsible AI and AI safety. To address these challenges, recent literature has introduced novel techniques for mitigating the risks associated with LLMs and testing the efficacy of safety protocols through red-teaming. Despite these efforts, concerns persist regarding the perpetuation of biases in next-generation LLMs and the lack of robustness in existing safety measures. Additionally, recent studies have highlighted the tendency of safety safeguards to over-fit certain keywords or identity terms, raising questions about their effectiveness in real-world scenarios. In this thesis proposal, we look at common evaluation practices in LLMs safety and their potential pitfalls.

Rethinking Safety Evaluation in Large Language Models: A Research Proposal

Large language models (LLMs) like GPT-4 and LLaMA-3 offer transformative potential across industries, e.g., enhancing customer service, revolutionizing medical diagnostics, or identifying crises in news articles. However, deploying LLMs faces challenges such as limited training data, high computational costs, and issues with transparency and explainability. Our research focuses on distilling compact, parameter-efficient tailored language models (TLMs) from LLMs for domain-specific tasks with comparable performance. Current approaches like knowledge distillation, fine-tuning, and model parallelism address computational efficiency but lack hybrid strategies to balance efficiency, adaptability, and accuracy. We present ANON - an adaptive knowledge distillation framework integrating knowledge distillation with adapters to generate computationally efficient TLMs without relying on labeled datasets. ANON uses cross-entropy loss to transfer knowledge from the teacher's outputs and internal representations while employing adaptive prompt engineering and a progressive distillation strategy for phased knowledge transfer. We evaluated ANON's performance in the crisis domain, where accuracy is critical and labeled data is scarce. Experiments showed that ANON outperforms recent approaches of knowledge distillation, both in terms of the resulting TLM performance and in reducing the computational costs for training and maintaining accuracy compared to LLMs for domain-specific applications.

Streamlining LLMs: Adaptive Knowledge Distillation for Tailored Language Models

Prompt optimization is crucial for improving output quality of Large Language Models (LLMs), but many existing methods are inefficient, requiring extensive computation and manual tuning. We propose Momentum-Aided Prompt Optimization (MAPO), which builds on ProTeGi by incorporating positive natural language "gradients" and a momentum-based memory mechanism to refine prompts while avoiding local minima and oscillations. It also employs beam search and an Upper Confidence Bound (UCB) algorithm for balanced candidate expansion and selection. MAPO achieves faster convergence time with fewer API calls and higher performance than ProTeGi, demonstrating it as a robust and scalable solution for automated prompt optimization in LLMs.

Momentum-Aided Natural Language Gradient Descent for Prompt Optimization

Rapid improvements in large language models have unveiled a critical challenge in human-AI interaction: sycophancy. In this context, sycophancy refers to the tendency of models to excessively agree with or flatter users, often at the expense of factual accuracy. While previous studies have primarily analyzed this behavior in single-turn interactions, its persistence and evolution in multi-step conversations remain largely unexplored. We introduce METHODNAME , a benchmark specifically designed to evaluate sycophancy in extended dialogues, where language models must navigate iterative user feedback, challenges, and persuasion. We prompt models to elicit four types of sycophantic biases. We then propose and test sycophancy reduction strategies, evaluating their effectiveness beyond single-step interactions.

TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Models

The recent large-scale emergence of LLMs has left an open space for dealing with their consequences, such as plagiarism or the spread of false information on the Internet. Coupling this with the rise of AI detector bypassing tools, reliable machine-generated text detection is in increasingly high demand. We investigate the paraphrasing attack resilience of various machine-generated text detection methods, evaluating three approaches: fine-tuned RoBERTa, Binoculars, and text feature analysis, along with their ensembles using Random Forest classifiers. We discovered that Binoculars-inclusive ensembles yield the strongest results, but they also suffer the most significant losses during attacks. In this paper, we present the dichotomy of performance versus resilience in the world of AI text detection, which complicates the current perception of reliability among state-of-the-art techniques.

Paraphrasing Attack Resilience of Various Machine-Generated Text Detection Methods

In recent years stylometry has been more investigated in many different fields. Hence, in this work, we are going to study detecting, generating, and evaluating the writing styles of different authors. In the first step, the sentences will be extracted from several different books, each belonging to a different author, to create a dataset. Then the selected models will be trained to detect the author of sentences in the dataset. After that generator models will be trained without paired dataset to generate sentences based on the authors' writing styles. Finally, in order to evaluate the performance of generators, the first trained models will be leveraged to evaluate the generated sentences.

Detecting, Generating, and Evaluating in the Writing Style of Different Authors

Data visualization is integral to any Exploratory Data Analysis (EDA) task. However, generating visualization requires expertise, presenting a steep learning curve and a significant cognitive load. Natural Language Interfaces (NLIs) for EDA aim to lower this barrier by allowing users to generate visualizations through natural language queries. However, complexity remains when EDA is performed collaboratively, requiring an environment to support multi-user interaction. In this thesis proposal, we discuss challenges in user-system interaction in a collaborative multi-user setup, such as errors in visualization generation due to misinterpretation of user requests. We hypothesize that a Conversational Assistant (CA) capable of understanding user-initiated clarification requests and generating accurate responses can improve user experience and support collaborative EDA tasks. To this end, we propose to develop such a CA (Figure [tab:system_issues]) and evaluate it through a user study, thus examining its impact on user experience in a collaborative environment for EDA.

Collaborative Data Exploration through Visualization: A Thesis Proposal Analyzing Impact of Conversational Assistants

This paper proposes SkipCLM, a novel method for improving multilingual machine translation in decoder transformers. We augment contrastive learning for cross-lingual alignment with a trainable skip connection to preserve information crucial for accurate target language generation. Experiments with XGLM-564M on the Flores-101 benchmark demonstrate improved performance, particularly for en-de and en-zh direction translation, compared to both direct sequence-to-sequence training and existing contrastive learning methods.

Next from NAACL 2025

CLEAR: Contrasting Textual Feedback with Experts and Amateurs for Reasoning

Similar lecture

RATSD: Retrieval Augmented Truthfulness Stance Detection from Social Media Posts Toward Factual Claims