United States

Bias and fairness risks in Large Language Mod els (LLMs) vary substantially across deploy ment contexts, yet existing approaches lack systematic guidance for selecting appropriate evaluation metrics. We present a decision framework that maps LLM use cases, charac terized by a model and population of prompts, to relevant bias and fairness metrics based on task type, whether prompts contain protected attribute mentions, and stakeholder priorities. Our framework addresses toxicity, stereotyp ing, counterfactual unfairness, and allocational harms, and introduces novel metrics based on stereotype classifiers and counterfactual adapta tions of text similarity measures. We release an open-source Python library, langfair, for practical adoption. Extensive experiments on use cases across five LLMs and five prompt populations demonstrate that fairness risks can not be reliably assessed from benchmark per formance alone: results on one prompt dataset likely overstate or understate risks for another, underscoring that fairness evaluation must be grounded in the specific deployment context.

ACL 2026

Bring Your Own Prompts: Use-Case-Specific Bias and Fairness Evaluation for LLMs

fairness

bias

### Welcome!
The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. Our Virtual Poster Sessions will take place online Thursday, August 22, 2024.

You are required to register for this event. **Please register [here](https://2024.aclweb.org/registration). **

If you have already registered, please check your inbox for an email from Underline granting you access to ACL 2024 content.

Please register!

ACL 2024

The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. More information will be announced soon.

workshop paper

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/acl2026

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

Multilingual Natural Language Understanding (NLU) systems often struggle to adapt when new languages or new semantic labels are in troduced with only a few annotated examples. This challenge is particularly pronounced for low-resource languages, where limited super vision and evolving label spaces make conven tional joint-label classification approaches un stable. Most existing multilingual NLU models treat each language–semantic pair as an inde pendent class, entangling linguistic and seman tic representations and hindering few-shot adap tation. We propose Dual-Axis Compositional Few-Shot Learning, a framework that explicitly factorizes the representation space into linguis tic and semantic embedding axes, enabling in dependent modeling of language variation and domain–intent semantics. Joint representations are constructed compositionally through mul tiplicative interaction of axis-specific embed dings, allowing controlled adaptation when ei ther the language set or the semantic label space evolves. The framework integrates factorized prototype learning, axis-structured contrastive alignment, and disentanglement regularization using HSIC-based statistical independence and Jacobian-based cross-axis decorrelation. Ex periments on six low-resource Indic languages spanning Indo-Aryan and Dravidian families (Hindi, Bengali, Sanskrit, Assamese, Tamil, and Telugu) demonstrate strong performance under two structured generalization regimes. The model achieves 81.12% accuracy when adapting to few-shot languages with known semantics and 63.5% accuracy when learning new semantic classes from few-shot examples, along with an accuracy of 89.56% on known languages and seen semantics. These results show that axis-factorized representations en able stable compositional generalization, offer ing a promising direction for scalable multilin gual NLU in linguistically diverse low-resource settings.

Dual-Axis Compositional Contrastive Few-Shot Learning using Prototypes Across Linguistic and Semantic Dimensions for Indic Low-Resource Multilingual NLU

Automatic text simplification (ATS) aims to en hance language accessibility for various target groups, particularly persons with intellectual disabilities. Recent advancements in large lan guage models (LLMs) have substantially im proved the quality of machine-generated text simplifications, however, existing LLM-based ATS systems do not incorporate preference feedback during post-training, resulting in a lack of personalization tailored to the specific needs of target group persons. In this work, we propose an ATS personalization framework using direct preference optimization (DPO). Specifically, we post-trained LLM-based ATS models using human feedback collected from persons with intellectual disabilities, reflecting their preferences of paired text simplifications generated by mainstream LLMs. Our pipeline for developing personalized LLM-based ATS systems encompasses data collection, model se lection, supervised fine-tuning (SFT) and DPO post-training, and result evaluation. Our find ings underscore the necessity of active partici pation of target group persons in designing per sonalized inclusive AI solutions aligned with human preferences.

Evaluating Direct Preference Optimization for Personalizing German Automatic Text Simplifications for Persons with Intellectual Disabilities

This paper presents a methodology that uses LLMs to align multilingual offensive lexi cons at the sense level. Lexicons of differ ent structures and origins in Arabic, Bulgar ian, Modern Greek, French, and Italian have been aligned directly without pivoting through English. The Modern Greek lexicon is LLM generated, and the other four lexicons are WordNet-compatible. For inter-language align ment of senses, an LLM-as-a-judge rubric was used over lemma–definition–example triples. The LLM makes 2.87M pairwise comparisons and yields 31 strict global-sense categories. The paper discusses the challenges involved in sense alignment tasks. The resource is avail able to support downstream applications such as Machine Translation and cross-lingual hate speech detection.

From Form to Meaning: Interlingua Sense-Alignment of Offensive Language with LLMs

Large Language Models (LLMs) perform strongly on many language tasks, but their capa bility in structurally constrained, accessibility critical modalities such as Braille remains un clear. We evaluate state-of-the-art LLMs on bidirectional Korean–Braille translation using a human-annotated dataset. Despite expecta tions that multilingual, instruction-tuned mod els can generalize to Braille via text repre sentations, we find consistently poor, unstable outputs and substantial disagreement with hu man judgments. These results point to missing Braille-aware tokenization and weak alignment between Korean and Braille patterns. In con trast, supervised fine-tuning of a small model (T5-small) on the same data yields large and stable gains over zero-shot and prompted LLM baselines across standard metrics (SacreBLEU, ChrF++, CER, BLEU, ROUGE-L, METEOR, CIDEr). Our findings reveal a systematic lim itation of current LLMs and demonstrate the effectiveness of modest task-specific supervi sion.

I'm Sorry, but I Can't Help with Braille: Revealing Accessibility Failures in State-of-the-Art LLMs

We investigate the role of large language mod els (LLMs) in promoting gender-inclusive lan guage by evaluating their ability to rewrite bi ased text and generate counterfactual narratives across multiple languages. We introduce a shared task with two subtasks: gender-inclusive rewriting and counterfactual generation. The task covers five languages English, German, Spanish, Tamil, and Kannada reflecting diverse grammatical gender systems and sociocultural contexts. We release curated word-level and sentence-level datasets to support controlled inclusive generation. A total of 50 teams reg istered for the shared task, and around 8 teams submitted results. Submissions are evaluated using a hybrid framework combining rubric based automatic scoring with expert human judgment. Finally, we provide an overview of participating systems and discuss key findings and challenges observed across languages.

Insights from Multilingual Gender Inclusive Language Generation Shared Task

Gender bias in multilingual language gener ation systems poses serious ethical and so cial issues, especially in languages with com plex morphology. In this study, we propose a lightweight multilingual approach that employs instruction-guided fine-tuning of the mT5 small transformer model for gender-inclusive language generation. The framework accom modates five languages: English, German, Spanish, Tamil, and Kannada. The approach uses the task-prefix rewriting method to trans form gender-specific sentences to their gender neutral versions. The training data from differ ent languages is combined into a single multi lingual dataset for sequence-to-sequence fine tuning. Beam search decoding with repetition constraints is used during inference to improve the quality of the output. The system’s perfor mance is measured using GIFI, semantic simi larity, and an overall combined score across all languages. Experimental results show that the system can eliminate gender-biased language while retaining semantic meaning in part across languages.

CAI@LTEDI 2026: Multilingual Gender Inclusive Language Generation using Instruction-Guided mT5 Transformer Model

The rapid growth of social media has also led to a rise in abusive and harmful content, which negatively affects the online environment for users. The frequent use of offensive language and hate speech contributes to making these platforms increasingly hostile. In particular, homophobic and transphobic remarks target members of the LGBT+ community. Detecting such comments is therefore essential so that they can be flagged promptly and appropriate warnings can be given to users involved in such behaviour. The problem becomes more serious when such content appears in other forms of communication used by younger generations, such as memes. This work tries to address this issue. We propose a method to detect such content using the meme dataset from the LT-EDI 2026 challenge and secured 8th rank for English and 6th rank for Chinese language dataset in the shared task. Our approach uses a multimodal technique that processes both image and text information. The dataset has limited data, which creates a challenge. To handle this, we pre–fine-tune the models on a similar dataset called PrideMM. The proposed multimodal approach achieved Macro F1-scores of 0.24 and 0.57 for English and Chinese memes respectively

CuriousVectors@LT-EDI 2026: Detection of Homophobic and Transphobic Memes on Social Media Using a Hybrid Multimodal Approach

Online hate speech is spreading rapidly, creat ing significant challenge, particularly in low resource language such as Tamil. Lack of de veloped automated content moderation systems makes it difficult to control harmful content effectively. In this study, we propose a com putational framework for generating Counter Narratives (CNs) using classical NLP tech niques. With this, we leverage TF-IDF fea tures with n-grams to identify the labels as Ho mophobic or Transphobic. Span detection is performed with TF-IDF features with n-grams and Machine learning models. Counter narra tives are then retrieved by computing cosine similarity, ensuring semantic alignment and contextual relevance. Evaluation on the ex panded human curated dataset demonstrates that our approach produces contextually appro priate and semantically coherent counter nar ratives. Notably, the proposed system is sub mitted at Task 2 shown a overall average score of 80.40 % for Tamil and 77.29 % for English and secured first and fourth rank respectively. GitHub: https://github.com/kannanrrk/Span Counter-Feature-Based

DLRG@LT-EDI 2026: Automating Counter-Narratives for Homophobic and Transphobic Comments

The detection and response to homophobic and transphobic comments are important chal lenges in Natural Language Processing. In this paper, we focus on the detection of span for homophobic and transphobic comments (Task 1) and generation of counter narratives for abu sive comments (Task 2) for the LT-EDI @ ACL 2026 shared task. Harmful comments made online against the LGBTQ+ community have created a hostile environment for users. In this paper, we have used the transformer model for the detection of span for homophobic and trans phobic comments and generation of counter narratives. In this task, the detection of the span of comments containing homophobic and trans phobic words and the generation of counter nar ratives for abusive comments have been done using the transformer model. The results show the efficiency of the transformer model in the detection of the span of comments and gener ation of counter narratives. This paper empha sizes the efficiency of the transformer model in creating a safe environment for users.

DuoNova@LTEDI 2026: Multilingual Span Detection and Counter-Narrative Generation on Homophobic and Transphobic Comments

The deployment of Large Language Models (LLMs) has intensified concerns regarding the propagation of societal stereotypes encoded with web-scale training corpora. This pa per presents a dual-paradigm framework spe cially designed to address multilingual gender inclusivity and counterfactual generation. For multilingual gender-neutral text transformation, a fine-tuned mT5 encoder–decoder model per forms controlled sentence rewriting with mini mal edits while preserving semantic fidelity and grammatical fluency. For counter-narrative gen eration, the Llama-3 8B decoder-only model is employed to generate empathetic and persua sive responses through structured prompt-based generation. The framework is evaluated using datasets from the LT-EDI ACL 2026 shared task across multiple languages, including En glish, Tamil, Kannada, German, and Spanish. Experimental results demonstrate strong effec tiveness in identifying and neutralizing gender markers, particularly in morphologically rich languages, while the counter-narrative compo nent achieves high performance in politeness, coherence, and relevance. Overall, the pro posed approach contributes toward the develop ment of responsible and inclusive multilingual NLPsystems.

Downloads

Next from ACL 2026

Dual-Axis Compositional Contrastive Few-Shot Learning using Prototypes Across Linguistic and Semantic Dimensions for Indic Low-Resource Multilingual NLU

Similar lecture

Why Don’t Prompt-Based Fairness Metrics Correlate?

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

Dual-Axis Compositional Contrastive Few-Shot Learning using Prototypes Across Linguistic and Semantic Dimensions for Indic Low-Resource Multilingual NLU

Similar lecture

Why Don’t Prompt-Based Fairness Metrics Correlate?

Downloads