United States

Research to tackle hate speech plaguing online media has made strides in providing solutions, analyzing bias and curating data. A challeng- ing problem is ambiguity between hate speech and offensive language, causing low perfor- mance both overall and specifically for the hate speech class. It can be argued that misclas- sifying actual hate speech content as merely offensive can lead to further harm against tar- geted groups. In our work, we mitigate this potentially harmful phenomenon by proposing an adversarial debiasing method to separate the two classes. We show that our method works for English, Arabic German and Hindi, plus in a multilingual setting, improving performance over baselines.

NAACL 2022

Separating Hate Speech and Offensive Language Classes via Adversarial Debiasing

debiasing

hate speech

adversarial training

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

workshop paper

With the widespread use of social media, online hate is increasing, and microaggressions, unintentional offensive remarks in everyday life, are receiving
attention. We explore the possibility of using pre-trained language models to
automatically generate messages that combat the associated offensive texts.
Specifically, we focus on using prompting to steer model generation as it requires fewer data and computation than fine-tuning and shows the potential for using prompting in the proposed generation task. We also propose a human evaluation perspective; offensiveness, stance, and informativeness. After obtaining 306 counterspeech and 42 micro intervention messages generated by GPT-2, GPT-Neo, and GPT-3, we conducted a human evaluation using Amazon Mechanical Turk and found that GPT-3 produces messages of the highest quality among the three systems. Also, We discuss the pros and cons of using our evaluation perspectives. We release a corpus of Countering HAte Speech and Microaggressions (CHASM), annotated machine-generated counternarratives along with the annotation to promote further research on automatic counternarrative generation and its evaluation.

Towards Automatic Generation of Messages Countering Online Hate Speech andMicroaggressions

This paper presents a comprehensive corpus for the study of socially unacceptable language in Dutch. The corpus extends and revise an existing resource with more data and intro- duces a new annotation dimension for offen- sive language, making it a unique resource in the Dutch language panorama. Each language phenomenon (abusive and offensive language) in the corpus has been annotated with a multi- layer annotation scheme modelling the explic- itness and the target(s) of the abuse/offence in the message. We have conducted a new set of experiments with different classification algo- rithms on all annotation dimensions. Monolin- gual Pre-Trained Language Models prove as the best systems, obtaining a macro-average F1 of 0.828 for binary classification of offensive language, and 0.579 for the targets of offensive messages. Furthermore, the best system obtains a macro-average F1 of 0.637 for distinguishing between abusive and offensive messages.

“Zo Grof !”: A Comprehensive Corpus for Offensive and Abusive Language in Dutch

Toxic language can take many forms, from explicit hate speech to more subtle microaggressions. Within this space, models identifying transphobic language have largely focused on overt forms. However, a more pernicious and subtle source of transphobic comments comes in the form of statements made by Trans-exclusionary Radical Feminists (TERFs); these statements often appear seemingly-positive and promote women's causes and issues, while simultaneously denying the inclusion of transgender women as women. Here, we introduce two models to mitigate this antisocial behavior. The first model identifies TERF users in social media, recognizing that these users are a main source of transphobic material that enters mainstream discussion and whom other users may not desire to engage with in good faith. The second model tackles the harder task of recognizing the masked rhetoric of TERF messages and introduces a new dataset to support this task. Finally, we discuss the ethics of deploying these models to mitigate the harm of this language, arguing for a balanced approach that allows for restorative interactions.

The subtle language of exclusion: Identifying the Toxic Speech of Trans-exclusionary Radical Feminists

In an era of increasingly large pre-trained language models, knowledge distillation is a powerful tool for
transferring information from a large model to a smaller one. In particular, distillation is of tremendous
benefit when it comes to real-world constraints such as serving latency or serving at scale. However, 
a loss of robustness in language understanding may be hidden in the process and not immediately revealed
when looking at high-level evaluation metrics. We investigate the hidden costs: what
is "lost in distillation", especially in regards to identity-based model bias using the case study
of toxicity modeling. With 
reproducible models using open source training sets, we investigate models distilled
from a BERT teacher baseline. Using both open source and proprietary
big data models, we investigate these hidden performance costs.

Lost in Distillation: A Case Study in Toxicity Modeling

We present a cleansed version of the Modern Greek branch of the multilingual lexicon HURTLEX. The new version contains 737 offensive words. We worked bottom-up in two annotation rounds and developed detailed diagnostics of "offensiveness" by cross-classifying words on three dimensions: context, reference, and thematic domain. Our work reveals a wider spectrum of thematic domains concerning the study of offensive language than those identified in the Greek lexicographic literature as well as social and cultural aspects that are not included in the original HURTLEX categories.

Cleansing and expanding the HURTLEX-EL with a multidimensional categorization of offensive words

Social platforms such as Gab and Parler, branded as `free-speech' networks, have seen a significant growth of their user base in recent years. This popularity is mainly attributed to the stricter moderation enforced by mainstream platforms such as Twitter, Facebook, and Reddit.
In this work we provide the first large scale analysis of hate-speech on Parler. We experiment with an array of algorithms for hate-speech detection, demonstrating the limitations of transfer learning in that domain, given the illusive and ever changing nature of the ways hate-speech is delivered. In order to improve classification accuracy we annotated 10K Parler posts, which we use to fine-tune a BERT classifier. Classification of individual posts is then leveraged for the classification of millions of users via label propagation over the social network. Classifying users by their propensity to disseminate hate, we find that hate mongers make about 16% of Parler active users, and that they have distinct characteristics comparing to other user groups. We find that hate mongers are more active, more central, express distinct levels of sentiment, and convey a distinct array of emotions like anger and sadness. 
We further complement our analysis by comparing the trends observed in Parler to those found in Gab. To the best of our knowledge, this is among the first works to analyze hate speech in Parler in a quantitative manner and on the user level.

Free speech or Free Hate Speech? Analyzing the Proliferation of Hate Speech in Parler

Most of the published approaches and resources for offensive language and hate speech detection are tailored for the English language. In consequence, cross-lingual and cross-cultural perspectives lack some essential resources. The lack of diversity of the datasets in Spanish is notable. Variations throughout Spanish-speaking countries make existing datasets not enough to encompass the task in the different Spanish variants. We manually annotated 9834 tweets from Chile to enrich the existing Spanish resources with different words and new targets of hate that have not been considered in previous studies. We conducted several cross-dataset evaluation experiments of the models published in the literature using our Chilean dataset and two others in English and Spanish. 
We propose a comparative framework for quickly conducting comparative experiments using different previously published models. In addition, we set up a Codalab competition for further comparison of new models in a standard scenario, that is, data partitions and evaluation metrics.
All resources can be accessed through a centralized repository for researchers to get a complete picture of the progress on the multilingual hate speech and offensive language detection task.

Multilingual Resources for Offensive Language Detection

We present a new offensive language dataset (DeTox-dataset) containing 10,278 annotated German social media comments collected in the first half of 2021. With twelve different annotation categories annotated by six annotators, it is far more comprehensive than other datasets, and goes beyond just hate speech detection. The labels aim in particular also at toxicity, criminal relevance and discrimination types of comments. Furthermore, about half of the comments are from coherent parts of conversations, which opens the possibility to consider the comments contexts and do conversation analyses in order to research the contagion of offensive language in conversations.

DeTox: A Comprehensive Dataset for German Offensive Language and Conversation Analysis

Hate speech detection models are typically evaluated on held-out test sets. However, this risks painting an incomplete and potentially misleading picture of model performance because of increasingly well-documented systematic gaps and biases in hate speech datasets. To enable more targeted diagnostic insights, recent research has thus introduced functional tests for hate speech detection models. However, these tests currently only exist for English-language content, which means that they cannot support the development of more effective models in other languages spoken by billions across the world. To help address this issue, we introduce MULTILINGUAL HATECHECK (MHC), a suite of functional tests for multilingual hate speech detection models. MHC covers 34 functionalities across ten languages, which is more languages than any other hate speech dataset. To illustrate MHC’s utility, we train and test a high-performing multilingual hate speech detection model, and reveal critical model weaknesses for monolingual and cross-lingual applications.

Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models

The subjectivity of recognizing hate speech makes it a complex task. This is also reflected by different and incomplete definitions in NLP. We present hate speech criteria, developed with perspectives from law and social science, with the aim of helping researchers create more precise definitions and annotation guidelines on five aspects: (1) target groups, (2) dominance, (3) perpetrator characteristics, (4) type of negative group reference, and the (5) type of potential consequences/effects. Definitions can be structured so that they cover a more broad or more narrow phenomenon. As such, conscious choices can be made on specifying criteria or leaving them open. We argue that the goal and exact task developers have in mind should determine how the scope of hate speech is defined. We provide an overview of the properties of English datasets from hatespeechdata.com that may help select the most suitable dataset for a specific scenario.

Separating Hate Speech and Offensive Language Classes via Adversarial Debiasing

Downloads

Next from NAACL 2022

Towards Automatic Generation of Messages Countering Online Hate Speech andMicroaggressions

Similar lecture

SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification

Separating Hate Speech and Offensive Language Classes via Adversarial Debiasing

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from NAACL 2022

Towards Automatic Generation of Messages Countering Online Hate Speech andMicroaggressions

Similar lecture

SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification

Downloads