Singapore

EMNLP 2023

Adversarial Robustness of Transformer-based Arabic Offensive Language Detectors

arabic offensive language

transformers

adversarial robustness

workshop paper

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

This study proposes a data-centric approach
for Neural Machine Translation (NMT) in low
resource languages, focusing on Moroccan di
alect called Darija. Synthetic data genera
tion from a Moroccan Arabic-English dictio
nary is employed, along with DarijaBERT and
ChatGPT for sequence generation and filter
ing. Transfer learning using mT5 models is
utilized, comparing a baseline model with a
synthetic data augmentation model. Results
show a significant improvement in translation
quality, demonstrating the efficacy of synthetic
data augmentation in addressing low-resource
NMT challenges.

Synthetic Data Augmentation for Low-Resource NMT A Case Study on Moroccan dialect

The Fon language, spoken by an average 2 million of people, is a truly low-resourced African language, with a limited online presence, and existing datasets (just to name but a few). Multitask learning is a learning paradigm that aims to improve the generalization capacity of a model by sharing knowledge across different but related tasks: this could be prevalent in very data-scarce scenarios. In this paper, we present the first explorative approach to multitask learning, for model capabilities enhancement in Natural Language Processing for the Fon language. Specifically, we explore the tasks of Named Entity Recognition (NER) and Part of Speech Tagging (POS) for Fon. We leverage two language model heads as encoders to build shared representations for the inputs, and we use linear layer blocks for classification relative to each task. Our results on the NER and POS tasks for Fon, show competitive (or better) performances compared to several multilingual pretrained language models finetuned on single tasks. Additionally, we perform a few ablation studies to leverage the efficiency of two different loss combination strategies and find out that the equal loss weighting approach works best in our case. Our code is open-sourced at https://github.com/bonaventuredossou/multitask_fon.

FonMTL: Towards Multitask Learning for the Fon Language

Currently there are a lot of State of the art (SOTA) models for text generation. These models are capable enough to generate text that looks as if they are written by a human. Despite the assistance it can provide for humans, these models can be leveraged to malicious tasks such as plagiarism. Especially when it comes to academia, there is an ethical concern based on the authenticity of the content in articles. Thus it is vital to distinguish machine generated text from human written content. Several novel detection tools claim higher detectability accuracies of more than 95%. However in our research we show that, when academic content is composed in a manner how a human would utilize generation models for the task, then those generated texts are difficult to be detected. We conduct experiments in detection by leveraging tools such as GPT2 output detector, DetectGPT and ZeroGPT and classification models such as SciBERT, RoBERTa and DeBERTa.

Is it an Easy Task to Accurately Detect Automatically Generated Academic Content?

This paper examines the main challenges in Amharic hate speech classification tasks. The potential sources of errors in Amharic hate speech classification are explored in detail. Out of a total of more than 18 million tweets in our Twitter corpus, we annotate over 15k tweets with two independent Amharic native-speaker annotators. A Cohen's kappa score of 48% for inter-annotator agreement is achieved. An adjudicator is employed to decide on disputed annotations and determine the final gold labels. Logistic regression, Linear SVM, and the two fine-tuned contextual embedding transformer models: AmFLAIR and AmRoBERTa are employed. Among all the models, AmFLAIR achieves the best performance result with an F1-score of 72%.

Error Analysis in Amharic Hate Speech Detection Task

YORC: Yoruba Reading Comprehension dataset

Due to significant increases in internet penetration and the development of smartphone technology during the preceding couple of decades, many people have started using social media as a communication platform. Social media has grown to be one of the most significant components, with several benefits. However, technology also poses a number of threats, challenges, and barriers, such as hate speech, disinformation, and fake news. all most all social media platforms can be accused of not doing enough to thwart hate speech on their platform. Amharic and Afaan Oromo language speakers frequently mix the two languages when conversing and posting on social media. 
In this work, a bilingual hate speech detection for Amharic and Afaan Oromo languages were conducted using four different deep learning classifiers (CNN, BiLSTM, CNN-BiLSTM, and BiGRU) and three feature extraction (Keras word embedding, word2vec, and FastText) techniques. According to the experiment result, BiLSTM with FastText feature extraction is an outperforming the other algorithm by achieving a 78.05 % percent of accuracy for Bilingual Amharic Afaan Oromo hate speech detection. Furthermore, we are working towards multilingual hate speech detection for other under-represented Ethiopian languages.

Bilingual Hate Speech Detection on Social Media Amharic and Afaan Oromo

Large Language Models (LLMs) have taken center stage in recent years in the field of NLP. But while they have shown significant performance gains in many NLP tasks, their performance is concentrated in Anglo-Centric languages. Although previous studies have highlighted shortcomings of LLMs---including perpetuating harms and biases---we know little about their failure cases in low-resourced languages. We investigate how ChatGPT responds in Amharic and Afan Oromo, two low-resourced languages spoken in Ethiopia. We taxonomize the failure cases we observed and provide examples to illustrate the shortcomings. Our results show text generated in these languages has 6 failure modes and additionally exhibits political utterances that are not related to the prompts. We hope our work will contribute to the broader understanding of LLMs for low-resourced languages.

Evaluating Large Language Models in Low-Resource Settings

Multilingual language models (MLMs) have democratized access to information and artificial intelligence. However, low-resource languages remain underrepresented. This study introduces code-switching as a test to assess the low-resource multilingual capabilities of MLMs using standard Basque-English code-switching as a benchmark. The models were evaluated focusing on output naturalness, code-switching aptitude, and hallucination frequency. Employing codeswitching as an evaluative measure revealed significant differences in naturalness and the ability to produce grammatical codeswitched outputs across models. The paper finds that models have accurate knowledge of language contact, as shown by the mixing of languages out of the prompt domain.

Do You Speak Basquenglish? Assessing Low-resource Multilingual Proficiency of Pretrained Language Models

Social media data has become a crucial re- source for understanding and detecting mental health challenges. However, there is a significant gap in our understanding of age-specific linguistic markers associated with classifying depression. This study bridges the gap by analyzing 25,241 text samples from 15,156 Reddit users with self-reported depression across two age groups: adolescents (13-20 year olds) and adults (21+). Through a quantitative exploratory analysis using LIWC, topic modeling, and data visualization, distinct patterns and topical differences emerged in the language of depression for adolescents and adults, including social concerns, temporal focuses, emotions, and cognition. These findings enhance our understanding of how depression is expressed on social media, bearing implications for accurate classification and tailored interventions across different age groups.

Premium content

Adversarial Robustness of Transformer-based Arabic Offensive Language Detectors

Next from EMNLP 2023

Synthetic Data Augmentation for Low-Resource NMT A Case Study on Moroccan dialect