Malta

Automated essay scoring of second-language learner essays is a high-stakes task, as it can affect the job and educational opportunities a student may have access to. Thus, it becomes imperative to make sure that the essays are graded based on the students&#39; language proficiency as opposed other reasons, such as personal names used in the text of the essay. Moreover, most of the research data is not freely accessible, as it tends to contain personal identifiable information. Because of that, pseudonlymization becomes an important tool to make sure that this data can be freely shared. Thus, our systems should not grade students based on which first names were used in the text of the essay, both for fairness and for privacy reasons. In this paper we explore how first names affect the CEFR level classification of essays of second language learners of Swedish. We used essays containing just one personal name and substituted it for names from lists of first names from four different ethnic origins, namely Swedish, Finnish, Anglo-American, and Arabic. We found that changing the names within the essays had no apparent effect on the classification task, regardless of whether a feature-based or a transformer-based model was used.

EACL 2024

Did the Names I Used within My Essay Affect My Score? Diagnosing Name Biases in Automated Essay Scoring

pseudonymization

automated essay scoring

fairness

bias

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible. The experience for on-site participants would closely approximate a normal pre-COVID *ACL conference, with 5-6 thematically organized parallel sessions and live Q/A and interactive discussion immediately after the talks. Presentations by virtual participants will be equitably interleaved with those of on-site participants, projected on the auditorium screens as if on-site, and also followed immediately by live Q/A and interactive discussion at a time during reasonable waking hours for the virtual presenter. For all participants, on-site and virtual, who are unable to attend a session due to either time-zone issues or because they are participating in another session live, talk recordings and slides will be available online at a minimum after the live presentation (and in many cases before as well), and questions may be submitted in advance on session-specific discussion boards and answered live in session with the usual visual aids if desired.

<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/ceby/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

Please Note: The EMNLP registration system is not currently connected to the underline site as we are still in the process of building out EMNLP 2021. You will receive access instructions from underline the week of November 1st. 

Access is given only to EMNLP upon registration, if you have not registered please do so [here](https://2021.emnlp.org/registration).

Registered attendees will receive access the week of November 1st.

EMNLP 2021

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible.

Automated essay scoring of second-language learner essays is a high-stakes task, as it can affect the job and educational opportunities a student may have access to. Thus, it becomes imperative to make sure that the essays are graded based on the students' language proficiency as opposed other reasons, such as personal names used in the text of the essay. Moreover, most of the research data is not freely accessible, as it tends to contain personal identifiable information. Because of that, pseudonlymization becomes an important tool to make sure that this data can be freely shared. Thus, our systems should not grade students based on which first names were used in the text of the essay, both for fairness and for privacy reasons. In this paper we explore how first names affect the CEFR level classification of essays of second language learners of Swedish. We used essays containing just one personal name and substituted it for names from lists of first names from four different ethnic origins, namely Swedish, Finnish, Anglo-American, and Arabic. We found that changing the names within the essays had no apparent effect on the classification task, regardless of whether a feature-based or a transformer-based model was used.

workshop paper

Welcome to the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL). Continuing its mission of expanding and involving the science community of all European countries, EACL has selected the Malta community for the 18th EACL. Considering the importance of physical interaction among researchers, the conference will be held at the Hotel Radisson Blu, St. Julians, in Malta, from 17 to 22 of March, 2024. The conference will also feature the possibility to participate virtually. As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.<br><br>
#### [Conference Handbook](https://drive.google.com/file/d/13Wn38q0yev6U3RTgnf4Tn5T9GuAuZ-xW/view?usp=sharing) *downloadable*<br><br><br><br>

To access the EACL 2024 event page you need to register [**here**](https://acl.swoogo.com/EACL2024). 

Please register!

As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

We describe the second-place submission for the shared task organized at the Fourth Workshop on Language Technology for Equality, Diversity, and Inclusion (LT-EDI-2024). The task focuses on detecting caste/immigration hate speech in Tamil. The included texts involve the Tamil language in both Tamil script and transliterated into Latin script, with some texts also in English. Considering different scripts, we examined the performance of 12 transformer language models on the dev set. Our analysis revealed that for the whole dataset, the model google/muril-large-cased performs the best. We used an ensemble of several models for the final challenge submission, achieving 0.81 for the test dataset.

kubapok@LT-EDI 2024: Evaluating Transformer Models for Hate Speech Detection in Tamil

This paper describes our homophobia/transphobia in social media comments detection system developed as part of the shared task at LT-EDI-2024. We took a transformer-based approach to develop our multiclass classification model for ten language conditions (English, Spanish, Gujarati, Hindi, Kannada, Malayalam, Marathi, Tamil, Tulu, and Telugu). We introduced synthetic and organic instances of script-switched language data during domain adaptation to mirror the linguistic realities of social media language as seen in the labelled training data. Our system ranked second for Gujarati and Telugu with varying levels of performance for other language conditions. The results suggest incorporating elements of paralinguistic behaviour such as script-switching may improve the performance of language detection systems especially in the cases of under-resourced languages conditions.

cantnlp@LT-EDI-2024: Automatic Detection of Anti-LGBTQ+ Hate Speech in Under-resourced Languages

This research focuses on Homophobia and Transphobia Detection in South Indian languages, specifically Telugu, Kannada, Tamil, and Malayalam. Leveraging the Homophobia/Transphobia Detection dataset, we propose an innovative approach employing a custom-designed tokenizer with a Bidirectional Long Short-Term Memory (BiLSTM) architecture. Our distinctive contribution lies in a tokenizer that reduces model sizes to below 7MB, improving efficiency and addressing real-time deployment challenges. The BiLSTM implementation demonstrates significant enhancements in hate speech detection accuracy, effectively capturing linguistic nuances. Low-size models efficiently alleviate inference challenges, ensuring swift real-time detection and practical deployment. This work pioneers a framework for hate speech detection, providing insights into model size, inference speed, and real-time deployment challenges in combatting online hate speech within South Indian languages.

byteLLM@LT-EDI-2024: Homophobia/Transphobia Detection in Social Media Comments - Custom Subword Tokenization with Subword2Vec and BiLSTM

In this paper, we describe our approaches and results for Task 2 of the LT-EDI 2024 Workshop, aimed at detecting homophobia and/or transphobia across ten languages. Our methodologies include monolingual transformers and ensemble methods, capitalizing on the strengths of each to enhance the performance of the models. The ensemble models worked well, placing our team, MasonTigers, in the top five for eight of the ten languages, as measured by the macro F1 score. Our work emphasizes the efficacy of ensemble methods in multilingual scenarios, addressing the complexities of language-specific tasks.

MasonTigers@LT-EDI-2024: An Ensemble Approach towards Detecting Homophobia and Transphobia in Social Media Comments

Speech recognition is known to be a specialized application of speech processing. Automatic speech recognition (ASR) systems are designed to perform the speech-to-text task. Although ASR systems have been the subject of extensive research, they still encounter certain challenges when speech variations arise. The speaker’s age, gender, vulnerability, and other factors are the main causes of the variations in speech. In this work, we propose a fine-tuned speech recognition model for recognising the spoken words of vulnerable individuals in Tamil. This research utilizes a dataset sourced from the LT-EDI@EACL2024 shared task. We trained and tested pre-trained ASR models, including XLS-R and Whisper. The findings highlight that the fine-tuned Whisper ASR model surpasses the XLS-R, achieving a word error rate (WER) of 24.452, signifying its superior performance in recognizing speech from diverse individuals.

CEN_Amrita@LT-EDI 2024: A Transformer based Speech Recognition System for Vulnerable Individuals in Tamil

Our work addresses the growing concern of abusive comments in online platforms, particularly focusing on the identification of Homophobia and Transphobia in social media comments. The goal is to categorize comments into three classes: Homophobia, Transphobia, and non-anti LGBT+ comments. Utilizing machine learning techniques and a deep learning model, our work involves training on a English dataset with a designated training set and testing on a validation set. This approach aims to contribute to the understanding and detection of Homophobia and Transphobia within the realm of social media interactions.In the era of digital communication, social media platforms play a crucial role in facilitating interactions, education, and information exchange. YouTube, a prominent social networking and video-sharing platform, allows users to create accounts, share videos, and engage with their audience through comments. Unfortunately, the presence of abusive comments on videos or responses to other users' comments can be offensive and detrimental to the mental well-being of platform users, especially when it comes to transgender and homo-sexual people.The surge in online communication has brought forth concerns regarding the dissemination of slander, hate speech, and online bullying. Slander, defined as a false spoken statement damaging to a person or group, is increasingly recognized for its harmful impact on individuals and society. Negative comments not only inflict psychological harm but also contribute to the spread of humor, division, and discontent online. Recognizing the profound consequences of hate speech, major platforms like YouTube, Facebook, Instagram, and Twitter have instituted policies and protocols to address hateful content and counteract negative behaviors. The objective of our study is to analyze and pinpoint offensive comments within an English dataset. English, known for its analytical or isolating nature, conveys grammatical relationships through word order and auxiliary words rather than extensive affixation. The task of detecting abusive comments is treated as a text classification problem, involving the extraction of features from the text and assigning it to predefined categories. This is achieved through a linear classifier operating on sentence embeddings. We conducted training on a variety of machine learning and deep learning models to identify abusive comments in English, and the results were compared to determine the optimal model.

KEC-AI-NLP@LT-EDI-2024:Homophobia and Transphobia Detection in social media comments using Machine Learning

This study investigates the robustness and generalization of transformer-based models for automatic media bias detection. We explore the behavior of current bias classifiers by analyzing feature attributions and stress-testing with adversarial datasets. The findings reveal a disproportionate focus on rare but strongly connotated words, suggesting a rather superficial understanding of linguistic bias and challenges in contextual interpretation. This problem is further highlighted by inconsistent bias assessment when stress-tested with different entities and minorities. Enhancing automatic media bias detection models is critical to improving inclusivity in media, ensuring balanced and fair representation of diverse perspectives.

Beyond the Surface: Spurious Cues in Automatic Media Bias Detection

As a large how-to website, wikiHow's mission is to empower every person on the planet to learn how to do anything. An important part of including everyone also linguistically is the use of gender-neutral language. In this short paper, we study in how far articles from wikiHow fulfill this criterion based on manual annotation and automatic classification. In particular, we employ a classifier to analyze how the use of gender-neutral language has developed over time. Our results show that although about 75% of all articles on wikiHow were written in a gender-neutral way from the outset, revisions have a higher tendency to add gender-specific language than to change it to inclusive wording.

A Diachronic Analysis of Gender-Neutral Language on wikiHow

Natural Language Processing techniques have been developed to assist in simplifying online content while preserving meaning. However, for low-resource languages, like Maltese, there are still numerous challenges and limitations. Lexical Simplification (LS) is a core technique typically adopted to improve content accessibility, and has been widely studied for high-resource languages such as English and French. Motivated by the need to improve access to Maltese content and the limitations in this context, this work set out to develop and evaluate an LS system for Maltese text. An LS pipeline was developed consisting of (1) potential complex word identification, (2) substitute generation, (3) substitute selection, and (4) substitute ranking. An evaluation data set was developed to assess the performance of each step. Results are encouraging and will lead to numerous future work. Finally, a single-blind study was carried out with over 200 participants, where the system's perceived quality in text simplification was evaluated. Results suggest that meaning is retained about 50% of the time, and when meaning is retained, about 70% of system-generated sentences are either perceived as simpler or of equal simplicity to the original. Challenges remain, and this study proposes a number of areas that may benefit from further research.

Towards Content Accessibility Through Lexical Simplification for Maltese as a Low-Resource Language

We introduce HATELEXICON, a lexicon of slurs and targets of hate speech for Brazil, Germany, India and Kenya, to aid model development and interpretability. First, we demonstrate how HATELEXICON can be used to interpret model predictions, showing that models developed to classify extreme speech rely heavily on target group names. Further, we propose a culturally-informed method to aid shot selection for training in low-resource settings. In few-shot learning, shot selection is of paramount importance to model performance and we need to ensure we make the most of available data. We work with HASOC German and Hindi data for training and the Multilingual HateCheck (MHC) benchmark for evaluation. We show that selecting shots based on our lexicon leads to models performing better than models trained on shots sampled randomly. Thus, when given only a few training examples, using HATELEXICON to select shots containing more sociocultural information leads to better few-shot performance. With these two use-cases we show how our HATELEXICON can be used for more effective hate speech detection.

Downloads

Next from EACL 2024

kubapok@LT-EDI 2024: Evaluating Transformer Models for Hate Speech Detection in Tamil

Similar lecture

Countering the Influence of Essay Length in Neural Essay Scoring

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from EACL 2024

kubapok@LT-EDI 2024: Evaluating Transformer Models for Hate Speech Detection in Tamil

Similar lecture

Countering the Influence of Essay Length in Neural Essay Scoring

Downloads