Canada

Introduction to our submission to SemEval-2023 Task 10: Explainable Detection of Online Sexism by Fine-tuning Language Model Ensembles.


ACL 2023

UIRISC at SemEval-2023 Task 10: Explainable Detection of Online Sexism by Ensembling Fine-tuning Language Models

sexism detection

fine-tuning

### Welcome to ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics! 
 The conference will be held in Toronto, Canada, July 9-14, 2023. 
Following the succession of the recent conferences in our field, ACL 2023 will adopt a hybrid format.
While the impact of Covid has considerably diminished in terms of traveling, obtaining visas to Canada
entails a very long process. Moreover, the global economic conditions pose challenges for many individuals to travel to conferences. Recognizing these circumstances, we know many participants may not be
able to attend the conference in person. Therefore, we are committed to providing a great virtual platform
so everyone has the opportunity to interact with other participants and enjoy the conference. Based on the
current registered participants, approxiately 30% have chosen to attend the conference virtually. Whether
you join us in person or virtually, we sincerely hope everyone has a remarkable conference experience. 
This General Chair’s message is where I express my gratitude to the many individuals who have made
enormous contributions to the conference over the past year.

Read [**ACL 2023 General Chair's message**](https://docs.google.com/document/d/1WobYM7norbG4dI48s75HfJoD89qgX5a_F-6U8AteLSA/edit?usp=sharing/) in full.

##### **[Conference Handbook](https://2023.aclweb.org/downloads/acl2023-handbook.pdf)**

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing.

workshop paper

The class imbalance problem can cause machine learning models to produce an undesirable performance on the minority class as well as the whole dataset. Using data augmentation techniques to increase the number of samples is one way to tackle this problem. We introduce a novel counterfactual data augmentation by verb replacement for the identification of medical claims. In addition, we investigate the impact of this method and compare it with 3 other data augmentation techniques, showing that the proposed method can result in a significant (relative) improvement in the minority class.

CAISA at SemEval-2023 Task 8: Counterfactual Data Augmentation for Mitigating Class Imbalance in Causal Claim Identification

Our study describes a multilingual persuasion detection system "ReDASPersuasion" that incorporates persuasion technique attributes for a multi-label classification task. The proposed method combines language agnostic feature embeddings with sequence classification using pre-trained multilingual transformers. It supports a total of 100 languages, guaranteed by the multilingual transformer module and the Google translator interface. Our persuasion system outperforms the SemEval Task 3 Substak3 baseline in all languages, except zero shot prediction languages, which was not the main focus of our research. With the highest F1-Micro score of 0.45 for Italian, it achieved the eighth position on the official leaderboard before the end of the test phase. The reopening of the leaderboard allowed us to improve the "ReDASPersuasion" system's classification which got us currently ranked second in both Italian and French.

Team ReDASPersuasion at SemEval-2023 Task 3: Persuasion Detection Using Multilingual Transformers and Language Agnostic Features

Literature permeates almost every facet of our lives, whether through books, magazines, or in- ternet articles. Moreover, every piece of written work contains ideas and opinions that we tend to relate to, accept or disregard, the debate over, or enlighten ourselves. However, the existence of subtle themes that are difficult to discern inspired us to utilize four machine learning al- gorithms: Decision Trees, Random Forest, Lo- gistic Regression, and Support Vector Classifier (SVC) to aid in their detection. Trained on the ValueEval data set as a multi-label classifica- tion problem, the supervised machine learning models did not perform as well as expected, with F1 metrics hovering from 0.0 to 0.04 for each value. Noting this, our paper discusses our approach’s limitations and weaknesses.

Friedrich Nietzsche at SemEval-2023 Task 4: Detection of Human Values from Text Using Machine Learning

In this presentation, we introduce our approach to Visual Word Sense Disambiguation (Visual-WSD) which leverages prompt engineering, text-to-image systems, and CLIP for evaluation. We compare the performance of GLIDE and DALL-E 2 as text-to-image systems, utilizing Simple and Generated prompts, with the evaluation carried out through the CLIP model. Our research highlights the effectiveness of image modality over cross-modality for Visual-WSD tasks and shows DALL-E 2's superior performance in image generation. The presentation provides insights into the potential of our approach and opens up new avenues for further development in multimodal word sense disambiguation.

Ebhaam at SemEval-2023 Task 1: Comparing Cross-Domain and Same-Domain CLIP-Based Approaches for Visual Word Sense Disambiguation

This paper presents the system developed by the Sartipi-Sedighin team for SemEval 2023 Task 2, which is a shared task focused on multilingual complex named entity recognition (NER), or MultiCoNER II. The goal of this task is to identify and classify complex named entities (NEs) in text across multiple languages. To tackle the MultiCoNER II task, we leveraged pre-trained language models (PLMs) fine-tuned for each language included in the dataset. In addition, we also applied a data augmentation technique to increase the amount of training data available to our models. Specifically, we searched for relevant NEs that already existed in the training data within Wikipedia, and we added new instances of these entities to our training corpus.
Our team achieved an overall F1 score of 61.25 in the English track and 71.79 in the multilingual track across all 13 tracks of the shared task that we submitted to.

Sartipi-Sedighin at SemEval-2023 Task 2: Fine-grained Named Entity Recognition with Pre-trained Contextual Language Models and Data Augmentation from Wikipedia

This work describes our submission to Task 10 at SemEval 2023-Explainable Detection of Online Sexism (EDOS), divided into three subtasks. The recent rise in social media platforms has seen an increase in disproportionate levels of sexism experienced by women on social media platforms. This has made detecting and explaining online sexist content more important than ever to make social media safer and more accessible for women. Our approach consists of experimenting and finetuning BERT-based models and using a Majority Voting ensemble model that outperforms individual baseline model scores. Our system achieves a macro F1 score of 0.8392 for Task A, 0.6092 for Task B and 0.4319 for Task C and the code is available on GitHub.

SSS at SemEval-2023 Task 10: Explainable Detection of Online Sexism Using Majority Voted Fine-Tuned Transformers

While static word embedding models are known to represent linguistic analogies as parallel lines in high-dimensional space, the underlying mechanism as to why they result in such geometric structures remains obscure. We find that an elementary contrastive-style method employed over distributional information performs competitively with popular word embedding models on analogy recovery tasks, while achieving dramatic speedups in training time. Further, we demonstrate that a contrastive loss is sufficient to create these parallel structures in word embeddings, and establish a precise relationship between the co-occurrence statistics and the geometric structure of the resulting word embeddings.

Contrastive Loss is All You Need to Recover Analogies as Parallel Lines

Diffusion probabilistic models have shown great success in generating high-quality images controllably, and researchers have tried to utilize this controllability into text generation domain. Previous works on diffusion-based language models have shown that they can be trained without external knowledge (such as pre-trained weights) and still achieve stable performance and controllability. In this paper, we trained a diffusion-based model on StylePTB dataset, the standard benchmark for fine-grained text style transfers. The tasks in StylePTB requires much more refined control over the output text compared to tasks evaluated in previous works, and our model was able to achieve state-of-the-art performance on StylePTB on both individual and compositional transfers. Moreover, our model, trained on limited data from StylePTB without external knowledge, outperforms previous works that utilized pretrained weights, embeddings, and external grammar parsers, and this may indicate that diffusion-based language models have great potential under low-resource settings.

Fine-grained Text Style Transfer with Diffusion-Based Language Models

Sentence embeddings induced with various transformer architectures encode much semantic and syntactic information in a distributed manner in a one-dimensional array. We investigate whether specific grammatical information can be accessed in these distributed representations. Using data from a task developed to test rule-like generalizations, our experiments on detecting subject-verb agreement yield several promising results. First, we show that while the usual sentence representations encoded as one-dimensional arrays do not easily support extraction of rule-like regularities, a two-dimensional reshaping of these vectors allows various learning architectures to access such information. Next, we show that various architectures can detect patterns in these two-dimensional reshaped sentence embeddings and successfully learn a model based on smaller amounts of simpler training data, which performs well on more complex test data. This indicates that current sentence embeddings contain information that is regularly distributed, and which can be captured when the embeddings are reshaped into higher dimensional arrays. Our results cast light on representations produced by language models and help move towards developing few-shot learning approaches.

Grammatical information in BERT sentence embeddings as two-dimensional arrays

Recent work has demonstrated that using parameter efficient tuning techniques such as prefix tuning (or P-tuning) on pretrained language models can yield performance that is comparable or superior to fine-tuning while dramatically reducing trainable parameters. Nevertheless, the effectiveness of such methods under the context of data augmentation, a common strategy to improve learning under low data regimes, has not been fully explored. In this paper, we examine the effectiveness of several popular task-agnostic data augmentation techniques, i.e., EDA, Back Translation, and Mixup, when using two general parameter efficient tuning methods, P-tuning v2 and LoRA, under data scarcity. We show that data augmentation can be used to boost the performance of P-tuning and LoRA models, but the effectiveness of each technique varies and certain methods can lead to a notable degradation in performance, particularly when using larger models and on harder tasks. We further analyze the sentence representations of P-tuning compared to fine-tuning to help understand the above behaviour, and reveal how P-tuning generally presents a more limited ability to separate the sentence embeddings from different classes of augmented data. In addition, it displays poorer performance on heavily altered data. However, we demonstrate that by adding a simple contrastive loss function it can help mitigate such issues for prefix tuning, resulting in sizable improvements to augmented data performance.

Downloads

Next from ACL 2023

CAISA at SemEval-2023 Task 8: Counterfactual Data Augmentation for Mitigating Class Imbalance in Causal Claim Identification

Similar lecture

SemEval-2023 Task 10: Explainable Detection of Online Sexism

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2023

CAISA at SemEval-2023 Task 8: Counterfactual Data Augmentation for Mitigating Class Imbalance in Causal Claim Identification

Similar lecture

SemEval-2023 Task 10: Explainable Detection of Online Sexism

Downloads