Spain

We describe the system submitted by the SWAGex team to the SemEval-2020 Commonsense Validation and Explanation Task. We used multiple methods on the pre-trained language model BERT (Devlin et al., 2018) for tasks that require the system to recognize sentences against commonsense and justify the reasoning behind this decision. Our best performing model is BERT trained on SWAG fine-tuned for the task. We investigate the ability to transfer commonsense knowledge from SWAG to SemEval-2020 by training a model for the Explanation task with Next Event Prediction data.

COLING 2020

Commonsense Explanation as Next Event Prediction

swag

explanation

commonsense reasoning

bert

stages 

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

poster

This poster describes the NTUAAILS submission for SemEval 2020 Task 11 Detection of Propaganda Techniques in News Articles. Using deep learning techniques to detect and classify propaganda techniques, we can automatically provide the means to address propaganda and fake news.
In order to achieve this goal, two separate models were created using bidirectional LSTMs and pre-trained word embeddings like ELMo and GloVe. 
At first, the detection sub-task (Span identification) was addressed as Named Entity Recognition problem (NER). The entities consisted of two categories (token-level) propaganda and no-propaganda.
After the detection, the second sub-task's (Technique Classification) goal was to classify the given propaganda fragments in the 14 propaganda classes.
The models achieved rank 28 among 36 teams with F1 score of 0.335 and ranked 25 among 31 teams with 0.463 F1 scores for Span Identification and Technique Classification sub-tasks respectively. Our results indicate that the proposed deep learning models, although relatively simple in architecture and fast to train, achieve satisfactory results in the tasks on hand.

Propaganda Detection and Classification: BiLSTMs and ELMo approach

OffensEval 2: Multilingual Offensive Language Identification in Social Media is Task 12 of SemEval-2020 international workshop on Semantic Evaluation. Our team created a solution dedicated to this task by leveraging two pre-trained state-of-the-art models: BERT and XLNet. The models were fine-tuned for each subtask separately and features taken from their hidden layers were combined and fed into a fully connected neural network. The model using aggregated Transformer features can serve as a powerful tool for offensive language identification problem. Our team was ranked 7th out of 40 in Sub-task C - Offense target identification with 64.727% macro F1-score and 64th out of 85 in Sub-task A - Offensive language identification (89.726% F1-score).

Aggregation of Transformer-based models’ features for offensive language recognition

We consider detection of the span of antecedents and consequents in argumentative prose a structural, grammatical task. Our system comprises a set of stacked Bi-LSTMs trained on two complementary linguistic annotations. We explore the effectiveness of grammatical features (POS and clause type) through ablation. The reported experiments suggest that a multi-task learning approach using this external, grammatical knowledge is useful for detecting the extent of antecedents and consequents and performs nearly as well without the use of word embeddings.

Multi-task Stacked Bi-LSTMs

We present our systems to solve Task 3 of Semeval-2020, which aims to predict the effect that context has on human perception of similarity of words. The task consists of two subtasks in English, Croatian, Finnish and Slovenian: (1) predicting the change of similarity, and (2) predicting the human scores of similarity, both of them for a pair of words within two different contexts. We tackled the problem by developing two systems, the first one uses a centroid approach and word vectors. The second one uses the ELMo language model, which is trained for each pair of words with the given context. Our approach achieved the highest score in subtask 2 for the English language.

Predicting Contextual Word Similarity Using a Centroid based Approach and Word Embeddings

This poster describes our approach to "DeftEval: Extracting Definitions from Free Text in Textbooks" competition held as a part of SemEval 2020. The task was devoted to finding and labeling definitions in texts.

Multi-task learning for Definition Extraction

We propose hybrid models (HybridE and HybridW) for meme analysis (SemEval 2020 Task 8), which involves sentiment classification (Subtask A), humor classification (Subtask B), and scale of semantic classes (Subtask C). The hybrid model consists of BLSTM and CNN for text and image processing respectively. HybridE provides equal weight to BLSTM and CNN performance, while HybridW provides weightage based on the performance of BLSTM and CNN on a validation set. The performances (macro F1) of our hybrid model on Subtask A are 0.329 (HybridE),0.328 (HybridW), on Subtask B are 0.507 (HybridE), 0.512 (HybridW), and on Subtask C are0.309 (HybridE), 0.311 (HybridW).

You’re not the only one
cursed with knowledge –Multi branch model memotion analysis

In this paper, we assess the ability of BERT and its derivative models (RoBERTa, DistilBERT, and ALBERT) for short-edits-based humor grading. We test these models for humor grading and classification tasks on the Humicroedit and the FunLines dataset. We perform extensive experiments with these models to test their language modeling and generalization abilities via zero-shot inference and cross dataset inference-based approaches. Further, we also inspect the role of self-attention layers in humor-grading by performing a qualitative analysis over the self-attention weights from the final layer of the trained BERT model. Our experiments show that all the pre-trained BERT derivative models show significant generalization capabilities for humor-grading related tasks.

Assessing the Ability of BERT and Derivative Models to Perform Short-Edits based Humor Grading

This work presents our contribution to the Offensive Language Classification Task (EnglishSubTask A) of Semeval 2020. We propose different Bert models trained on several offensive language classification and profanity datasets, and combine their output predictions in an ensemble model. We experimented with different ensemble approaches, such as SVMs, Gradient boosting,AdaBoosting and Logistic Regression. We further propose an under-sampling approach of thecurrent SOLID dataset, which removed the most uncertain partitions of the dataset, increasing therecall of the dataset. Our best model, an average ensemble of four different Bert models, achieved11th place out of 82 participants with a macro F1 score of 0.91344 in the English SubTask A.

Ensemble of offensive language classifiers

This paper describes our approach to the task of identifying offensive languages in a multilingual setting. We investigate two data augmentation strategies: using additional semi-supervised labels with different thresholds and cross-lingual transfer with data selection. Leveraging the semi-supervised dataset resulted in performance improvements compared to the baseline trained solely with the manually-annotated dataset. We propose a new metric, Translation Embedding Distance, to measure the transferability of instances for cross-lingual data selection. We also introduce various preprocessing steps tailored for social media text along with methods to fine-tune the pre-trained multilingual BERT (mBERT) for offensive language identification. Our multilingual systems achieved competitive results in Greek, Danish, and Turkish at OffensEval 2020.

Improving Offensive Language Detection with Cross-lingual Transfer

Sentiment analysis for code-mixed social media text continues to be an under-explored area. This work adds two common approaches: fine-tuning large transformer models and sample efficient methods like ULMFiT \cite{ulmfit}. Prior work demonstrates the efficacy of classical ML methods for polarity detection. Fine-tuned general-purpose language representation models, such as those of the BERT family are benchmarked along with classical machine learning and ensemble methods. We show that NB-SVM beats RoBERTa by 6.2F1. The best performing model is a majority-vote ensemble which achieves an F1 of 0.707.

Downloads

Next from COLING 2020

Propaganda Detection and Classification: BiLSTMs and ELMo approach

Similar lecture

Leveraging Heterogeneous Knowledge Resources for Commonsense Validation and Explanation

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from COLING 2020

Propaganda Detection and Classification: BiLSTMs and ELMo approach

Similar lecture

Leveraging Heterogeneous Knowledge Resources for Commonsense Validation and Explanation

Downloads