Spain

We present our submitted systems for SemEval-2020 Task 8 titled &quot;Memotion Analysis&quot;. For this task we used a multimodal architecture to determine the emotion expressed in a meme. Our architecture utilizes both textual and visual information present in a meme. To extract image features we experimented with pre-trained VGG-16 and Inception-V3 classifiers and to extract text features we used LSTM and BERT classifiers. Both FastText and GloVe embeddings were experimented with for the LSTM classifier. The best F1 scores our classifier obtained on the official analysis results are 0.3309, 0.4752, and 0.2897 for Task A, B, and C respectively. In our study, we found that combining both textual and visual information expressed in a meme improves the performance of the classifier as opposed to using standalone classifiers that use only text or visual data.

COLING 2020

A Multimodal Approach to Detect Offensive, Sarcastic and Humorous Memes

inception-v3

memotion analysis

bilstm

vgg-16

offensive

memes

sarcasm

multimodal

bert

sentiment

stages 

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

We present our submitted systems for SemEval-2020 Task 8 titled "Memotion Analysis". For this task we used a multimodal architecture to determine the emotion expressed in a meme. Our architecture utilizes both textual and visual information present in a meme. To extract image features we experimented with pre-trained VGG-16 and Inception-V3 classifiers and to extract text features we used LSTM and BERT classifiers. Both FastText and GloVe embeddings were experimented with for the LSTM classifier. The best F1 scores our classifier obtained on the official analysis results are 0.3309, 0.4752, and 0.2897 for Task A, B, and C respectively. In our study, we found that combining both textual and visual information expressed in a meme improves the performance of the classifier as opposed to using standalone classifiers that use only text or visual data.

poster

In this paper, we describe an approach for modelling causal reasoning in natural language by detecting counterfactuals in text using multi-head self-attention weights. We use pre-trained transformer models to extract contextual embeddings and self-attention weights from the text. We show the use of convolutional layers to extract task-specific features from these self-attention weights. Further, we describe a fine-tuning approach with a common base model for knowledge sharing between the two closely related sub-tasks for counterfactual detection. We analyze and compare the performance of various transformer models in our experiments. Finally, we perform a qualitative analysis with the multi-head self-attention weights to interpret our models' dynamics.

Modelling Causal Reasoning in Language with Multi-Head Self-Attention Weights Based Counterfactual Detection

- Proposing a domain-tuned contextualized language model for
Offensive Language Detection task.
- Experimenting with different components and configurations (e.g.,
a multi-view SVM) stacked upon BERT models for specific sub-tasks.
- Achieving F1 scores of 91.7% in Sub-task A, 66.5% in Sub-task B,
and 63.2% in Sub-task C, ranking 5th, 6th, and 11th respectively.
- Performing error analysis to show common misclassification errors
made by our model and outlining research directions for future.

Domain-Tuned Contextualized Models for Offensive Language Detection

We describe the system submitted by the SWAGex team to the SemEval-2020 Commonsense Validation and Explanation Task. We used multiple methods on the pre-trained language model BERT (Devlin et al., 2018) for tasks that require the system to recognize sentences against commonsense and justify the reasoning behind this decision. Our best performing model is BERT trained on SWAG fine-tuned for the task. We investigate the ability to transfer commonsense knowledge from SWAG to SemEval-2020 by training a model for the Explanation task with Next Event Prediction data.

Commonsense Explanation as Next Event Prediction

This poster describes the NTUAAILS submission for SemEval 2020 Task 11 Detection of Propaganda Techniques in News Articles. Using deep learning techniques to detect and classify propaganda techniques, we can automatically provide the means to address propaganda and fake news.
In order to achieve this goal, two separate models were created using bidirectional LSTMs and pre-trained word embeddings like ELMo and GloVe. 
At first, the detection sub-task (Span identification) was addressed as Named Entity Recognition problem (NER). The entities consisted of two categories (token-level) propaganda and no-propaganda.
After the detection, the second sub-task's (Technique Classification) goal was to classify the given propaganda fragments in the 14 propaganda classes.
The models achieved rank 28 among 36 teams with F1 score of 0.335 and ranked 25 among 31 teams with 0.463 F1 scores for Span Identification and Technique Classification sub-tasks respectively. Our results indicate that the proposed deep learning models, although relatively simple in architecture and fast to train, achieve satisfactory results in the tasks on hand.

Propaganda Detection and Classification: BiLSTMs and ELMo approach

OffensEval 2: Multilingual Offensive Language Identification in Social Media is Task 12 of SemEval-2020 international workshop on Semantic Evaluation. Our team created a solution dedicated to this task by leveraging two pre-trained state-of-the-art models: BERT and XLNet. The models were fine-tuned for each subtask separately and features taken from their hidden layers were combined and fed into a fully connected neural network. The model using aggregated Transformer features can serve as a powerful tool for offensive language identification problem. Our team was ranked 7th out of 40 in Sub-task C - Offense target identification with 64.727% macro F1-score and 64th out of 85 in Sub-task A - Offensive language identification (89.726% F1-score).

Aggregation of Transformer-based models’ features for offensive language recognition

We consider detection of the span of antecedents and consequents in argumentative prose a structural, grammatical task. Our system comprises a set of stacked Bi-LSTMs trained on two complementary linguistic annotations. We explore the effectiveness of grammatical features (POS and clause type) through ablation. The reported experiments suggest that a multi-task learning approach using this external, grammatical knowledge is useful for detecting the extent of antecedents and consequents and performs nearly as well without the use of word embeddings.

Multi-task Stacked Bi-LSTMs

We present our systems to solve Task 3 of Semeval-2020, which aims to predict the effect that context has on human perception of similarity of words. The task consists of two subtasks in English, Croatian, Finnish and Slovenian: (1) predicting the change of similarity, and (2) predicting the human scores of similarity, both of them for a pair of words within two different contexts. We tackled the problem by developing two systems, the first one uses a centroid approach and word vectors. The second one uses the ELMo language model, which is trained for each pair of words with the given context. Our approach achieved the highest score in subtask 2 for the English language.

Predicting Contextual Word Similarity Using a Centroid based Approach and Word Embeddings

This poster describes our approach to "DeftEval: Extracting Definitions from Free Text in Textbooks" competition held as a part of SemEval 2020. The task was devoted to finding and labeling definitions in texts.

Multi-task learning for Definition Extraction

We propose hybrid models (HybridE and HybridW) for meme analysis (SemEval 2020 Task 8), which involves sentiment classification (Subtask A), humor classification (Subtask B), and scale of semantic classes (Subtask C). The hybrid model consists of BLSTM and CNN for text and image processing respectively. HybridE provides equal weight to BLSTM and CNN performance, while HybridW provides weightage based on the performance of BLSTM and CNN on a validation set. The performances (macro F1) of our hybrid model on Subtask A are 0.329 (HybridE),0.328 (HybridW), on Subtask B are 0.507 (HybridE), 0.512 (HybridW), and on Subtask C are0.309 (HybridE), 0.311 (HybridW).

You’re not the only one
cursed with knowledge –Multi branch model memotion analysis

In this paper, we assess the ability of BERT and its derivative models (RoBERTa, DistilBERT, and ALBERT) for short-edits-based humor grading. We test these models for humor grading and classification tasks on the Humicroedit and the FunLines dataset. We perform extensive experiments with these models to test their language modeling and generalization abilities via zero-shot inference and cross dataset inference-based approaches. Further, we also inspect the role of self-attention layers in humor-grading by performing a qualitative analysis over the self-attention weights from the final layer of the trained BERT model. Our experiments show that all the pre-trained BERT derivative models show significant generalization capabilities for humor-grading related tasks.

Downloads

Next from COLING 2020

Modelling Causal Reasoning in Language with Multi-Head Self-Attention Weights Based Counterfactual Detection

Similar lecture

Study of Multimodal Techniques for Memes Analysis

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from COLING 2020

Modelling Causal Reasoning in Language with Multi-Head Self-Attention Weights Based Counterfactual Detection

Similar lecture

Study of Multimodal Techniques for Memes Analysis

Downloads