Brazil

Data augmentation is a widely adopted method for improving model performance in image classification tasks. Although it still not as ubiquitous in Natural Language Processing (NLP) community, some methods have already been proposed to increase the amount of training data using simple text transformations or text generation through language models. However, recent text classification tasks need to deal with domains characterized by a small amount of text and informal writing, e.g., Online Social Networks content, reducing the capabilities of current methods. Facing these challenges by taking advantage of the pre-trained language models, low computational resource consumption, and model compression, we proposed the PRE-trained Data AugmenTOR (PREDATOR) method. Our data augmentation method is composed of two modules: the Generator, which synthesizes new samples grounded on a lightweight model, and the Filter, that selects only the high-quality ones. The experiments comparing Bidirectional Encoder Representations from Transformer (BERT), Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) and Multinomial Naive Bayes (NB) in three datasets exposed the effective improvement of accuracy. It was obtained 28.5% of accuracy improvement with LSTM on the best scenario and an average improvement of 8% across all scenarios. PREDATOR was able to augment real-world social media datasets and other domains, overcoming the recent text augmentation techniques.

BRACIS 2020

Pre-trained Data Augmentation for Text Classification

online social networks

text classification

data augmentation

[![](https://assets.underline.io/uploads/markdown_image/1/image/81a3c0317d24f663b49b996875024d45.png)](https://aclanthology.org/volumes/2022.coling-1/)



### THE CONFERENCE WE KNOW AND WE WANT.
**COLING**, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes but also in industrial research departments including tech-startups. COLING provides opportunities for all these communities to showcase their exciting discovery.

In fall of 2022, COLING will be held in Gyeongju in a hybrid format. All participants can either present at the venue site or join virtually. As more people get vaccinated, we are happy to provide safer environments for our colleagues. We believe that COLING 2022 will be one of the conferences, free from the pandemic. The hybrid format gives presenters and sponsors a valuable opportunity to promote their companies in both an online and in-person venue. For the first time in a long time, customers can interact with their sponsor's products first-hand. The online venue, too, gives sponsors the chance to network with those unable to attend the in-person session.

The hybrid format ultimately lends itself to greater exposure for our sponsors: COLING2022 will let you reach more potential partners and customers than ever before!<br><br><br>

To gain access to this event page you are required to register.<br> Find more information and pay the registration fee on event the organizer’s website **[https://coling2022.org/reg](https://coling2022.org/reg)**

COLING 2022

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

technical paper

The 9th Brazilian Conference on Intelligent Systems (BRACIS), the 16h Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), and the 12th Concurso de Teses e Dissertações em Inteligência Artificial e Computacional (CTDIAC) was held from 20 to 23 October, 2020.

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence - SBIA (22 editions), and the Brazilian Symposium on Neural Networks - SBRN (13 editions). ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. CTDIAC is a satellite event whose goals are to disseminate and award the best dissertations and theses (MSc and PhDs concluded in Brazilian universities from June 2018 to May 2020) in Artificial and Computational Intelligence areas.

In this year, the Symposium on Knowledge Discovery, Mining and Learning (KDMiLe) will be held in conjunction with BRACIS 2020. This sysposium aims at integrating researchers, practitioners, developers, students and users to present their research results, to discuss ideas, and to exchange techniques, tools, and practical experiences – related to the Data Mining and Machine Learning areas.

In Natural Language Processing, stance detection is the computational task of deciding whether a piece of text expresses a favourable or unfavourable attitude (or stance) towards a given topic. Stance detection may be divided into two subtasks: deciding whether a piece of text conveys any stance towards the target topic and, once we have established that the text does convey a stance, determining its polarity (e.g., favourable or unfavourable) towards the target. Both tasks - hereby called stance recognition and (stance) polarity classification - are the focus of the present work. Taking as a basis a corpus of 13.7k tweets in the Brazilian Portuguese language, and which conveys stances towards five moral issues (abortion legislation, death penalty, drug legalisation, lowering of criminal age, and racial quotas at universities), we compare a number of long short-term memory (LSTM) and bidirectional LSTM (BiLSTM) models for stance recognition and polarity classification. In doing so, the two tasks are addressed both independently and as a joint model. Results suggest that the use of BiLSTM models with attention mechanism outperform the alternatives under consideration, and pave the way for more comprehensive studies in this domain.

Twitter Moral Stance Classification using Long Short-Term Memory Networks

Deep language models, like ELMo, BERT and GPT, have achieved impressive results on several natural language tasks. These models are pretrained on large corpora of unlabeled general domain text and later supervisedly trained on downstream tasks. An optional step consists of finetuning the language model on a large intradomain corpus of unlabeled text, before training it on the final task. This aspect is not well explored in the current literature. In this work, we investigate the impact of this step on named entity recognition (NER) for Portuguese legal documents. We explore different scenarios considering two deep language architectures (ELMo and BERT), four unlabeled corpora and three legal
NER tasks for the Portuguese language. Experimental findings show a significant improvement on performance due to language model finetuning on intradomain text. We also evaluate the finetuned models on two general-domain NER tasks, in order to understand whether the aforementioned improvements were really due to domain similarity or simply due to more training data. The achieved results also indicate that finetuning on a legal domain corpus hurts performance on the general-domain NER tasks. Additionally, our BERT model, finetuned on a legal corpus, significantly improves on the state-of-the-art performance on the LeNER-Br corpus, a Portuguese language NER corpus for the legal domain.

A Study on the Impact of Intradomain Finetuning of Deep Language Models for Legal Named Entity Recognition in Portuguese

The development of treatments based on the patient’s individual characteristics has been an emergent medical approach. The objective is to improve individual responses and overall survival. Thus, there is a need for computational tools able to identify and describe subgroups of patients for which the survival response significantly differs from the overall behaviour. However, there are few algorithms that address this matter. The majority of works of literature aim at building predictive models rather than understanding the characteristics that delineate subgroups with unusual survival. The approaches that provide understanding on factors that interfere in the survival behaviour usually resort to the stratification of the data based on previously known variable’s interactions, lacking the ability to shed light into new, possibly unknown, interactions. In contrast to the existent predictive approaches, we propose the use of supervised descriptive pattern mining in order to discover local patterns able to describe subsets of patients that present unusual survival behaviour. In this presentation, we present the ESM-AM (Exceptional Survival Model Ant Miner) algorithm, an Exceptional Model Mining approach to the discovery of subgroups with exceptional survival functions that explores the use of ant-colony optimization as search heuristic for the pattern mining task.

Exceptional Survival Model Mining

The most recent successful time series prediction models are
a combination of three elements: traditional stochastic models, machine
learning models and signal processing techniques. CEEMDAN-LSTM
models have combined empirical mode decomposition and long shortterm memory neural networks to achieve state-of-the-art results for financial data. In this work, we propose a generalized CEEMDAN-LSTM
architecture for time series forecasting capable of dealing with exogenous features as input, and the consequences of input data growth, such
as convergence difficulties. Our model was applied to time series from
10 of the most liquid Brazilian stocks, and results show that accuracy
is overall improved when compared to the original single feature input
CEEMDAN-LSTM architecture.

Financial time series forecasting via CEEMDAN-LSTM with exogenous features

Data Augmentation methods - a family of techniques designed
for synthetic generation of training data - have shown remarkable
results in various Deep Learning and Machine Learning tasks. Despite its
widespread and successful adoption within the computer vision community,
data augmentation techniques designed for natural language processing
(NLP) tasks have exhibited much slower advances and limited
success in achieving performance gains. As a consequence, with the exception
of applications of back-translation to machine translation tasks,
these techniques have not been as thoroughly explored by the wider
NLP community. Recent research on the subject also still lacks a proper
practical understanding of the relationship between data augmentation
and several important aspects of model design, such as hyperparameters
and regularization parameters. In this paper, we perform a comprehensive
study of NLP data augmentation techniques, comparing their relative
performance under different settings. We also propose Deep Back-
Translation, a novel NLP data augmentation technique and apply it to
benchmark datasets. We analyze the quality of the synthetic data generated,
evaluate its performance gains and compare all of these aspects
to previous existing data augmentation procedures.

DeepBT and NLP Data Augmentation Techniques: a new proposal and a comprehensive study

Suicidal ideation is one of the main predictors of the risk of suicide attempt and can be described as thoughts, ideas, planning, and desire to commit suicide. Fast detection of such ideation in early stages is essential for effective treatment. Many expressions of suicidal ideation can be found in publications in social networks, especially by young people. Previous works explore the automatic detection of suicidal ideation in social networks for the English language using machine learning algorithms. In this work, we present the first exploration of machine learning algorithms for suicidal ideation detection for the Portuguese language. We compared three classifiers in Twitter data: SVM, LSTM, and BERT (multilingual and Portuguese). Results suggest that BERT is effective for suicidal ideation identification in Portuguese data, achieving 79% of F1 score and less than 9% false negative score.

Machine learning for suicidal ideation identification on Twitter for the Portuguese language

The Winograd Challenge has been advocated as a test of computer understanding with respect to commonsense reasoning. The challenge is based on Winograd Schemas: sentences that contain correferential ambiguities. Most Winograd Schemas are relatively easy for human subjects, and today the best computer systems for the Winograd Challenge can work close to human performance. In this paper, we examine the assumptions behind the Winograd Challenge, and investigate how far we can push the difficulty level of Winograd Schemas, proposing various strategies to build really challenging schemas.

The Winograd Schemas from Hell

Evolutionary algorithms are a non-deterministic approach to solving optimization problems that cannot be solved in polynomial time, like the classic NP-Hard problems. The Permutation Flow Shop (PFS) is a combinatorial optimization problem related to the production environment. In the PFS, jobs must be processed by machines, maintaining the same processing flow on the considered machines. In this work, we adopt the multiobjective approach for the PFS, minimizing the makespan and total flowtime. NSGA-II with Tabu Search were considered in this hybrid approach and applied to 11 instances of PSF with different dimensions. We include an analysis on the use of prohibition rules in Tabu Search and its restrictiveness. The results were analyzed using the IGD and Empirical Attainment Functions metrics and compared with the canonical NSGA-II.

Hybrid Algorithm for the Multi-objective Permutation Flow Shop Problem

This paper proposes a temporal analysis for music metadata using a generative probabilistic model for collections of discrete dataset such as text corpora. This method is also a topic model that is used for discovering abstract topics from a collection of documents. The method is then applied to audio metadata and song lyrics extracted with Echo Nest engine, Spotify and Lyrics Genius API. Song data time series are generated by grouping data items by release date, genre and dominant topics (from LDA analysis). Using a technique from Network Theory we visualise how these topics, in this case genres, are related with each other through time.

Temporal analysis and visualisation of music metadata and topic modelling of song lyrics

O objetivo da presente pesquisa consiste em verificar a eficácia das armas de fogo, fuzil e pistola, em eventos de tiroteios em massa em ambiente fechado e quais os parâmetros que influenciam no número de vítimas, a fim de apoiar a análise e elaboração de políticas públicas. Para isso, realizou-se a Simulação Multiagente através do software NetLogo. Os resultados mostram que o tipo de arma utilizada no evento tem grande influência no número de vítimas. Além disso, também observou-se que a diferença no número de vítimas das duas armas diminui à medida em que se aumenta a quantidade de munição e se diminui a dimensão da porta de saída, devido ao efeito funil.

Pre-trained Data Augmentation for Text Classification

Downloads

Next from BRACIS 2020

Twitter Moral Stance Classification using Long Short-Term Memory Networks

Similar lecture

Improved BERT-based classification models with Rdrop and PolyLoss

Pre-trained Data Augmentation for Text Classification

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from BRACIS 2020

Twitter Moral Stance Classification using Long Short-Term Memory Networks

Similar lecture

Improved BERT-based classification models with Rdrop and PolyLoss

Downloads