Brazil

Deep language models, like ELMo, BERT and GPT, have achieved impressive results on several natural language tasks. These models are pretrained on large corpora of unlabeled general domain text and later supervisedly trained on downstream tasks. An optional step consists of finetuning the language model on a large intradomain corpus of unlabeled text, before training it on the final task. This aspect is not well explored in the current literature. In this work, we investigate the impact of this step on named entity recognition (NER) for Portuguese legal documents. We explore different scenarios considering two deep language architectures (ELMo and BERT), four unlabeled corpora and three legal
NER tasks for the Portuguese language. Experimental findings show a significant improvement on performance due to language model finetuning on intradomain text. We also evaluate the finetuned models on two general-domain NER tasks, in order to understand whether the aforementioned improvements were really due to domain similarity or simply due to more training data. The achieved results also indicate that finetuning on a legal domain corpus hurts performance on the general-domain NER tasks. Additionally, our BERT model, finetuned on a legal corpus, significantly improves on the state-of-the-art performance on the LeNER-Br corpus, a Portuguese language NER corpus for the legal domain.

BRACIS 2020

A Study on the Impact of Intradomain Finetuning of Deep Language Models for Legal Named Entity Recognition in Portuguese

natural language processing

named entity recognition

deep learning

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

NAACL 2022

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

technical paper

The 9th Brazilian Conference on Intelligent Systems (BRACIS), the 16h Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), and the 12th Concurso de Teses e Dissertações em Inteligência Artificial e Computacional (CTDIAC) was held from 20 to 23 October, 2020.

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence - SBIA (22 editions), and the Brazilian Symposium on Neural Networks - SBRN (13 editions). ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. CTDIAC is a satellite event whose goals are to disseminate and award the best dissertations and theses (MSc and PhDs concluded in Brazilian universities from June 2018 to May 2020) in Artificial and Computational Intelligence areas.

In this year, the Symposium on Knowledge Discovery, Mining and Learning (KDMiLe) will be held in conjunction with BRACIS 2020. This sysposium aims at integrating researchers, practitioners, developers, students and users to present their research results, to discuss ideas, and to exchange techniques, tools, and practical experiences – related to the Data Mining and Machine Learning areas.

The development of treatments based on the patient’s individual characteristics has been an emergent medical approach. The objective is to improve individual responses and overall survival. Thus, there is a need for computational tools able to identify and describe subgroups of patients for which the survival response significantly differs from the overall behaviour. However, there are few algorithms that address this matter. The majority of works of literature aim at building predictive models rather than understanding the characteristics that delineate subgroups with unusual survival. The approaches that provide understanding on factors that interfere in the survival behaviour usually resort to the stratification of the data based on previously known variable’s interactions, lacking the ability to shed light into new, possibly unknown, interactions. In contrast to the existent predictive approaches, we propose the use of supervised descriptive pattern mining in order to discover local patterns able to describe subsets of patients that present unusual survival behaviour. In this presentation, we present the ESM-AM (Exceptional Survival Model Ant Miner) algorithm, an Exceptional Model Mining approach to the discovery of subgroups with exceptional survival functions that explores the use of ant-colony optimization as search heuristic for the pattern mining task.

Exceptional Survival Model Mining

The most recent successful time series prediction models are
a combination of three elements: traditional stochastic models, machine
learning models and signal processing techniques. CEEMDAN-LSTM
models have combined empirical mode decomposition and long shortterm memory neural networks to achieve state-of-the-art results for financial data. In this work, we propose a generalized CEEMDAN-LSTM
architecture for time series forecasting capable of dealing with exogenous features as input, and the consequences of input data growth, such
as convergence difficulties. Our model was applied to time series from
10 of the most liquid Brazilian stocks, and results show that accuracy
is overall improved when compared to the original single feature input
CEEMDAN-LSTM architecture.

Financial time series forecasting via CEEMDAN-LSTM with exogenous features

Data Augmentation methods - a family of techniques designed
for synthetic generation of training data - have shown remarkable
results in various Deep Learning and Machine Learning tasks. Despite its
widespread and successful adoption within the computer vision community,
data augmentation techniques designed for natural language processing
(NLP) tasks have exhibited much slower advances and limited
success in achieving performance gains. As a consequence, with the exception
of applications of back-translation to machine translation tasks,
these techniques have not been as thoroughly explored by the wider
NLP community. Recent research on the subject also still lacks a proper
practical understanding of the relationship between data augmentation
and several important aspects of model design, such as hyperparameters
and regularization parameters. In this paper, we perform a comprehensive
study of NLP data augmentation techniques, comparing their relative
performance under different settings. We also propose Deep Back-
Translation, a novel NLP data augmentation technique and apply it to
benchmark datasets. We analyze the quality of the synthetic data generated,
evaluate its performance gains and compare all of these aspects
to previous existing data augmentation procedures.

DeepBT and NLP Data Augmentation Techniques: a new proposal and a comprehensive study

Suicidal ideation is one of the main predictors of the risk of suicide attempt and can be described as thoughts, ideas, planning, and desire to commit suicide. Fast detection of such ideation in early stages is essential for effective treatment. Many expressions of suicidal ideation can be found in publications in social networks, especially by young people. Previous works explore the automatic detection of suicidal ideation in social networks for the English language using machine learning algorithms. In this work, we present the first exploration of machine learning algorithms for suicidal ideation detection for the Portuguese language. We compared three classifiers in Twitter data: SVM, LSTM, and BERT (multilingual and Portuguese). Results suggest that BERT is effective for suicidal ideation identification in Portuguese data, achieving 79% of F1 score and less than 9% false negative score.

Machine learning for suicidal ideation identification on Twitter for the Portuguese language

The Winograd Challenge has been advocated as a test of computer understanding with respect to commonsense reasoning. The challenge is based on Winograd Schemas: sentences that contain correferential ambiguities. Most Winograd Schemas are relatively easy for human subjects, and today the best computer systems for the Winograd Challenge can work close to human performance. In this paper, we examine the assumptions behind the Winograd Challenge, and investigate how far we can push the difficulty level of Winograd Schemas, proposing various strategies to build really challenging schemas.

The Winograd Schemas from Hell

Evolutionary algorithms are a non-deterministic approach to solving optimization problems that cannot be solved in polynomial time, like the classic NP-Hard problems. The Permutation Flow Shop (PFS) is a combinatorial optimization problem related to the production environment. In the PFS, jobs must be processed by machines, maintaining the same processing flow on the considered machines. In this work, we adopt the multiobjective approach for the PFS, minimizing the makespan and total flowtime. NSGA-II with Tabu Search were considered in this hybrid approach and applied to 11 instances of PSF with different dimensions. We include an analysis on the use of prohibition rules in Tabu Search and its restrictiveness. The results were analyzed using the IGD and Empirical Attainment Functions metrics and compared with the canonical NSGA-II.

Hybrid Algorithm for the Multi-objective Permutation Flow Shop Problem

This paper proposes a temporal analysis for music metadata using a generative probabilistic model for collections of discrete dataset such as text corpora. This method is also a topic model that is used for discovering abstract topics from a collection of documents. The method is then applied to audio metadata and song lyrics extracted with Echo Nest engine, Spotify and Lyrics Genius API. Song data time series are generated by grouping data items by release date, genre and dominant topics (from LDA analysis). Using a technique from Network Theory we visualise how these topics, in this case genres, are related with each other through time.

Temporal analysis and visualisation of music metadata and topic modelling of song lyrics

O objetivo da presente pesquisa consiste em verificar a eficácia das armas de fogo, fuzil e pistola, em eventos de tiroteios em massa em ambiente fechado e quais os parâmetros que influenciam no número de vítimas, a fim de apoiar a análise e elaboração de políticas públicas. Para isso, realizou-se a Simulação Multiagente através do software NetLogo. Os resultados mostram que o tipo de arma utilizada no evento tem grande influência no número de vítimas. Além disso, também observou-se que a diferença no número de vítimas das duas armas diminui à medida em que se aumenta a quantidade de munição e se diminui a dimensão da porta de saída, devido ao efeito funil.

Multiagent Simulation to Support Disarmament Policies

The increase in transmission rates required for new Internet-related services is driving further studies on network optimization. Optimization algorithms with local search have shown promise in an attempt to design networks that satisfy different objectives such as: cost, blocking probability, energy expenditure and robustness indicators. This work proposes a new strategy for selecting individuals for local search, aiming to improve Pareto's quality in terms of hypervolume, and also to obtain efficiency gains. The proposed strategy was compared with a base method, in a simulated scenario of a backbone network for the state of Pernambuco. According to the results, the proposed strategy was able to make the algorithm more effective, improving its performance in all objectives, and more efficient; surpassing the base method in both aspects studied.

A Novel Strategy for Selecting Individuals in Manyobjective Algorithms with Local Search Applied to the Network Design Problem

This work presents the development of an explainable image classifier,
trained for the task of determining whether a cell is infected by malaria. The
classifier consists of a residual neural network, with a classification accuracy
of 96%, trained on the National Institute of Health Malaria Dataset. Explainable
Artificial Intelligence (XAI) techniques are employed to make classifications
more interpretable. These explanations are generated by using two different
methodologies: Local Interpretable Model Agnostic Explanations (LIME) and
SquareGrid. The explanations provide novel, key insights into the decision making
patterns of such a high performing model for a medical task.

A Study on the Impact of Intradomain Finetuning of Deep Language Models for Legal Named Entity Recognition in Portuguese

Downloads

Next from BRACIS 2020

Exceptional Survival Model Mining

Similar lecture

silpa_nlp at SemEval-2022 Tasks 11: Transformer based NER models for Hindi and Bangla languages

A Study on the Impact of Intradomain Finetuning of Deep Language Models for Legal Named Entity Recognition in Portuguese

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from BRACIS 2020

Exceptional Survival Model Mining

Similar lecture

silpa_nlp at SemEval-2022 Tasks 11: Transformer based NER models for Hindi and Bangla languages

Downloads