Brazil

BERT has revolutionized the NLP field by enabling transfer learning with large language models that can capture complex textual patterns, reaching the state-of-the-art for an expressive number of NLP applications. For text classification tasks, BERT has already been extensively explored. However, aspects like how to better cope with the different embeddings provided by the BERT output layer and the usage of language-specific instead of multilingual models are not well studied in the literature, especially for the Brazilian Portuguese language. The purpose of this article is to conduct an extensive experimental study regarding different strategies for aggregating the features produced in the BERT output layer, with a focus on the sentiment analysis task. The experiments include BERT models trained with Brazilian Portuguese corpora and the multilingual version, contemplating multiple aggregation strategies and open-source datasets with predefined training, validation, and test partitions to facilitate the reproducibility of the results. BERT achieved the highest ROC-AUC values for the majority of cases as compared to TF-IDF. Nonetheless, TF-IDF represents a good trade-off between the predictive performance and computational cost.

PROPOR 2022

BERT for Sentiment Analysis: Pre-trained and Fine-Tuned Alternatives

machine learning

sentiment analysis

natural language processing

transformers

transfer learning

**BRACIS** originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – **SBIA** (22 editions), and the Brazilian Symposium on Neural Networks – **SBRN** (13 editions). **ENIAC** is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. **STIL** is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: **CE-IA**, **CE-IC** and **CE-PLN**.

To access the 10th Brazilian Conference on Intelligent Systems (**BRACIS**), you need to register for the conference.  Please visit the Conference official site to register

[Register now](http://c4ai.inova.usp.br/bracis2021/)

Register now

BRACIS 2021

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – SBIA (22 editions), and the Brazilian Symposium on Neural Networks – SBRN (13 editions). 
ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. 
STIL is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: CE-IA, CE-IC and CE-PLN

poster

The International Conference on Computational Processing of Portuguese (PROPOR) is the main event in the area of human language processing that is focused on theoretical and technological issues of written and spoken Portuguese and Galician. The meeting has been a very rich forum for the exchange of ideas and partnerships for the research and industry communities dedicated to the automated processing of Portuguese, promoting the development of methodologies, resources and projects that can be shared among researchers and practitioners in the field.

Find more information and pay the registration fee on [event organizer’s website](https://sites.universidadedefortaleza.com/propor2022/?page_id=401)

The International Conference on Computational Processing of Portuguese (PROPOR) is the main event in the area of human language processing that is focused on theoretical and technological issues of written and spoken Portuguese and Galician.

We propose a methodology to populate the verb type hierarchy of a deep computational grammar in the HPSG formalism using syntactic and morphological information from Universal Dependencies (UD) treebanks. It is exemplified by means of the UD Bosque corpus and PorGram, a computational grammar for Portuguese constructed in the LinGO Grammar Matrix framework, but it can be applied to analogous grammars of other languages using other UD treebanks. The main component of the methodology is a Python module that extracts from the annotated sentences the core arguments and other features that are relevant to determine verb valence. This module enables the creation of a Python dictionary that maps valence frames to verb objects. This dictionary facilitates not only determining which frames occur with which verbs, but also detecting annotation errors. The potential of the module for rapid expansion of the lexical coverage of PorGram and corpus annotation error detection is illustrated with concrete examples.

Extracting valences from a dependency treebank for populating the verb lexicon of a Portuguese HPSG grammar

Fake news has become a research topic of great importance in Natural Language Processing due to its negative impact on our society. Although its pertinence, there are few datasets available in Brazilian Portuguese and mostly comprise few samples. Therefore, this paper proposes creating a new fake news dataset named FakeRecogna that contains a greater number of samples, more up-to-date news, and covering a few of the most important categories. We perform a toy evaluation over the created dataset using traditional classifiers such as Naive Bayes, Optimum-Path Forest, and Support Vector Machines. A Convolutional Neural Network is also evaluated in the context of fake news detection in the proposed dataset.

A New Brazilian Corpus for Fake News Detection

Most misinformation corpora are composed of explicitly real and fake news content. This results from the idea that misinformation can be approached as a binary classification problem. However, such approach oversimplifies the diversity of properties usually associated with credibility of different textual genres and types. To address this problem, we created MINT, a comprehensive annotated corpus of online articles collected from mainstream and independent Portuguese media sources, over a full year period. The collected articles include: hard news, opinion articles, soft news, satirical news, and conspiracy theories. This paper describes the main linguistic properties underlying each category, and provides some insights based on the analysis of an annotation initiative performed by online readers. The results show that (i) conspiracy theories and opinion articles present similar levels of subjectivity, and make use of fallacious arguments; (ii) irony and sarcasm are not only prevalent in satirical news, but also in conspiracy and opinion articles; and (iii) hard news differ from soft news by resorting to more sources of information, and presenting a higher degree of objectivity.

Mainstream and Independent News Text Corpus

Wepresent a new resource for discourse studies in Portuguese, the CRPC Discourse Bank (CRPC-DB). CRPC-DB follows the Penn Discourse Treebank style of annotation. The annotation is performed on the PAROLE corpus, a free subset of the Reference Corpus of Con temporary Portuguese (CRPC) that includes news, fiction and didactic/scientific texts. The discourse bank covers explicit and implicit relations at intra and inter-sentential levels, and includes for now a total of 14,436 discourse relations. We present the main guidelines of our annotation and discuss specific cases. An experiment in inter-annotator agreement was performed and holds results of 0.88 F1-score for discourse relation identification, 0.71 Cohen’s K for the classification of discourse relation types, and 0,75 for top-level sense classification. The CRPC-DB will be distributed free of charge through the PORTULAN CLARIN infrastructure.

]CRPC-DB A Discourse Bank for Portuguese

Although very useful, aspect-based sentiment analysis shows several challenges. The occurrence of implicit aspects is one of them. To improve our understanding of implicit aspects, this work analyzes their composition and categorizes the implicit aspect clues found in two corpora of opinionated texts of different domains in the Portuguese language. As results of this work, in addition to the implicit aspect typology, we present four lexicons of implicit aspect clues for different domains and the annotation of implicit aspect clues in an existing corpus.

Implicit opinion aspect clues in Portuguese texts: analysis and categorization

Text classification is an essential task in Natural Language Processing. The researchers and developers need data in the desired language to build new models and algorithms to develop this task. In this presentation, I discuss and make available three datasets of text classification in Portuguese based on data of the B2W group, which is an important contribution to the research in the field.

Should I buy or should I pass: E-Commerce datasets in Portuguese

We report the construction of a web-based Galician corpus and its language model, both made publicly available,by making use of CCNet tools and data. An in-depth analysis of thecorpus is made so as to provide insights on how to achieve optimumquality through the use of heuristics to lower the perplexity.

Revisiting CCNet for Quality Measurements in Galician

We introduce the problem of identifying characters in literary text, and mention some specific issues that are special for Portuguese, in the context of presenting DIP, a shared task to foster work in the area and produce resources for computational literature studies in Portuguese. We describe how the task is organized, the resources that will be created, and how we plan to evaluate the results produced by the participant systems.

Identifying literary characters in Portuguese: Challenges of an international shared task

The lack of a pattern widely accepted in Corpus Linguistics for the construction of text corpora has resulted in multiple possible paths to compilation, resulting in research products that are difficult to interface and have low continuity. The varie-ty of existing corpora aggravates this problem since individual studies use differ-ent forms of expression to describe and solve similar problems. Thus, a gap is identified in the production initiatives to create corpora regarding the “explorato-ry” way of conducting such initiatives. Based on the Lapelinc Workflow for the construction of historical corpora, the Lapelinc Framework presents a working pattern for construction and research activities involving Portuguese language corpora, thereby establishing a set of steps and by-products common to corpora-based linguistic research. Providing a set of tools in the Import Stage with a re-duced learning curve and allowing the convergence of efforts during develop-ment, the Lapelinc Framework tools add functionalities for multiple purposes in the construction of corpora, simplifying the construction of multi-type corpora, facilitated by the resources it makes available.

The systematic construction of multiple types of corpora through the Lapelinc Framework

Deep learning techniques have been shown to be efficient in various tasks, especially in the development of speech recognition systems, that is, systems that aim to transcribe an audio sentence in a sequence of written words. Despite the progress in the area, speech recognition can still be considered difficult, especially for languages lacking available data, such as Brazilian Portuguese (BP). In this sense, this work presents the development of an public Automatic Speech Recognition (ASR) system using only open available audio data, from the fine-tuning of the Wav2vec 2.0 XLSR-53 model pre-trained in many languages, over BP data. The final model presents an average word error rate of 12.4% over 7 different datasets (10.5% when applying a language model). According to our knowledge, the obtained error is the lowest among open end-to-end (E2E) ASR models for BP.

Premium content

BERT for Sentiment Analysis: Pre-trained and Fine-Tuned Alternatives

Downloads

Next from PROPOR 2022

Extracting valences from a dependency treebank for populating the verb lexicon of a Portuguese HPSG grammar

Similar lecture

Iterative machine learning applied to annotation of text datasets