Brazil

We present PetroBERT, a BERT model adapted to the oil and gas domain.

PROPOR 2022

PetroBERT: a domain adaptation language model for oil and gas applications in Portuguese

neural networks

text classification

bert

transformer

# Welcome everyone to ACL 2022!

The 60th Annual Meeting of the Association for Computational Linguistics is taking place May 22-27, 2022 as a hybrid event, in Dublin and online. We are happy to welcome all of you to this anniversary edition with an almost 50-50 in-person and virtual participation. 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, three amazing keynote events and two new initiatives of invited talks: Spotlight Talks for Young Rising Stars (STIRS) and The Next Big Idea Talks. Posters (including Findings of ACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop will have an oral session and a poster session as part of Poster Session 1. The program also features eight Tutorials and 28 Workshops. 

 
We wish you a wonderful conference! 
[**The ACL 2022 Organizing Committee**](https://www.2022.aclweb.org/organisers)
 
[**Conference Handbook**](https://drive.google.com/file/d/1_BUCMfhMVrjG9E2e71aHdHeE28KSje0l/view?usp=sharing) 
[**Mini Handbook**](https://drive.google.com/file/d/1qlBKl0wzmlVF1oCeMQl3BahLd9nLP5Ce/view?usp=sharing) 
[**Posters and Demo guides**](https://drive.google.com/file/d/1UucMAoCNncIOaH1rMMDa0owuG9qgvJTG/view?usp=sharing)

ACL 2022

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing (NLP). 

technical paper

The International Conference on Computational Processing of Portuguese (PROPOR) is the main event in the area of human language processing that is focused on theoretical and technological issues of written and spoken Portuguese and Galician. The meeting has been a very rich forum for the exchange of ideas and partnerships for the research and industry communities dedicated to the automated processing of Portuguese, promoting the development of methodologies, resources and projects that can be shared among researchers and practitioners in the field.

Find more information and pay the registration fee on [event organizer’s website](https://sites.universidadedefortaleza.com/propor2022/?page_id=401)

The International Conference on Computational Processing of Portuguese (PROPOR) is the main event in the area of human language processing that is focused on theoretical and technological issues of written and spoken Portuguese and Galician.

Artificial Intelligence applied to Law is getting attention in the community, both from the Judiciary and the lawyers, due to possible gains in procedural celerity and the automation of repetitive tasks, among other use cases. Many of its bene-fits can be derived from Natural Language Processing since legal proceedings are textual document-based. Named Entity Recognition is the NLP task of identifying and classifying named entities in unstructured text to help achieve these goals. In recent years, transformers emerged as the new state-of-the-art architecture for some tasks in NLP systems. NLP resources in Portuguese, such as models and datasets, are needed to allow Portuguese speaking countries to benefit from this new NLP level. This paper presents the first fine-tuned BERT models trained ex-clusively on Brazilian Portuguese for Legal NER. The models achieved new state-of-the-art on LeNER-Br dataset, a Portuguese NER corpus for the legal domain. We also built a prototype application for Judiciary users to evaluate how the model performed with authentic law documents. The results showed that the models were able to extract information with good quality. Both the models and the prototype application are publicly available for the community.

Fostering Judiciary applications with new fine-tuned models for Legal Named Entity Recognition in Portuguese

Most approaches to role-filler entity extraction (REE) rely on large labeled training corpora in which entity mentions are directly annotated in the input document. In this work, we leverage an existing knowledge base (KB) of entities to perform document-level REE from drug seizure petitions. We propose a system that learns to extract entities from petitions to fill 29 roles of a drug seizure event. Although we have access to a KB covering more than 170 thousand entities and six thousand petitions, such that each entity in the KB is linked to a specific petition, the mentions to an entity within a petition's text are not annotated. The lack of these annotations brings challenges related to mismatches between entity values in the KB and entity mentions in the documents. Additionally, there are entities with same type or same value. Thus, we propose a distant annotation method to overcome these challenges and automatically label petition documents using the available KB. This annotation method includes a parameter that controls the balance between precision and recall. We also propose a strategy to effectively tune this parameter in order to optimize a given metric. We then train a BERT-based sequence labeling model that learns to identify entity mentions and label them. Our system achieves an F1-score of 78.59 with precision over 82%. We also report ablation studies regarding the distant annotation method.

Entity Extraction from Portuguese Legal Documents Using Distant Supervision

We present some results of applying natural language processing (NLP) techniques in the domain of History, having as object of investigation the Brazilian Historical-Biographical Dictionary (Dicionário Histórico-Biográfico Brasileiro, DHBB). After improving or adding annotation of specific fields, information extraction techniques based on manually derived patterns were applied to three relevant problems: the age of entrance in Brazilian politics, the academic background of Brazilian politicians, and family ties among the political elites.

Automatic information extraction: a distant reading of the Brazilian Historical-Biographical Dictionary

Products, services, among many other things in life have a quality standard, are inclusive, or do not harm customers. Regulations required from their manufacturers or providers make it possible. This type of requirement also exists in the finance sector. Governments, international agencies, or civil institutions are responsible for creating, applying, and inspecting these regulations. Regulators from all spheres (federal, state, and municipal) constantly demand changes in the finance sector to meet current needs adequately. This paper presents the constant evolution of a banking compliance application in Brazil. It aims to classify the relevance or irrelevance of regulatory documents published by more than 100 Brazilian regulators, affecting the businesses of more than 40 departments of Banco do Brasil. The application uses a hybrid strategy, combining machine learning and rules for a binary classification challenge involving each company department. This work also presents a particular type of corpus imbalance called The Imbalance Within Class.

Banking Regulation Classification in Portuguese

Deep learning transformers have drastically improved systems that automatically answer questions in natural language. However, different questions demand different answering techniques; here we propose, build and validate an architecture that integrates different modules to answer two distinct kinds of queries. Our architecture takes a free-form natural language text and classifies it to send it either to a Neural Question Answering Reasoner or a Natural Language parser to SQL. We implemented a complete system for the Portuguese language, using some of the main tools available for the language and translating training and testing datasets. Experiments show that our system selects the appropriate answering method with high accuracy (over 99%), thus validating a modular question answering strategy.


Integrating Question Answering and Text-to-SQL in Portuguese

Named entity recognition (NER) tasks imply token-level labels. Annotating documents can be time-consuming, costly, and prone
to human error. In many real-life scenarios, the lack of labeled data has
become the biggest bottleneck preventing NER being effectively used in
some domains and with some natural languages, with negative impacts
on the quality of some tasks. To overcome the barrier of the lack of annotated data for new application domains in some natural languages,
we propose a method that uses the output of an ensemble of NER’s to
automatically annotate the data needed to train a Bidirectional Encoder
Representations from Transformers (BERT) based NER for Portuguese.
The performance was assessed using MiniHAREM dataset with promising results. For domain relevant classes such as LOCAL, F1, Precision
and Recall above 50% were obtained when training only with automatically annotated data.

Named Entity Extractors for New Domains by Transfer Learning with Automatically Annotated Data

Open Information Extraction (OpenIE) is the task of extracting structured information from text. Recent advances in applying Deep Learning to OpenIE tasks have improved the state of the art for the task, although few works have been produced for languages other than English. In this work, we propose PortNOIE, a neural framework for open information extraction for the Portuguese language. We evaluate our method on a manually annotated corpus of Open IE extractions, obtaining better performance than the current state of the art for OpenIE for Portuguese, both based on rule-based approaches or neural methods.

PortNOIE: A neural framework for Open Information Extraction for the Portuguese language

Tax evasion is a problem that affects our society, costing billions of Brazilian reais of public funds every year. Stopping this practice is a complex challenge that involves analyzing a large and diverse volume of data. In this work, we propose an approach to analyze invoices and extract information about measures and units from product descriptions using a neural network with the BiLSTM-CRF architecture. Our method can validate product quantity information to, for instance, check whether any product was bought or sold by a business without issuing an invoice. The results were evaluated according to precision, recall, and f-score. The proposed approach can correctly detect more than 90% of cases of each type of information, showing its feasibility to process invoice data.

Automatic Recognition of Units of Measurement in Product Descriptions from Tax Invoices Using Neural Networks

Automatic book genre classification is a hard task as it requires the whole book's content or a high-quality summary, which is challenging to write automatically. On the other hand, online reviews are an accessible resource for readers to evaluate a book or even get a general sense about it, including its genre. As the amount of book reviews is always increasing, using such information to genre classification needs a robust solution to deal with high volumes of data. In such a context, we introduce a model for automatically classifying book genres by analyzing online text reviews. We build a dataset of compiled texts from online book reviews. Then, we use multiple machine learning algorithms to categorize a book into a specific genre. Such a process enables to compare algorithms and detect the best classifiers. Hence, the most efficient machine learning algorithm completed the task with an accuracy of 96%; i.e., the proposed model is convenient for various information retrieval systems due to its high certainty and efficiency.

Book Genre Classification Based on Reviews of Portuguese-language Literature

The amount of legislative documents produced within the past decade has risen dramatically, making it diﬃcult for law practitioners to consult and update legislation. Named Entity Recognition (NER) systems have the untapped potential to extract information from legal documents, which can improve information retrieval and decision-making processes. We introduce the UlyssesNER-Br, a corpus of Brazilian Legislative Documents for NER with quality baselines. The presented corpus consists of bills and legislative consultations from Brazilian Chamber of Deputies. We implemented Conditional Random Field (CRF) and Hidden Markov Model (HMM) models, and the promising F1-score of 80.8% in the analysis by categories and 81.04% in the analysis by types, was achieved with the CRF model. The entities with the best average F1-score results were “FUNDlei” and “DATA”, and the ones with the worst results were “EVENTO” and “PESSOAgrupoind”. The corpus was also evaluated using a BiLSTM-CRF and Glove architectures provided by the pioneering state-of-the-art paper, achieving F1-score of 76.89% in the analysis by categories and 59.67% in the analysis by types.

Premium content

PetroBERT: a domain adaptation language model for oil and gas applications in Portuguese

Downloads

Next from PROPOR 2022

Fostering Judiciary applications with new fine-tuned models for Legal Named Entity Recognition in Portuguese

Similar lecture

Early Stopping Based on Unlabeled Samples in Text Classification