Brazil

The translation of natural language questions to SQL queries has attracted growing attention, in particular in connection with transformers and similar language models. A large number of techniques are geared towards the English language; in this work, we thus investigated translation to SQL when input questions are given in the Portuguese language. To do so, we properly adapted state-of-the-art tools and resources. We changed the RAT-SQL+GAP system by relying on a multilingual BART model (we report tests with other language models), and we produced a translated version of the Spider dataset. Our experiments expose interesting phenomena that arise when non-English languages are targeted; in particular, it is better to train with original and translated training datasets together, even if a single target language is desired. This multilingual BART model fine-tuned with a double-size training dataset (English and Portuguese) achieved 83\% of the baseline, making inferences for the Portuguese test dataset. This investigation can help other researchers to produce results in Machine Learning in a language different from English. Our multilingual ready version of RAT-SQL+GAP and the data are available, open-sourced as mRAT-SQL+GAP at: https://github.com/C4AI/gap-text2sql

BRACIS 2021

mRAT-SQL+GAP: A Portuguese Text-to-SQL Transformer

spider dataset

nl2sql

deep learning

**BRACIS** originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – **SBIA** (22 editions), and the Brazilian Symposium on Neural Networks – **SBRN** (13 editions). **ENIAC** is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. **STIL** is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: **CE-IA**, **CE-IC** and **CE-PLN**.

To access the 10th Brazilian Conference on Intelligent Systems (**BRACIS**), you need to register for the conference.  Please visit the Conference official site to register

[Register now](http://c4ai.inova.usp.br/bracis2021/)

Register now

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – SBIA (22 editions), and the Brazilian Symposium on Neural Networks – SBRN (13 editions). 
ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. 
STIL is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: CE-IA, CE-IC and CE-PLN

technical paper

Collaborative filtering recommender systems are essential tools in many modern applications. Their main advantage compared with the alternatives is that they require only a matrix of user-item interactions to recommend a subset of relevant items for a given user. However, the increasing volume of the data consumed by these systems may lead to a representation model with very high sparsity and dimensionality. Several approaches to overcome this problem have been proposed, neural embeddings being one of the most recent. Since then, many recommender systems were made using this representation model, but few consumed temporal information during the learning phase. This study shows how to adapt a pioneering method of item embeddings by adding a sliding window over time, in conjunction with a split in the user's interaction history. Results indicate that considering temporal information when learning neural embeddings for items can significantly improve the quality of the recommendations.

Time-Dependent Item Embeddings for Collaborative Filtering

In software development, code autocomplete can be an essential tool in order to accelerate coding. However, many of these tools built into the IDEs are limited to suggesting only methods or arguments, often presenting to the user long lists of irrelevant items. Since innovations introduced by transformer-based models that have reached the state of the art performance in tasks involving natural language processing (NLP), the application of these models also in tasks involving code intelligence, such as code completion, has become a frequent object of study in recent years. In this paper, we present a transformer-based model trained on 1.2 million Java files gathered from top-starred Github repositories. Our evaluation approach was based on measuring the model's ability to predict the completion of a line, proposing a new metric to measure the applicability of the suggestions that we consider better adapted to the practical reality of the code completion task. With a recently developed Java web project as a test set, our experiments showed that in 55.9% of the test cases the model brought at least one suggestion applicable, while the best baseline model presented this in 26.5%.

Code autocomplete using Transformers

Time series classification has a wide variety of possible applications, like outlines of figure types, signs of movement and sensor signals. This diversity may present different results with the application of machine learning techniques. Among the different ways to classify time series, two, in particular, are explored in this paper: the shapelet primitive and the neural network classification. A shapelet is a sub-sequence of the time series symbolic, like a high-level descriptor, that are representative for the class to which they belong. These features act as common knowledge used by domain experts. This paper proposes a CNN training protocol to achieve better results in the classification of time series.
The idea consists of decomposing the original time series into shapelets and noise. Then, the shapelets are used to train a classifier while the no-shapelets (“signal noise”) is used to train another classifier.
The original time series is then used to train two final classifiers starting with the weights of shapelet and no-shapelet classifiers. This previous extraction of this representation can improve classification ability in a convolutional neural network using transfer learning. The experimental evaluation shows that the pre-selection of shapelets before the network training changes the classifier results for several databases and, consequently, improves classification accuracy.

Transfer Learning of Shapelets for Time Series Classification Using Convolutional Neural Network

Semantic textual similarity (STS) measures how semantically similar two sentences are. In the context of the Portuguese language, STS literature is still incipient but includes important initiatives like the ASSIN and ASSIN 2 shared tasks. The state-of-the-art
for those datasets is a contextual embedding produced by a Portuguese
pre-trained and fine-tuned BERT model. In this work, we investigate
the application of Sentence-BERT (SBERT) contextual embeddings to
these datasets. Compared to BERT, SBERT is a more computationally efficient approach, enabling its application to scalable unsupervised
learning problems. Given the absence of SBERT models pre-trained in
Portuguese and the computational cost for such training, we adopt multilingual models and also fine-tune them for Portuguese. Results showed
that SBERT embeddings were competitive especially after fine-tuning,
numerically surpassing the results of BERT on ASSIN 2 and the results
observed during the shared tasks for all datasets considered.

Comparing contextual embeddings for semantic textual similarity in Portuguese

In this paper, we present an efficient method for training models for speaker recognition using small or under-resourced datasets. This method requires less data than other SOTA (State-Of-The-Art) methods, e.g. the Angular Prototypical and GE2E loss functions, while achieving similar results to those methods. This is done using the knowledge of the reconstruction of a phoneme in the speaker's voice. For this purpose, a new dataset was built, composed of 40 male speakers, who read sentences in Portuguese, totaling approximately 3h. We compare the three best architectures trained using our method to select the best one, which is the one with a shallow architecture. Then, we compared this model with the SOTA method for the speaker recognition task: the Fast ResNet--34 trained with approximately 2,000 hours, using the loss functions Angular Prototypical and GE2E. Three experiments were carried out with datasets in different languages. Among these three experiments, our model achieved the second best result in two experiments and the best result in one of them. This highlights the importance of our method, which proved to be a great competitor to SOTA speaker recognition models, with 500x less data and a simpler approach.

Speech2Phone: A Novel and Efficient Method for Training Speaker Recognition Models

Event Extraction (EE) is the task of identifying mentions of
particular event types and their arguments in text, and it constitutes an
important and challenging task within the area of Information Extraction
(IE). However, in the context of the Portuguese language, very little work
has been conducted on this topic. In this paper, we propose a neural-
based method for EE, as well as a data resource to mitigate this research
gap. We also present a data augmentation strategy for EE, employing
an Open Information Extraction (OIE) system, aiming to overcome the
shortage in annotated data for the problem in the Portuguese language.
Our experimental results show that our method is able to predict event
types and arguments automatically, and the proposed method of data
augmentation, in one of the two evaluated samples, contributes to the
performance of the tested models in the subtask of argument role pre-
diction. Further, an implementation of our method is available to the
community, as the models trained in our experiments.

Joint Event Extraction with Contextualized Word Embeddings for the Portuguese Language

Automatic Text Simplification (ATS) has played a significant role in the Natural Language Processing (NLP) field.
ATS is a sequence-to-sequence problem aiming to create a new version of the original text removing complex and domain-specific words. 
It can improve communication and understanding of documents from specific domains, as well as support second language learning. 
This paper presents an empirical study on the use of state-of-the-art ATS methods to simplify texts in Portuguese. It is important to remark that the literature reports the challenge in analyzing Portuguese texts due to the lack of resources compared to other languages (i.e., English).
More specifically, this work evaluated different Neural Machine Translation (NMT) techniques for ATS in Portuguese. The experiments showed that NMT achieved promising results in Portuguese texts, obtaining 40.89 BLEU score using multiple parallel corpora and raising the overall readability score by more than 5 points.

Portuguese Neural Text Simplification using Machine Translation

This paper presents a technique that employs linguistic resources to refine PoS tagging using the Universal Dependencies (UD) model.
The technique is based on the development and use of lists of non-ambiguous single tokens and non-ambiguous co-occuring tokens in Portuguese (regardless of whether they constitute multiword expressions or not). These lists are meant to automatilly correct the tags for such tokens after tagging. The technique is applied over the output of two well-known state of the art systems - UDPipe and UDify - and the results for a real data set have shown a significant improvement of annotation accuracy. Overall, we improve tagging accuracy by up to 1.4%, and, in terms of the number of fully correct tagged sentences, our technique produces results that are 13.9% more accurate than the corresponding original system.

Universal Dependencies-based PoS Tagging Refinement through Linguistic Resources

Forced phonetic alignment (FPA) is the task of associating a given phonetic unit to a timestamp interval in the speech waveform. Phoneticians are able mark the boundaries with precision, but as the corpus grows it becomes infeasible to do it by hand. For Brazilian Portuguese (BP) in particular, only three tools appear to perform FPA: EasyAlign, Montreal Forced Aligner (MFA), and UFPAlign. Therefore, this work aims to develop resources based on Kaldi toolkit for UFPAlign, including their release alongside all scripts under open licenses; and to bring forth a comparison to the other two aforementioned aligners. Evaluation took place in terms of the phone boundary metric over a dataset of 385 hand-aligned utterances, and results show that Kaldi-based aligners perform better overall, and that UFPAlign models are more accurate than MFA's. Furthermore, complex deep-learning-based approaches did not seem to improve performance compared to simpler models.

Experiments on Kaldi-based Forced Phonetic Alignment for Brazilian Portuguese

The challenge of climate change and biome conservation is one of the most 
pressing issues of our time -- particularly in Brazil, where key environmental reserves are located. Given the availability of large textual databases on ecological themes, it is natural to resort to question answering (QA) systems to increase social awareness and understanding about these topics. In this work, we introduce multiple QA systems that combine in novel ways the BM25 algorithm, a sparse retrieval technique, with PTT5, a pre-trained state-of-the-art language model. Our QA systems focus on the Portuguese language, thus offering resources not found elsewhere in the literature. As training data, we collected questions from open-domain datasets, as well as content from the Portuguese Wikipedia and news from the press. We thus contribute with innovative architectures and novel applications, attaining an F1-score of 36.2 with our best model.

Premium content

mRAT-SQL+GAP: A Portuguese Text-to-SQL Transformer

Downloads

Next from BRACIS 2021

Time-Dependent Item Embeddings for Collaborative Filtering

Similar lecture

Code autocomplete using Transformers