Brazil

Training deep neural networks is a relevant problem with open questions related to convergence and quality of learned representations. Gradient-based optimization methods are used in practice, but cases of failure and success are still to be investigated. In this context, we set out to better understand the convergence properties of different optimization strategies, under different parameter options. Our results show that (i) feature embeddings are impacted by different optimization settings, (ii) suboptimal results are achieved by the use of default parameters, (iii) significant improvement is obtained by making educated choices of parameters, (iv) learning rate decay should always be considered. Such findings offer guidelines for training and deployment of deep networks.

BRACIS 2021

Optimization Matters: Guidelines to Improve Representation Learning with Deep Networks

deep networks

optimization

**BRACIS** originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – **SBIA** (22 editions), and the Brazilian Symposium on Neural Networks – **SBRN** (13 editions). **ENIAC** is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. **STIL** is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: **CE-IA**, **CE-IC** and **CE-PLN**.

To access the 10th Brazilian Conference on Intelligent Systems (**BRACIS**), you need to register for the conference.  Please visit the Conference official site to register

[Register now](http://c4ai.inova.usp.br/bracis2021/)

Register now

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – SBIA (22 editions), and the Brazilian Symposium on Neural Networks – SBRN (13 editions). 
ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. 
STIL is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: CE-IA, CE-IC and CE-PLN

technical paper

Textual information, despite being digital, are not computationally structured, requiring the use of techniques to structure and extract information. This work aims to evaluate machine learning NER (Named entity recognition) tools for the Brazilian and European Portuguese language variants. The Apache OpenNLP, Stanford CoreNLP and spaCy tools were selected for the evaluation. The HAREM corpus was used to train and evaluate the generated machine learning model; a tool was developed to preprocess the HAREM corpus. Two comparisons were performed: a general comparison and between the Portuguese variants. It was found that variants can confuse the training and evaluation process of NER models.

A Comparative Analysis of Machine Learning Named Entity Recognition Tools for the Brazilian and European Portuguese Language Variants

Automatically dealing with Natural Language User-Generated Con-tent (UGC) is a challenging task of utmost importance, given the amount of information available over the web. We present in this paper an effort on building tokenization and Part of Speech (PoS) tagging systems for tweets in Brazilian Portuguese, following the guidelines of the Universal Dependencies (UD)project. We propose a rule-based tokenizer and the customization of current state-of-the-art UD-based tagging strategies for Portuguese, achieving a 98%f-score for tokenization, and a 95% f-score for PoS tagging. We also introduceDANTEStocks, the corpus of stock market tweets on which we base our work, presenting preliminary evidence of the multi-genre capacity of our PoS tagger.

Universal Dependencies for Tweets in Brazilian Portuguese: Tokenization and Part of Speech Tagging

Species Distribution Modelling (SDM) is used for biodiversity preservation and wildlife management. In order to apply it, a large and reliable
dataset about the species occurrence is required. However, in some cases, this
can be difficult when there are only a few occurrence records. In this context,
uncertainty handling techniques can be applied. Thus, Bayesian inference was
used in this study to perform SDM in the Amazon Basin region near Manaus
(AM) with data collected by the GoAmazon 2014/15 project. The results were
compared with those obtained in statistical models and were similar.

Handling uncertainty through Bayesian inference for Species Distribution Modelling in the Amazon Basin region

Computational applications based on sensor data is a reality, but the
data collected and transmitted to the applications rarely arrives ready for use
due to losses and noise of various types. In this work, an approach based on spatial temporal correlation is developed for cleaning data from multiple temporal
series of sensors for noise, missing data and outliers. The method was tested
on six publicly available real datasets and its performance was compared with
a baseline method, an autoencoder denoising, and another published method.
The results show that the proposed approach is competitive and requires less
training data than competitors.

Data Cleansing of Multiple Environmental Monitoring Time Series Using Spatio-Temporal Correlation

Time series have great applicability in the most diverse scenarios, including the scientific, agricultural, economic domains, among others.
Therefore, creating effective representations of a time series is a challenging task, as it allows more accurate analysis and, consequently, more assertive results and conclusions in various machine learning tasks.
One of the main tasks is classification, which can be performed from different computational representations of time series.
The main objective of this work is to improve the efficiency of classification tasks using a representation of time series obtained by the Beam Angle Statistics algorithm, a contour feature extractor based on angular statistics.

Time Series Classification using Shape Features based on Angle Statistics

The use of Artificial Intelligence and Machine Learning algorithms in everyday life is common nowadays in several areas, bringing many possibilities and benefits to society. However, since there is room for learning algorithms to make decisions, the range of related ethical issues was also expanded. There are many complaints about Machine Learning applications that identify some kind of bias, disadvantaging or favoring some group, with the possibility of causing harm to a real person. The present work aims to shed light on the existence of biases, analyzing and comparing the behavior of different learning algorithms – namely Decision Tree, MLP, Naive Bayes, Random Forest, Logistic Regression and SVM – when being trained using biased data. We employed pre-processing algorithms for mitigating bias provided by IBM’s framework AI Fairness 360.

Performance analysis of machine learning algorithms trained on biased data

Chess is becoming more popular and accessible by the day. For instance, online chess enables matches between players from different parts of the world, bringing new ways of learning the game and interacting with other Web users. With this growth in popularity, there is a possibility to empower amateur players with rich computer analysis and tools, which may assist them in their learning process. One of the ways to analyze chess matches is through the study of errors. In this context, we present a new approach for the task of error prediction in chess. Our motivation is that knowing when players are likely to make a mistake is knowing what types of situations lead to difficulties in making the right decision. To that end, we add an abstraction layer to the already studied error prediction task, providing graph-based features to the machine learning models. Our results show an increase in the accuracy of the tested models, improving the results obtained in recent studies.

A lightweight approach for predicting errors in chess matches

Policy Search (PS) methods have been used to learn optimization algorithms, obtaining encouraging results. In this work, we used PS methods to learn optimization algorithms for global optimization problems, considering a little-studied scenario: high-dimensional functions and the optimization algorithms do not have access to the derivatives of the function to be optimized. The results indicate that, despite the difficulties, the learned optimization algorithms have promising performance in the studied scenario.

Zeroth Order Policy Search Methods for Global Optimization Problems: An Experimental Study

There is a growing interest in systems that detect violence automati-
cally using ambient audio. In this work, we built and evaluated four classifiers
with this proposal. However, instead of directly processing the audio signals, we
converted them to images, known as Mel-spectrograms, and then used Convolu-
tional Neural Networks (CNN) to treat as an image classification problem using
pre-trained networks in this context. We tested the architectures Inception v3,
VGG-16, MobileNet v2, and ResNet152 v2, and the classifier coming from the
MobileNet architecture obtained the best classification results when evaluated
on the HEAR Dataset created to carry out this research.

Deep Learning and Mel-spectrograms for Physical Violence Detection in Audio

We propose a solution based on Multi-Layer Perceptron (MLP) to predict the offset of the center of gravity of an offshore platform. We also perform a comparative study with three optimization algorithms – Random Search, Simulated Annealing, and Bayesian Optimization (BO) – to find the best MLP architecture. Although BO obtained the best architecture in the shortest time, ablation studies developed with hyperparameters of the optimization process showed that the result is sensitive to them and deserves attention in the Neural Architecture Search process.

Premium content

Optimization Matters: Guidelines to Improve Representation Learning with Deep Networks

Downloads

Next from BRACIS 2021

A Comparative Analysis of Machine Learning Named Entity Recognition Tools for the Brazilian and European Portuguese Language Variants

Similar lecture

Transfer Learning of Shapelets for Time Series Classification Using Convolutional Neural Network