Brazil

Communication has become increasingly dynamic with the popularization of social networks and applications that allow people to express themselves and communicate instantly. In this scenario, distributed representation models have their quality impacted by new words that appear frequently or that are derived from spelling errors. These words that are unknown by the models, known as out-of-vocabulary (OOV) words, need to be properly handled to not degrade the quality of the natural language processing (NLP) applications, which depend on the appropriate vector representation of the texts. To better understand this problem and finding the best techniques to handle OOV words, in this study, we present a comprehensive performance evaluation of deep learning models for representing OOV words. We performed an intrinsic evaluation using a benchmark dataset and an extrinsic evaluation using different NLP tasks: text categorization, named entity recognition, and part-of-speech tagging. Although the results indicated that the best technique for handling OOV words is different for each task, Comick, a deep learning method that infers the embedding based on the context and the morphological structure of the OOV word, obtained promising results.

BRACIS 2020

Deep learning models for representing out-of-vocabulary words

natural language processing; machine learning; out-ofvocabulary words

2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics.

**Whova App** 
Stay in touch with your fellow conference attendees via the [Whova App](https://whova.com/portal/webapp/nacon_202106/)

**Conference Structure**
https://2021.naacl.org/blog/conference-structure/

**Walkthrough video of how to NAACL 2021** 

Please take a moment to view this video explaining how to navigate the platform, attend sessions network with other attendees. 


<figure class="video_container">
  <iframe src="https://screencast-o-matic.com/watch/crhwbGVh3vx?v=6&ff=1&title=0&controls=1" width=640  height=350  frameborder="0" allowfullscreen="true"> </iframe>
</figure>

<br>
<br>

NAACL 2021

technical paper

The 9th Brazilian Conference on Intelligent Systems (BRACIS), the 16h Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), and the 12th Concurso de Teses e Dissertações em Inteligência Artificial e Computacional (CTDIAC) was held from 20 to 23 October, 2020.

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence - SBIA (22 editions), and the Brazilian Symposium on Neural Networks - SBRN (13 editions). ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. CTDIAC is a satellite event whose goals are to disseminate and award the best dissertations and theses (MSc and PhDs concluded in Brazilian universities from June 2018 to May 2020) in Artificial and Computational Intelligence areas.

In this year, the Symposium on Knowledge Discovery, Mining and Learning (KDMiLe) will be held in conjunction with BRACIS 2020. This sysposium aims at integrating researchers, practitioners, developers, students and users to present their research results, to discuss ideas, and to exchange techniques, tools, and practical experiences – related to the Data Mining and Machine Learning areas.

In recent years, the advances in technology have produced datasets of increasing size, not only regarding the number of samples but also the number of features. Unfortunately, creating a sufﬁciently large amount of adequately labeled data with enough examples for each class is not easy. Labeling is a challenging, expensive, and time-consuming task. It is usually done manually, which may contribute to the insertion of noise and errors in the data. Hence, it is of great importance to put forward intelligent models that can beneﬁt from the distinct information that both labeled and unlabeled data can provide, since, for many applications, there is a plentiful amount of unlabeled data, but insufﬁcient labeled ones. Semi-Supervised Learning (SSL) is employed to achieve this. It is halfway between supervised and unsupervised learning. In this sense, we highlight two very inﬂuential models: Self-Organizing Maps (SOM) and Learning Vector Quantization (LVQ). SOM is a biologically inspired neural model that uses unsupervised and incremental learning to produce prototypes of the input data, whereas the LVQ can be seen as its supervised counterpart. The unsupervised characteristic of SOM makes it unfeasible to execute SSL. In that way, the current work proposes new models that incorporate standard concepts from LVQ to the SOM algorithm to build semi-supervised approaches. Such proposals can dynamically switch between the two types of learning at training time, according to the availability of labels and automatically adjust themselves to the local variance observed in each cluster. The experimental results show that the proposed models can surpass the performance of other traditional methods not only in terms of classiﬁcation but also regarding clustering quality. It also enhances the range of applications of SOM and LVQ-based models by combining them with Deep Learning in a synergic way to allow dealing with complex data structures, such as images and sound. Moreover, we explore forms of learning good representations of the input data, and manners to estimate the unsupervised error when no labels are provided. Our approaches demonstrated to be good at producing a meaningful topology and clustering prototypes that appropriately represent the data.

Semi-Supervised Self-Organizing Maps with Time-Varying Structures for Clustering and Classification

This work studies quantitative measures for ranking judicial decisions by the Brazilian Supreme Court (STF).
The measures are based on a network built over decisions whose cases were finalized in the Brazilian Supreme Court between 01/2001 and 12/2019, obtained by crawling publicly available STF records.
Three ranking measures are proposed; two are adaptations of the PageRank algorithm, and one adapts Kleinberg's Algorithm.
All are compared with respect to agreement on top 100 rankings; we also analyze each measure robustness based on self-agreement under perturbation.

We conclude that all algorithms show the network of citations is highly robust under perturbation.
Both versions of PageRank, even if producing different rankings, achieved robustness results which are indistinguishable via statistical tests; Kleinberg's algorithm achieves more promising results to rank leading cases at the top, but it does need more research to achieve this goal.


Robust Ranking of Brazilian Supreme Court Decisions

Recent research on Convolutional Neural Networks focuses on how to create models with a reduced number of parameters and a smaller storage size while keeping the model's ability to perform its task, allowing the use of the best CNN for automating tasks in limited devices, with reduced processing power, memory, or energy consumption constraints. There are many different approaches in the literature: removing parameters, reduction of the floating-point precision, creating smaller models that mimic larger models, neural architecture search (NAS), etc. With all those possibilities, it is challenging to say which approach provides a better trade-off between model reduction and performance, due to the difference between the approaches, their respective models, the benchmark datasets, or variations in training details. Therefore, this article contributes to the literature by comparing three state-of-the-art model compression approaches to reduce a well-known convolutional approach for object detection, namely YOLOv3. Our experimental analysis shows that it is possible to create a reduced version of YOLOv3 with 90% fewer parameters and still outperform the original model by pruning parameters. We also create models that require only 0.43% of the original model's inference effort.

An Experimental Analysis of Model Compression Techniques for Object Detection

The world around us is composed of images that often need
to be translated into words. This translation can take place in parts,
converting regions of the image into textual descriptions what is also
known as dense captioning. By doing so, the information present in this
region is converted into words expressing the way objects relate to each
other. Computational models have been proposed to perform this task in
a similar way to human beings, mainly using deep neural networks. As the
same region of the image can be described in several different forms, this
study proposes to use the Abstract Meaning Representation (AMR) in
the generation of descriptions for a given region. We hypothesize that by
using AMR it would be possible to extract the meaning of the text and,
as a consequence, improve the quality of the sentences produced by the
models. AMR was investigated as a semantic representation formalism
evolving the so far proposed models that are based only on purely natural
language. The results show that the models trained with sentences in the
form of AMR led to better descriptions and the performance achieved
was superior in almost all evaluations.

Dense Captioning using Abstract Meaning Representation

How to choose a route that takes you from A to B? This is an issue that is turning more and more important in modern societies. One way to address this agenda is through the use of communication between the infrastructure (network), and the demand (vehicles). In this paper, we use car-to-infrastructure (C2I) communication to investigate whether the road users (agents) can accelerate their learning process regarding route choice problem, via reinforcement learning (RL). We employ a microscopic simulator in order to compare our method with two others: RL without communication and an iterative method. Experimental results show that our method outperforms both methods in terms of effectiveness and efficiency.

Accelerating Learning of Route Choices With C2I: A Preliminary Investigation

This work addresses a critical problem in the use of the Face Recognition by a police department state of Brazil. Great advances have been shown in the last years and the state of art works are based on large labeled datasets and supervised training. But with this approach, the lack of representative data distribution is an issue, known as data bias, mainly according to some aspects that makes FR harders: gender and race. Recent works have suggested that these two aspects may cause a significant drop in accuracy. This work is concerned over the data bias problem for Brazilian faces. Using pre-trained models trained with public datasets, we demonstrate that even with small size dataset it is possible to improve the accuracy of Brazilian faces with simple yet effective implementation tricks in fine-tuning. Two important conclusions was obtained from this study: First, there is a strong suggestion the there is data bias concerning ethnicity when evaluating models on Brazilian faces trained with public datasets, and second, the fine-tuning task using a brazillian non-public dataset resulted in significant mitigation of this problem. 


Improving Face Recognition Accuracy for Brazilian Faces in a Criminal Investigation Department

The internet connection is present in people's lives all the time, through smartphones, tablets, computers, among others. The use of social networks is increasingly common around the world. Many companies and people use them to spread products and services and publish opinions, facts that have turned the social networks into powerful sources of information on various topics. Identifying these feelings is a great strategy for many types of decision making. Thus, the purpose of this paper is to collect messages from a specific social network, in this case Twitter, referring to the 2018 Brazilian presidential elections and classify them as: positive, negative and neutral, in order to discover a possible relationship between opinions of social network users and the final outcome of the elections. For this, a corpus was built, preprocessed and evaluated by two different machine learning approaches: Naive Bayes and SVM (Support Vector Machine). The results showed that this social network is a good source of information to perform sentiment analysis and that the number of tweets classified as positive have a strong relationship with the researchers and the final result of the 2018 elections.

A Sentiment Analysis of Brazilian Elections Tweets

Dependency Parsers (DP) are parsers that analyze dependencies between words in a sentence. Currently, dependency parser evaluation is a problem whose solutions are not well defined in the scientific community. Although the DP intrinsic metrics are the foremost choice of evaluation, extrinsic evaluation enables a different evaluation based on a downstream. Different results of DP can vary according to the domain task. Thus, this work applies an Open Information Extraction (OIE) method in Portuguese to provide an extrinsic evaluation of a set of CONLL Dependency Parsers. Our results demonstrate that there is a difference in the evaluation of Dependency Parsers considering a particular task. 


CoNLL Dependency Parser: Extrinsic Evaluation through the Open Information Extraction task

The lackadaisical quantum walk is a graph search algorithm for 2D grids whose vertices have a self-loop of weight l. Since the technique depends considerably on this l, research efforts have been estimating the optimal value for different scenarios, including 2D grids with multiple solutions. However, specifically
two previous works have used different stopping conditions for the simulations. Here, firstly, we show that these stopping conditions are not interchangeable. After doing such a pending investigation to define the stopping condition properly, we analyze the impacts of multiple solutions on the final results achieved by
the technique, which is the main contribution of this work. In doing so, we demonstrate that the success probability is inversely proportional to the density of vertices marked as solutions and directly proportional to the relative distance between solutions. These relations presented here are guaranteed only for high
values of the input parameters because, from different points of view, we show that a disturbed transition range exists in the small values.

Impacts of Multiple Solutions on the Lackadaisical Quantum Walk Search Algorithm

Standard features used for Credit Scoring includes mainly registration and financial data from customers. However, exploring new features is of great interest for financial companies, since slight improvements in the person score directly impact the company revenue. In this work, we categorize features from open credit scoring datasets and compare them with the features found in a real company dataset. The company dataset contains unusual feature groups such as historical, geolocation, web behavior, and demographic data. We performed bivariate tests using the Kolmogorov-Smirnov metric and features to assess the performance of the particular feature groups. We also generated a score of good payer by using AdaBoost, Multilayer Perceptron, and XGBoost algorithms. Then, we analyzed the results with different metrics and compared them with the real company results. Our main finding was that these features added a small improvement to current datasets. We also identified the most promising feature groups and noticed that the tuned XGBoost performed better than the company solution in three out of four deployed metrics.


Deep learning models for representing out-of-vocabulary words

Downloads

Next from BRACIS 2020

Semi-Supervised Self-Organizing Maps with Time-Varying Structures for Clustering and Classification

Similar lecture

Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks

Deep learning models for representing out-of-vocabulary words

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from BRACIS 2020

Semi-Supervised Self-Organizing Maps with Time-Varying Structures for Clustering and Classification

Similar lecture

Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks

Downloads