Brazil

Machine learning models are widespread in many different fields due to their remarkable performances in many tasks. Some require greater interpretability, which often signifies that it is necessary to understand the mechanism underlying the algorithms. Feature importance is the most common explanation and is essential in data mining, especially in applied research. There is a frequent need to compare the effect of features over time, across models, or even across studies. For this, a single metric for each feature shared by all may be more suitable. Thus, analysts may gain better first-order insights regarding feature behavior across these different scenarios. The β-coefficients of additive models, such as logistic regressions, have been widely used for this purpose. They describe the relationships among predictors and outcomes in a single number, indicating both their direction and size. However, for black-box models, there is no metric with these same characteristics. Furthermore, even the β-coefficients in logistic regression models have limitations. Hence, this paper discusses these limitations together with the existing alternatives for overcoming them, and proposes new metrics of feature importance. As with the coefficients, these metrics indicate the feature effect’s size and direction, but in the probability scale within a model-agnostic framework. An experiment conducted on openly available breast cancer data from the UCI Archive verified the suitability of these metrics, and another on real-world data demonstrated how they may be helpful in practice.

BRACIS 2021

Interpreting classification models using feature importance based on marginal local effects

ale plots

explainable artificial intelligence

feature importance

The 9th Brazilian Conference on Intelligent Systems (BRACIS), the 16h Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), and the 12th Concurso de Teses e Dissertações em Inteligência Artificial e Computacional (CTDIAC) was held from 20 to 23 October, 2020.

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence - SBIA (22 editions), and the Brazilian Symposium on Neural Networks - SBRN (13 editions). ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. CTDIAC is a satellite event whose goals are to disseminate and award the best dissertations and theses (MSc and PhDs concluded in Brazilian universities from June 2018 to May 2020) in Artificial and Computational Intelligence areas.

In this year, the Symposium on Knowledge Discovery, Mining and Learning (KDMiLe) will be held in conjunction with BRACIS 2020. This sysposium aims at integrating researchers, practitioners, developers, students and users to present their research results, to discuss ideas, and to exchange techniques, tools, and practical experiences – related to the Data Mining and Machine Learning areas.

BRACIS 2020

technical paper

**BRACIS** originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – **SBIA** (22 editions), and the Brazilian Symposium on Neural Networks – **SBRN** (13 editions). **ENIAC** is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. **STIL** is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: **CE-IA**, **CE-IC** and **CE-PLN**.

To access the 10th Brazilian Conference on Intelligent Systems (**BRACIS**), you need to register for the conference.  Please visit the Conference official site to register

[Register now](http://c4ai.inova.usp.br/bracis2021/)

Register now

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence – SBIA (22 editions), and the Brazilian Symposium on Neural Networks – SBRN (13 editions). 
ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. 
STIL is the bi-annual Language Technology event.
All events are supported by the Brazilian Computer Society – Special Interest Groups: CE-IA, CE-IC and CE-PLN

Context: in the field of machine learning models are trained to learn from data, however often the context at which a model is deployed changes, degrading the performances of trained models and giving rise to a problem called Concept Drift (CD), which is a change in data distribution.Motivation: CD has attracted attention in machine learning literature, with works proposing modification to well-known algorithms’ structures, ensembles, online learning and drift detection, but most of the CD literature regards classification, while regression drift is still poorly explored. Objetive: The goal of this work is to perform a comparative study of CD detectors in the context of regression. Results: we found that (i) PH, KSWIN and EDDM showed higher detection averages; (ii) the base learner has a strong impact in CD detection and (iii) the rate at which CD happens also affects the detection process. Conclusion: our experiments were executed in a framework that can easily be extended to include new CD detectors and base learners, allowing future studies to use it.

A Comparative Study on Concept Drift Detectors for Regression

Automatic fingerprint identification systems (AFIS) are among the most used people identification solutions. As the size of fingerprint databases is continuously growing, studying fingerprint indexing mechanisms is desirable to facilitate the search process in a large-scale database. This work presents a method for fingerprint indexing, which uses both exact and approximation methods of nearest neighbors (ANNs), which are very efficient in terms of runtime, even if they sacrifice a little accuracy by presenting approximate solutions. In the presented approach, searches with ANN methods are made from deep embedding vectors extracted from image databases using a convolutional neural network (CNN). In this work, a CNN ResNet18 was used to extract the deep feature embeddings vectors, and the vectors vary in size between 64, 96, and 128. The ANNs methods tested for the query step were ANNOY, NGT, HNSW, and Nanoflann. The results were quite promising when using the FVC fingerprint databases (2000, 2002, and 2004), once we reached 100% hits in the searches with a penetration rate of 1%, with very low run times.

Deep Convolutional Features for Fingerprint Indexing

Reviews are valuable sources of information to support the decision making process. Therefore, the task of classifying reviews according to their helpfulness has paramount importance to facilitate the access of truly informative content. In this context, previous studies have unveiled several aspects and architectures that are beneficial for the task of review perceived helpfulness prediction. The present work aims to further investigate the influence of the text specificity aspect, defined as the level of details conveyed in a text, with the same purpose. First, we explore an unsupervised domain adaptation approach for assigning text specificity scores for sentences from product reviews and we propose an evaluation measure named Specificity Prediction Evaluation (SPE) in order to achieve more reliable specificity predictions. Then, we present domain-oriented guidelines on how to incorporate, into a CNN architecture, either hand-crafted features based on text specificity or the text specificity prediction task as an auxiliary task in a multitask learning setting. In the experiments, the perceived helpfulness classification models embodied with text specificity showed significant higher precision results in comparison to a popular SVM baseline.

Incorporating Text Specificity into a Convolutional Neural Network for the Classification of Review Perceived Helpfulness

The Federal Government Purchase Card (CPGF) has been used in Brazil since 2005, allowing agencies and entities of the federal public administration to make purchases of material and provision of services through this method. Although this payment system offers several advances, in the technological and administrative aspect, it is also susceptible to possible cases of card misuse and, consequently, waste of public funds, in the form of purchases that do not comply with the terms of the current legislation. In this work, we approach this problem by testing and evaluating unsupervised learning techniques on detecting anomalies in CPGF historical data. Four different methods are considered for this task: K-means, agglomerative clustering, a network-based approach, which is also introduced in this study, and a hybrid model. The experimental results obtained indicate that unsupervised methods, in particular the network-based approach, can indeed help in the task of monitoring government purchase card expenses, by flagging suspect transactions for further investigation without requiring the presence of a specialist in this process.

Anomaly Detection in Brazilian Federal Government Purchase Cards Through Unsupervised Learning Techniques

The popular participation in Law-making is an important resource in the evolution of Democracy and Direct Legislation. The amount of legislative documents produced within the past decade has risen dramatically, making it difficult for law practitioners to attend to legislation and still listen to the opinion of the citizens. This work focuses on the use of topic models for summarizing and visualizing Brazilian comments about legislation (bills). In this paper, we provide a qualitative evaluation from a legal expert and compare it with the topics predicted by our model. For such, we designed a specific sentence embedding technique able to induce models for Portuguese texts, and we used these models as topic model, obtaining very good results. We experimentally compared our proposal with other techniques for multilingual sentence embeddings, evaluating them in three topical corpora prepared by us, two of them annotated by a specialist and the other automatically annotated by hashtags.

Evaluating Topic Models in Portuguese Political Comments about Bills from Brazil's Chamber of Deputies

The dynamism of fake news evolution and dissemination plays a crucial role in influencing and confirming personal beliefs. To minimize the spread of disinformation approaches proposed in the literature, automatic fake news detection generally learns models through binary supervised algorithms considering textual and contextual information. However, labeling significant amounts of real news to build accurate classifiers is difficult and time-consuming due to their broad spectrum. Positive and unlabeled learning (PUL) can be a good alternative in this scenario. PUL algorithms learn models considering little labeled data of the interest class and use unlabeled data to increase classification performance. This paper proposes a heterogeneous network variant of the PU-LP algorithm, a PUL algorithm based on similarity networks. Our network incorporates different linguistic features to characterize fake news, such as representative terms, emotiveness, pausality, and average sentence size. Also, we considered two representations of the news to compute similarity: term frequency-inverse document frequency, and Doc2Vec, which creates fixed-sized document representations regardless of its length. We evaluated our approach in six datasets written in Portuguese or English, comparing its performance with a binary semi-supervised baseline algorithm, using two well-established label propagation algorithms: LPHN and GNetMine. The results indicate that PU-LP with heterogeneous networks can be competitive to binary semi-supervised learning. Also, linguistic features such as representative terms and pausality improved the classification performance, especially when there is a small amount of labeled news.

A Heterogeneous Network-based Positive and Unlabeled Learning Approach to Detect Fake News

This work presents a study on an ethical issue in Artificial Intelligence related to the presence of racist biases by detecting faces in images. Our analyses were performed on a real-world system designed to detect fraud in public transportation in Salvador (Brazil). Our experiments were conducted by taking into account three steps. Firstly, we individually analyzed a sample of images and added specific labels related to the users' gender and race. Then, we used well-defined detectors, based on different Convolutional Neural Network architectures, to find faces in the previously labeled images. Finally, we used statistical tests to assess whether or not there is some relation between the error rates and such labels. According to our results, we had noticed important biases, thus leading to higher error rates when images were taken from black people. We also noticed errors are more likely in both black men and women. Based on our conclusions, we recall the risk of deploying computational software that might affect minority groups that are historically neglected.

Ethics of AI: Do the face detection models act with prejudice?

The “neat” and “scruffy” portraits have long been painted to describe viewpoints, styles of reasoning and methodologies in AI research. Essentially, the neats defend techniques based on first principles and grounded in mathematical rigor, while the scruffies advocate diversity within cognitive architectures, sometimes meant to be models of parts of the brain, sometimes just kludges or ad-hoc pieces of engineered code. The recent success of deep learning has revived the debate between these two approaches to AI; in this context, some natural questions arise. How can we characterize, and how can we classify, these positions given the history of AI? More importantly, what is the relevance of these positions for the future of AI? How should AI research be pursued from now on, neatly or scruffly? These are the questions we address in this paper, resorting to historical analysis and to recent research trends to articulate possible ways to allocate energy so as to take the field to maximal fruition.

The future of AI: neat or scruffy?

Hierarchical data stream classification inherits the properties and constraints of hierarchical classification and data stream classification concomitantly. Therefore, it requires novel approaches that (i) can handle class hierarchies, (ii) can be updated over time, and (iii) are computationally light-weighted regarding processing time and memory usage. In this study, we propose the Gaussian Naive Bayes for Hierarchical Data Streams (GNB-hDS) method: an incremental Gaussian Naive Bayes for classifying potentially unbounded hierarchical data streams. GNB-hDS uses statistical summaries of the data stream instead of storing actual instances. These statistical summaries allow more efficient data storage, keep constant computational time and memory, and calculate the probability of an instance belonging to a specific class via the Bayes' Theorem. We compare our method against a technique that stores raw instances, and results show that our method obtains equivalent prediction rates while being significantly faster.

Classifying Potentially Unbounded Hierarchical Data Streams with Incremental Gaussian Naive Bayes

An effective way to cope with classification problems, among others, is by using Fuzzy Rule-Based Classification Systems (FRBCSs). These systems are composed by two main components, the Knowledge Base (KB) and the Fuzzy Reasoning Method (FRM). The FRM is responsible for performing the classification of new examples based on the information stored in the KB. A key point in the FRM is how the information given by the fired fuzzy rules is aggregated. Precisely, the aggregation function is the component that differs from the two most widely used FRMs in the specialized literature. In this paper we provide a revision of the literature discussing the generalizations of the Choquet integral that has been applied in the FRM of a FRBCS. To do so, we consider an analysis of different generalizations, by t-norms, copulas, and by F functions. Also, the main contributions of each generalization are discussed.

Premium content

Interpreting classification models using feature importance based on marginal local effects

Downloads

Next from BRACIS 2021

A Comparative Study on Concept Drift Detectors for Regression

Similar lecture

An Argumentation-based Approach for Explaining Goal Selection in Intelligent Agents