Brazil

It has been advocated that post-hoc explanation techniques are crucial for increasing the trust in complex Machine Learning (ML) models. However, it is so far not well understood whether such explanation techniques are useful or easy for users to understand. In this work, we explore the extent to which SHAP&#39;s explanations, a state-of-the-art post-hoc explainer, help humans to make better decisions. In the malaria classification scenario, we have designed an experiment with 120 volunteers to understand whether humans, starting with zero knowledge about the classification mechanism, could replicate the complex ML classifier&#39;s performance after having access to the model explanations. Our results show that this is indeed the case, i.e., when presented with the ML model outcomes and the explanations, humans can improve their classification performance, indicating that they understood how the ML model makes its decisions.

BRACIS 2020

Evaluation of Post-hoc Explanations for Malaria Detection

explanation evaluation

shap

explainability

deep learning

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


ICPR 2020

It has been advocated that post-hoc explanation techniques are crucial for increasing the trust in complex Machine Learning (ML) models. However, it is so far not well understood whether such explanation techniques are useful or easy for users to understand. In this work, we explore the extent to which SHAP's explanations, a state-of-the-art post-hoc explainer, help humans to make better decisions. In the malaria classification scenario, we have designed an experiment with 120 volunteers to understand whether humans, starting with zero knowledge about the classification mechanism, could replicate the complex ML classifier's performance after having access to the model explanations. Our results show that this is indeed the case, i.e., when presented with the ML model outcomes and the explanations, humans can improve their classification performance, indicating that they understood how the ML model makes its decisions.

technical paper

The 9th Brazilian Conference on Intelligent Systems (BRACIS), the 16h Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), and the 12th Concurso de Teses e Dissertações em Inteligência Artificial e Computacional (CTDIAC) was held from 20 to 23 October, 2020.

BRACIS originated from the combination of the two most important scientific events in Brazil in Artificial Intelligence (AI) and Computational Intelligence (CI): the Brazilian Symposium on Artificial Intelligence - SBIA (22 editions), and the Brazilian Symposium on Neural Networks - SBRN (13 editions). ENIAC is a national forum for researchers, practitioners, educators and students to present and discuss innovations, trends, experiences, developments and work in progress in the fields of Artificial and Computational Intelligence. CTDIAC is a satellite event whose goals are to disseminate and award the best dissertations and theses (MSc and PhDs concluded in Brazilian universities from June 2018 to May 2020) in Artificial and Computational Intelligence areas.

In this year, the Symposium on Knowledge Discovery, Mining and Learning (KDMiLe) will be held in conjunction with BRACIS 2020. This sysposium aims at integrating researchers, practitioners, developers, students and users to present their research results, to discuss ideas, and to exchange techniques, tools, and practical experiences – related to the Data Mining and Machine Learning areas.

Visualizing multidimensional Big Data is defying: high dimensionalities hinder or even preclude visual inspections. A means of tackling this issue is to use DR (Dimensionality Reduction) techniques, producing low-dimensional representations of high-dimensional data. Popular DR algorithms (e.g., Principal Component Analysis, t-Distributed Stochastic Neighbor Embedding), albeit helpful, are computationally expensive. Most have quadratic or cubic ATC (Asymptotic Time Complexity) and/or calculate pairwise distances of the entire data set, exceeding available memory and rendering Big Data DR time-consuming or impracticable. These issues impede the employment of DR for online learning appliances, where recurrent, cumulative model updates are habitual. The stochastic factor of some approaches similarly obstructs any meaningful inspection on how knowledge is spatially disposed. The recently introduced PCS (Polygonal Coordinate System)—an incremental, geometric-based technique with linear ATC—is compelling; however, its restriction to 2-D embeddings amounts to significant information loss. We propose the Big Data ready, incremental PES (Pyramidal Embedding System), which builds on PCS virtues by additionally generating 3-D embeddings through its pyramid-like interspace, mitigating quality degradation. Visual inspections, as well as pairwise distance based statistical analyses, validate the PES ability to retain structural arrangements when embedding high- and low-dimensional data while retaining flexibility in resources consumption.

An Online Pyramidal Embedding Technique for High Dimensional Big Data Visualization

Machine translation is intrinsically associated with the study and development of computerized methodologies for idiomatic translations' production. The most common approaches are the statistical and methods based on neural networks. One of the deficiencies pointed out by these methods is the possible lack of coherence between the translated sentences. In this project, we propose using techniques based on Graph Theory to preserve the coherence in the translation of texts from English to Portuguese. The studied method presents large performance variability; however, some translations produce sentences 90% better evaluated than the statistical translator Moses and 10% superior to Google Translate.

Post-processing of machine translation texts based on graph theory

This study presents an algorithmic strategy to analyze a small set of social network information to monitor the dengue disease. Previous studies have achieved similar results based on large datasets of Twitter microblogs. In this study, we successfully map dengue cases using a small data collection of tweets from a medium-size city. A set of modules were constructed to collect, categorize, and display dengue-related tweets. We compared the collected tweets with real data from confirmed dengue cases. We showed a significant correlation between the number of confirmed dengue cases and the number of dengue-related tweets, even considering such a small dataset. The results of this approach may be relevant in public health policies.

Statistical analysis of small twitter data collection to identify dengue outbreaks

Since 1989, the first year of the democratic presidential election after a long period of a dictatorship regime, Brazil conducted eight presidential elections. This period was marked by short and long-term shifts of power and two impeachment processes. Such instability is a case of study in electoral studies, e.g., the study of the population voting behavior. Understanding patterns in the population behavior can give us insight into factors and influences that affect the quality of democratic political decisions. In light of this, our paper focuses on analyzing the Brazilian presidential election voting behavior across the years and the Brazilian territory. Following a data science pipeline, we divided the analysis process into five steps: (i) data selection; (ii) data preprocessing; (iii) identification of spatial patterns, in which we seek to understand the role of space in the election results using spatial autocorrelation techniques; (iv) identification of temporal patterns, where we investigate similar trends of votes over the years using a hierarchical clustering method; and (v) evaluation of the results. It is noteworthy that the data in this work represents the election results at the municipal level, from 1994 to 2018, of the two most relevant parties of this period: the Brazilian Social Democracy Party (PSDB) and the Workers' Party (PT). Through the results obtained, we found the existence of spatial dependence in every electoral year investigated. Moreover, despite the changes in the political-economic context over the years, neighboring cities seem to present similar voting behavior trends.

Brazilian Presidential Elections: Analysing Voting Patterns in Time and Space Using a Simple Data Science Pipeline

Many multi-agent reinforcement learning (MARL) scenarios lead towards Nash equilibria, which is knownto not always be socially efficient. In my research we aimed to align the social optimization objective of the system with the individual objectives of the agents by adopting a central controller which can interact with the agents.

Inducing selfish agents towards social efficient solutions

As technology evolves and electronic devices become widespread, the amount of data produced in the form of stream increases in enormous proportions. Data streams are an online source of data, meaning that it keeps producing data continuously. This creates the need for fast and reliable methods to analyse and extract information from these sources. Stream mining algorithms exist for this purpose, but the use of supervised machine learning is extremely limited in the stream domain since it is unfeasible to label every data instance requested to be processed. Tackling this problem, our paper proposes the use of active learning techniques for stream mining algorithms, specifically incremental Hoeffding trees-based. It is important to mention that the active learning techniques were implemented to match the stream mining constraints regarding low computational cost. We took advantage of the incremental tree original structure to avoid overburdening the original computational cost when selecting a label. In other words, the statistical strategy to grow each incremental tree has supported the execution of active learning. Using techniques of uncertainty sampling, we were able to drastically reduce the number of labels required at the cost of a very small reduction in accuracy. Particularly with Budget Entropy there was an average negative impact of accuracy about 4% using only 14% of samples labelled.

Active Learning embedded in incremental decision trees

As commodities milho e soja são produtos consumidos em grande escala no mundo. As flutuações nos preços de mercado têm efeitos de longo alcance sobre os consumidores, agricultores e processadores de grãos. Assim, a previsão dos preços desses grãos tem atraído significativa atenção dos pesquisadores. Os modelos de previsão geralmente usam dados quantitativos de série temporal. No entanto, fatores qualitativos externos podem influenciar os dados em séries temporais, como eventos políticos, crises econômicas e o mercado de câmbio estrangeiro. Essas informações não são explícitas nos dados da série temporal e esses fatores podem influenciar a previsão dos valores das variáveis. Os dados textuais extraídos de notícias, fóruns e redes sociais podem ser uma fonte de conhecimento sobre fatores externos e potencialmente úteis para modelos de previsão de séries temporais. Alguns estudos apresentam técnicas de mineração de texto para combinar dados textuais com séries temporais. Porém, as representações existentes apresentam algumas limitações, como a maldição da dimensionalidade e atributos ineficazes. Este trabalho aplica métodos de pré-processamento em séries temporais e usa representações combinadas com dados textuais para prever o preço futuro do milho e da soja. Os resultados indicam que os métodos utilizados podem ser uma alternativa para melhorar o desempenho de previsão em tarefas de regressão.

Previsão do preço futuro do milho e da soja: Uma análise do uso de informação textual para enriquecer séries temporais.

Automatic diagnoses of diseases has been a long term challenge for Computer Science and related disciplines. Textual clinical reports can be used as a great source of data for such diagnoses. However, building classification models from them is not a trivial task. The problem tackled in this work is the identification of the medical diagnoses that are indicated in these reports. In the past, several methods have been proposed for addressing this problem, but a method developed for reports in the cardiology area that are written in Portuguese is still needed. In this paper we describe a method that is able to handle the peculiarities of clinical reports, including the medical terminology, and that is implemented to estimate correctly the disease based on raw clinical reports and a list of the possible diagnoses. Experimental results show that our method has a high degree of accuracy, even for infrequent classes and complex databases.

Automated classification of cardiology diagnoses based on textual medical reports

Labor Accidents in Brazil: a Descriptive Analysis

The opinion issued by consumers of products and services has become increasingly valued, both by other consumers and by companies. The automatic interpretation of review texts to generate information is of paramount importance. With opinion mining at the aspect level, it is possible to extract and summarize opinions about different components of a product or service. This paper evaluates the behavior of a method for extracting aspects using natural language processing tools for the Portuguese language. The aim is to investigate the maturity of the tools for Portuguese compared to the already consolidated tools for the English language. The evaluation was carried out in three datasets from two different domains with original texts in Portuguese and their translations into English, and vice versa, and the results indicate that there is no difference between languages.

Evaluation of Post-hoc Explanations for Malaria Detection

Downloads

Next from BRACIS 2020

An Online Pyramidal Embedding Technique for High Dimensional Big Data Visualization

Similar lecture

Attack-agnostic Adversarial Detection on Medical Data Using Explainable Machine Learning

Evaluation of Post-hoc Explanations for Malaria Detection

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from BRACIS 2020

An Online Pyramidal Embedding Technique for High Dimensional Big Data Visualization

Similar lecture

Attack-agnostic Adversarial Detection on Medical Data Using Explainable Machine Learning

Downloads