Italy

This paper profiles the recent research work on eXplainable AI (XAI), at the Insight Centre for Data Analytics. This work concentrates on post-hoc ex-planation-by-example solutions to XAI as one approach to explaining black box deep-learning systems. Three different methods of post-hoc ex-planation are outlined for image and time-series datasets: that is, factual, counterfactual, and semi-factual methods). The future landscape for XAI solutions is discussed.

ICPR 2020

Post-Hoc Explanation Options for XAI in Deep Learning: The Insight Centre for Data Analytics Perspective

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


technical paper

Neural networks are increasingly applied to high-stakes tasks, such as autonomous driving and medical applications. For these tasks, it is important to explain the contributions of data components to a model prediction. In recent times, mask-based methods have been proposed for visual explanations. They optimize masks that maximally affect the model output. In this work, we propose a novel mask-based saliency method for given black box classifiers. We jointly optimize positive and negative masks to achieve a faithful feature importance map. To optimize them effectively, we define a distance between them in terms of the selected activation maps; the corresponding kernels express the important features of an input image. Then, we impose the distance to the objective function of each mask as a regularizer. By forcing both masks to be dissimilar in terms of the influential features, they can focus on essential parts of an object while alleviating noises.

Jointly Optimize Positive and Negative Saliencies for Black Box Classifiers

In the astrophysics domain, the detection and description of gamma rays is a research direction for our understanding of the universe. Gamma-ray reconstruction from Cherenkov telescope data is multi-task by nature. The image recorded in the Cherenkov camera pixels relates to the type, energy, incoming direction and distance of a particle from a telescope observation. We propose γ-PhysNet, a physically inspired multi-task deep neural network for gamma/proton particle classification, and gamma energy and direction reconstruction. As ground truth does not exist for real data, γ-PhysNet is trained and evaluated on large-scale
Monte Carlo simulations. Robustness is then crucial for the transfer of the performance to real data. Relying on a visual explanation method, we evaluate the influence of attention on the variability due to weight initialization, and how it helps improve the robustness of the model. All the experiments are conducted in the context of single telescope analysis for the Cherenkov Telescope Array simulated data analysis.

Deep Learning for Astrophysics, Understanding the Impact of Attention on Variability Induced by Parameter Initialization

Machine Learning (ML) is becoming an increasingly critical technol-ogy in many areas such as health, business but also in everyday applications of significant societal importance. However, the lack of explainability or ability of ML systems to offer explanation on how they work, which refers to the model (related to the whole data) and sample explainability (related to specific sam-ples) poses significant challenges in their adoption, verification, and in ensur-ing the trust among users and general public. We present novel integrated Ran-dom Forest Model and Sample Explainer – RFEX. RFEX is specifically de-signed for important class of users who are non-ML experts but are often the domain experts and key decision makers. RFEX provides easy to analyze one-page Model and Sample explainability summaries in tabular format with wealth of explainability information including classification confidence, tradeoff be-tween accuracy and features used, as well as ability to identify potential outlier samples and features. We demonstrate RFEX on two case studies: mortality prediction for COVID-19 patients from the data obtained from Huazhong Uni-versity of Science and Technology, Wuhan, China, and classification of cell type clusters for human nervous system based on the data from J. Craig Venter Institute. We show that RFEX offers simple yet powerful means of explaining RF classification at model, sample and feature levels, as well as providing guid-ance for testing and developing explainable and cost-effective operational pre-diction models.

Random Forest Model and Sample Explainer for non-experts in Machine Learning – Two Case Studies

Emotions are very important in dealing with human decisions, interactions, and cognitive processes. Art is an imaginative human creation that should be appreciated, thought-provoking, and elicits an emotional response. The automatic recognition of emotions triggered by art is of considerable importance. It can be used to categorize artworks according to the emotions they evoke, recommend paintings that accentuate or balance a particular mood, and search for paintings of a particular style or genre that represent custom content in a custom state of impact. In this paper, we propose an attention-based multi-modal approach to emotion recognition that aims to use information from both the painting and title channels to achieve more accurate emotion recognition. Experimental results on the WikiArt emotion dataset showed the efficiency of the model we proposed and the usefulness of image and text modalities in emotion recognition.

Attention-based Multi-modal Emotion Recognition From Art

Document images dewarping is still a challenge especially when documents are captured with one camera in an uncontrolled environment. In this paper we propose a generic approach based on vanishing points (VP) to reconstruct the 3D shape of document pages. Unlike previous methods we do not need to segment the text included in the documents. Therefore, our approach is less sensitive to pre-processing and segmentation errors. The computation of the VPs is robust and relies on the a-contrario framework, which has only one parameter whose setting is based on probabilistic reasoning instead of experimental tuning. Thus, our method can be applied to any kind of document including text and non-text blocks and extended to other kind of images. Experimental results show that the proposed method is robust to a variety of distortions.

Generic 10 - 752

In previous works, a mobile application was developed using an unmodified commercial smartphone to recognize whole-body exercises. The working principle was based on the ultrasound Doppler sensing with the device built-in hardware. Applying such a lab environment trained model on realistic application variations causes a significant drop in performance, and thus decimate its applicability. The reason of the reduced performance can be manifold. It could be induced by the user, environment, and device variations in realistic scenarios. Such scenarios are often more complex and diverse, which can be challenging to anticipate in the initial training data. To study and overcome this issue, this paper presents a database with controlled and uncontrolled subsets of fitness exercises. We propose two concepts to utilize small adaption data to successfully improve model generalization in an uncontrolled environment, increasing the recognition accuracy by two to six folds compared to the baseline for different users.

Generalization of Fitness Exercise Recognition from Doppler Measurements by Domain-Adaption and Few-Shot Learning

An empirical comparison between different skeleton-based methods for both pose estimation and action recognition speaker actions in lecture videos.

Skeleton-based Methods for Speaker ActionClassification on Lecture Videos

Machine learning techniques and algorithms are employed in many application domains such as financial applications, recommendation systems, medical diagnosis systems, and self-driving cars. They play a crucial role in harnessing the power of Big Data being produced every day in our digital world. In general, building a well-performing machine learning pipeline is an iterative and complex process that requires a solid understanding of various techniques that can be used in each component of the machine learning pipeline. Feature engineering(FE) is one of the most time-consuming steps in building machine learning pipelines. It requires a deep understanding of the domain and data exploration to discover relevant hand-crafted features from raw data. In this work, we empirically evaluate the impact of integrating an automated feature extraction tool (AutoFeat) into two automated machine learning frameworks, namely, Auto-Sklearn and TPOT, on their predictive performance. Besides, we discuss the limitations of AutoFeat that need to be addressed in order to improve the predictive performance of the automated machine learning frameworks on real-world datasets.

An Empirical Analysis of Integrating Feature Extraction to Automated Machine Learning Pipeline

Knowledge Tracing (KT) is the task of tracing evolving knowledge state of each student as (s)he engages with a sequence of learning activities and can provide personalized instructions. However, exiting methods such as Bayesian Knowledge Tracing (BKT) and Deep Knowledge Tracing (DKT) ei-ther cannot capture the relationship among different concepts or lack of in-terpretability. Although Knowledge Tracing Machines (KTM) makes up for these shortcomings, it only uses a linear function to model students' knowledge states, which cannot capture more information contained in each feature. To solve these problems, this work introduces a novel model called Input-aware Neural Knowledge Tracing Machine (INKTM). Unlike standard FM-based methods that focus on the feature interactions, our model focuses more on the information contained in each feature itself. INKTM converts the weight of each feature to a multidimensional vector, which can be used to learn a unique weight for each feature in different instances by an atten-tion network, so as to highlight the most important features and enhance its interpretability. What’s more, it can capture the non-linear and complex in-herent structure of real-world data through deep neural networks. Especial-ly, when one problem is related with multiple knowledge components, our model can capture more useful information than other FM-based methods by retaining all information of 2-order feature interactions and distinguish-ing contributions of each features. Extensive experiment results show our model consistently outperform existing models in a range of KT datasets.

Input-aware Neural Knowledge Tracing Machine

Training machine learning models in a supervised manner
requires vast amounts of labeled data, where the labels are typically
provided by humans manually annotating samples using a variety of
tools. In this work, an intelligent annotation tool is proposed to combine
the fast and efficient labeling capabilities of modern machine learning
models with the reliable and accurate but slow correction capabilities of
human annotators. For the example of instance segmentation in videos,
techniques to interactively condition a model on previously predicted
and manually annotated (corrected) frames are explored to create an
iterative workflow combining the advantages of the intelligent model and
the human annotator. Thereby, the intelligent model conducts the bulk
of the work, performing instance detection, tracking, and segmentation,
and enables the human annotator to correct individual frames selectively
and instances. The proposed approach is based on the one-shot learning
paradigm, and, thus, avoids the computational cost of online retraining.
For this purpose, we use Siamese neural networks to transfer annotations
from one video frame to another. Multiple interaction options regarding
the choice of the additional input data to the neural network, e.g., model
predictions or manual corrections, are explored to refine the labeling
performance of the given model and speed up the annotation process.

Premium content

Post-Hoc Explanation Options for XAI in Deep Learning: The Insight Centre for Data Analytics Perspective

Downloads

Next from ICPR 2020

Jointly Optimize Positive and Negative Saliencies for Black Box Classifiers

Similar lecture

Scientific Document Summarization using Citation Context and Multi-objective Optimization