Italy

The performance of Convolutional Neural Networks for image classification has vastly and steadily increased during the last years. This success goes hand in hand with the need to explain and understand their decisions: opening the black box. The problem of attribution specifically deals with the characterization of the response of Convolutional Neural Networks by identifying the input features responsible for the model&#39;s decision. Among all attribution methods, perturbation-based methods are an important family based on measuring the effect of perturbations applied to the input image in the model&#39;s output. In this paper, we discuss the limitations of existing approaches and propose a novel perturbation-based attribution method guided by semantic segmentation. Our method inhibits specific image areas according to their assigned semantic label. Hereby, perturbations are link up with a semantic meaning and a complete attribution map is obtained for all image pixels. In addition, we propose a particularization of the proposed method to the scene recognition task which, differently than image classification, requires multi-focus attribution models. The proposed semantic-guided attribution method enables us to delve deeper into scene recognition interpretability by obtaining for each scene class the sets of relevant, irrelevant and distracting semantic labels.

ICPR 2020

AIVisualizing the Effect of Semantic Classes in the Attribution of Scene Recognition Models

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


The performance of Convolutional Neural Networks for image classification has vastly and steadily increased during the last years. This success goes hand in hand with the need to explain and understand their decisions: opening the black box. The problem of attribution specifically deals with the characterization of the response of Convolutional Neural Networks by identifying the input features responsible for the model's decision. Among all attribution methods, perturbation-based methods are an important family based on measuring the effect of perturbations applied to the input image in the model's output. In this paper, we discuss the limitations of existing approaches and propose a novel perturbation-based attribution method guided by semantic segmentation. Our method inhibits specific image areas according to their assigned semantic label. Hereby, perturbations are link up with a semantic meaning and a complete attribution map is obtained for all image pixels. In addition, we propose a particularization of the proposed method to the scene recognition task which, differently than image classification, requires multi-focus attribution models. The proposed semantic-guided attribution method enables us to delve deeper into scene recognition interpretability by obtaining for each scene class the sets of relevant, irrelevant and distracting semantic labels.

technical paper

COVID-19 pandemic has adversely and disproportionatelyimpacted people suffering from mental health issues and substance useproblems. This has been exacerbated by social isolation during the pan-demic and the social stigma associated with mental health and sub-stance use disorders, making people reluctant to share their struggles andseek help. Due to the anonymity and privacy they provide, social mediaemerged as a convenient medium for people to share their experiencesabout their day to day struggles. Reddit is a well-recognized social mediaplatform that provides focused and structured forums called subreddits,that users subscribe to and discuss their experiences with others. Tem-poral assessment of the topical correlation between social media post-ings about mental health/substance use and postings about Coronavirusis crucial to better understand public sentiment on the pandemic andits evolving impact, especially related to vulnerable populations. In thisstudy, we conduct a longitudinal topical analysis of postings between sub-reddits r/depression, r/Anxiety, r/SuicideWatch, and r/Coronavirus, andpostings between subreddits r/opiates, r/OpiatesRecovery, r/addiction,and r/Coronavirus from January 2020 - October 2020. Our results show ahigh topical correlation between postings in r/depression and r/Coronavi-rus in September 2020. Further, the topical correlation between postingson substance use disorders and Coronavirus fluctuates, showing the high-est correlation in August 2020. By monitoring these trends from plat-forms such as Reddit, epidemiologists, and mental health professionalscan gain insights into the challenges faced by communities for targetedinterventions.

COVID-19 and Mental Health/Substance Use Disorders on Reddit: A Longitudinal Study

The objective of this study is to evaluate and compare the performance of a set of low-cost multi-modal head tracking systems incorporating facial gestures as a switching mechanism. The proposed systems are aimed to enable severely disabled patients to access a computer. In this paper, we are comparing RGB (2D) and RGB-D (3D) sensors for both head tracking and facial gesture recognition. System evaluations and usability assessment were carried out on 21 healthy individuals. Two
types of head tracking systems were compared - a web camera-based and another using the Kinect sensor. The two facial switching mechanisms were eye blink and eyebrows movement. Fitts' Test is used to evaluate the proposed systems. Movement Time (MT) was used to rank the performance of the proposed systems. The Kinect-Eyebrows system had the lowest MT, followed by the Kinect-Blink, Webcam-Blink and Webcam-Eyebrows systems. The 3D Kinect systems performed better than the 2D Vision systems for both gestures. Both Kinect systems have the lowest MT and best performance, thus showing the advantage of using depth.

Development and evaluation of a mouse emulator using multi-modal real-time headtracking systems with facial gesture recognition as a switching mechanism

THDL system include THDL-Det sub-system for text detection and THDL-Rec sub-system for text recognition. The THDL-Det sub-system is an end-to-end text spotter based on the Mask R-CNN instance segmentation framework. The THDL-Rec sub-system adopts a CNN-LSTM-CTC framework to recognize Arabic text lines in videos.

TH-DL system for AcTiV2020 Competition

Deep image clustering approaches typically use autoencoder
architectures to learn compressed latent representations suitable for clustering tasks. However, they do not effectively regulate the latent space
during training, leading to low performance and diminished applicability
to different datasets. In this paper, we propose a deep clustering model
combining maximum mean discrepancy (MMD) regularization and self-learning clustering optimization to mitigate this problem. Specifically,
we first train the network to improve its image reconstruction ability
by minimizing both reconstruction loss and MMD divergence from a
target distribution. Then, the model gradually learns from its own high-confidence predictions to further optimize the latent distribution. We
validate the network’s performance on different benchmark image sets
using standard clustering metrics, without changing network configuration or adjusting hyper-parameters between datasets. The proposed
model provides top clustering performance across datasets while being
more robust than state-of-the-art methods.

Deep Image Clustering Using Self-Learning Optimization in a Variational Auto-Encoder

To facilitate deep learning based table extraction from unstructured digital files, we disclose a standard Chinese financial dataset from PDF files for table extraction benchmark test, which is diverse, sufficient and comprehensive. We also propose a GCN-based algorithm GFTE as a baseline with a novel idea of integrating all possible types of ground truth together.

GFTE: Graph-based Financial Table Extraction

Relative attributes indicate the strength of a particular attribute between image pairs. We introduce a deep Siamese network with rank SVM loss function, called Deep-RankSVM, that can decide which one of a pair of images has a stronger presence of a specific attribute. The network is trained in an end-to-end fashion to jointly learn the visual features and the ranking function. The trained network for an attribute can predict the relative strength of that attribute in novel images. We demonstrate the effectiveness of our approach against the state-of-the-art methods on four image benchmark datasets: LFW-10, PubFig, UTZap50K-2 and UTZap50K-lexi datasets. Deep-RankSVM surpasses state-of-art in terms of the average accuracy across attributes, on three of the four image benchmark datasets.

Relative Attribute Classification with Deep-RankSVM

In this paper, we propose the research of the recursive division approach to get an explanation for a particular decision of the shallow black box model. The core of the proposed method is the division of the image being classified into separate rectangular parts, followed by the analysis of their influence on the classification result. Such divisions are repeated recursively until the explanation of the classification result is found, or the size of parts is too small. As a result, the pair of images with complement hidden parts is discovered, the first one of which preserves both the most valuable parts and the classification result of the initial image. The second image represents the result of hiding the most valuable parts of an initial image that leads to the different classification for the binary classification problem. Experimental research (applied for Food-5K and concrete crack images datasets) proved that the quality of the proposed method might be close to LIME or even better, while the performance of recursive division is better.

Recursive Division of Image for Explanation of Shallow CNN Models

Deep Neural Networks are becoming the prominet solution
when using machine learning models. However, they suffer of a black-
box effect that renders complicated their inner workings interpretation
and thus the understanding of their successes and failures. Information
visualisation is one way among others to help in their interpretability
and hypothesis deduction. This paper presents a novel way to visualize
a trained DNN in order to depict at the same time its architecture
and its way of treating the classes of a test dataset at the layer level.
This way, it is possible to visually detect where the DNN starts to be
able to discriminate the classes or where it could decrease its separation
ability (and thus detect an oversized network). We have implemented the
approach and validated it using several well-known datasets and networks.
Results show the approach is promising and deserves further studies.

Samples Classification Analysis Across DNN Layers with Fractal Curves

Explaining the behaviors of deep neural networks, usually considered as black boxes, is critical especially when they are now being adopted over diverse aspects of human life. Taking the advantages of interpretable machine learning (interpretable ML), this paper proposes a novel tool called Catastrophic Forgetting Dissector (or CFD) to explain catastrophic forgetting in continual learning settings. We also introduce a new method called Critical Freezing based on the observations of our tool. Experiments on ResNet-50 articulate how catastrophic forgetting happens, particularly showing which components of this famous network are forgetting. Our new continual learning algorithm defeats various recent techniques by a significant margin, proving the capability of the investigation. Critical freezing not only attacks catastrophic forgetting but also exposes explainability.

Explaining How Deep Neural Networks Forget by Deep Visualization

In this talk we present our work on a perplexity-based sparsity definition to derive and visualise layer-wise activation measures. These novel explainable AI strategies reveal a surprising relationship between activation sparsity and overfitting. Our methods reveal novel insights into the contradicting concepts of activation sparsity and network capacity by demonstrating that dense activations can enable discriminative feature learning while efficiently exploiting the capacity of deep models without suffering from overfitting, even when trained excessively.

Premium content

AIVisualizing the Effect of Semantic Classes in the Attribution of Scene Recognition Models

Downloads

Next from ICPR 2020

COVID-19 and Mental Health/Substance Use Disorders on Reddit: A Longitudinal Study

Similar lecture

Efficient correlation filter tracking with adaptive training sample update scheme