Italy

This work explores the biases in learning processes based on deep neural network architectures. We analyze how bias affects deep learning processes through a toy example using the MNIST database and a case study in gender detection from face images. We employ two gender detection models based on popular deep neural networks. We present a comprehensive analysis of bias effects when using an unbalanced training dataset on the features learned by the models. We show how bias impacts in the activations of gender detection models based on face images. We finally propose InsideBias, a novel method to detect biased models. InsideBias is based on how the models represent the information instead of how they perform, which is the normal practice in other existing methods for bias detection. Our strategy with InsideBias allows to detect biased models with very few samples (only 15 images in our case study). Our experiments include 72K face images from 24K identities and 3 ethnic groups.

ICPR 2020

InsideBias: Measuring Bias in Deep Networks and Application to Face Gender Biometrics

neural activations

face

biometrics

deep learning

bias

detection

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster

Image-text matching has made great progresses recently, but there still remains challenges in fine-grained matching. To deal with this problem, we propose an Improved Visual Semantic Reasoning model (VSR++), which jointly models 1)
global alignment between images and texts and 2) local correspondence between regions and words in a unified framework. To exploit their complementary advantages, we also develop a suitable learning strategy to balance their relative importance. As a result, our model can distinguish image regions and text words in a fine-grained level, and thus achieves the current state-of-the-art performance on two benchmark datasets.

VSR++: Improving Visual Semantic Reasoning for Fine-Grained Image-Text Matching

Current compressed video action recognition methods are mainly based on complete data. However, in a real transmission scenario, the compressed video packets are usually disorderly received and even lost due to network jitters or
congestion. To recognize actions in early phases with limited packets, e.g. for quickly forecasting possible potential risks, in this paper, we propose a Temporal Enhanced Multi-Stream Network (TEMSN) towards practical compressed video action
recognition. First, we make use of three modalities in the compressed domain as complementary cues and build a multistream network to capture rich information from compressed video packets. Second, we design a temporal enhanced module
based on an Encoder-Decoder structure, which is applied to each stream to infer missing packets, generating more accurate action dynamics. Thanks to the multiple modalities and their temporal enhancement, our approach better models actions with partial available compressed video packets. Experiments on the HMDB-51 and UCF-101 datasets validate its effectiveness and efficiency.

Towards Practical Compressed Video Action Recognition: A
Temporal Enhanced Multi-Stream Network


This paper strives for action recognition and detection in video modalities like RGB, depth maps or 3D-skeleton sequences when only limited modality-specific labeled examples
are available. For the RGB, and derived optical-flow, modality
many large-scale labeled datasets have been made available. They
have become the de facto pre-training choice when recognizing
or detecting new actions from RGB datasets that have limited
amounts of labeled examples available. Unfortunately, large-scale
labeled action datasets for other modalities are unavailable for
pre-training. In this paper, our goal is to recognize actions from
limited examples in non-RGB video modalities, by learning from
large-scale labeled RGB data. To this end, we propose a two-step
training process: (i) we extract action representation knowledge
from an RGB-trained teacher network and adapt it to a nonRGB student network. (ii) we then fine-tune the transfer model
with available labeled examples of the target modality. For the
knowledge transfer we introduce feature-supervision strategies,
which rely on unlabeled pairs of two modalities (the RGB and
the target modality) to transfer feature level representations from
the teacher to the student network. Ablations and generalizations
with two RGB source datasets and two non-RGB target datasets
demonstrate that an optical-flow teacher provides better action
transfer features than RGB for both depth maps and 3Dskeletons, even when evaluated on a different target domain, or
for a different task. Compared to alternative cross-modal action
transfer methods we show a good improvement in performance
especially when labeled non-RGB examples to learn from are
scarce.

Feature-Supervised Action Modality Transfer

Feature point matching is an important problem because its applications cover a wide range of tasks in computer vision. Deep learning-based methods for learning local features have recently shown superior performance. However, it is not easy to collect the training data in these methods, especially in cross-spectral settings such as the correspondence between RGB and near-infrared images. In this paper, we propose an unsupervised learning method for general feature point matching. Because we train a convolutional neural network as a feature extractor in order to satisfy the cycle consistency of the correspondences between an input image pair, the proposed method does not require supervision and works even in cross-spectral settings. In our experiments, we apply the proposed method to stereo matching, which is a dense feature point matching problem. The experimental results, which simulate cross-spectral settings with three different settings, i.e., RGB stereo, RGB vs gray-scale, and anaglyph (red vs cyan), show that our proposed method outperforms the compared methods, which employ handcrafted features for stereo matching, by a significant margin.

Feature Point Matching in Cross-Spectral Images with Cycle Consistency Learning

Generative Adversarial Network (GAN) has been widely used for object attribute editing. However, the semantic correlation, resulted from the feature map interaction in the generative network of GAN, may impair the generalization ability of the generative network. In this work, semantic disentanglement is introduced in GAN to reduce the attribute correlation. The feature maps of the generative network are first grouped with an efficient clustering algorithm based on hash encoding, which are used to excavate hidden semantic attributes and calculate
the group-wise orthogonality loss for the reduction of attribute entanglement.
Meanwhile, the feature maps falling in the intersection regions of different groups are further suppressed to reduce the attribute-wise interaction.
Extensive experiments reveal that the proposed GAN generated more genuine objects than the state of the arts. Quantitative results of classification accuracy, inception score and FID score further justify the effectiveness of the proposed GAN.

Group-wise Feature Orthogonalization and Suppression for GAN based Facial Attribute Translation



Malware classification helps to understand its purpose and is also an important part of attack detection. And it is also an important part of discovering attacks. Due to
continuous innovation and development of artificial intelligence, it is a trend to combine deep learning with malware classification. In this paper, we propose an improved malware image rescaling algorithm (IMIR) based on local mean algorithm. Its main goal of IMIR is to reduce the loss of information from samples during
the process of converting binary files to image files. Therefore, we construct a neural network structure based on VGG model, which is suitable for image classification. In the real world, a mass of malware family labels are inaccurate or lacking. To deal with this situation, we propose a novel method to train the deep neural network by Semi-supervised Generative Adversarial Network (SGAN), which only needs a small amount of malware that have accurate labels about families. By integrating SGAN
with weak coupling, we can retain the weak links of supervised part and unsupervised part of SGAN. It improves the accuracy of malware classification by making classifiers more independent of discriminators. The results of experimental demonstrate that our model achieves exhibiting favorable performance. The recalls
of each family in our data set are all higher than 93.75%.

A Weak Coupling of Semi-Supervised Learning with Generative Adversarial Networks for Malware Classification


Object detection in Earth Vision has achieved great progress in recent years. However, tiny object detection in aerial images remains a very challenging problem since the tiny objects contain a small number of pixels and are easily confused with
the background. To advance tiny object detection research in aerial images, we present a new dataset for Tiny Object Detection in Aerial Images (AI-TOD). Specifically, AI-TOD comes with 700,621 object instances for eight categories across 28,036 aerial images. Compared to existing object detection datasets in aerial images, the mean size of objects in AI-TOD is about 12.8 pixels, which is much smaller than others. To build a benchmark for tiny object detection in aerial images, we evaluate the state-of-the-art object detectors on our AI-TOD dataset. Experimental results show that direct application of these approaches on AI-TOD
produces suboptimal object detection results, thus new specialized detectors for tiny object detection need to be designed. Therefore, we propose a multiple center points based learning network (M-CenterNet) to improve the localization performance of tiny object detection, and experimental results show the significant
performance gain over the competitors.

Tiny Object Detection in Aerial Images

Skeleton-based action recognition task has drawn
much attention for many years. Graph Convolutional Network
(GCN) has proved its effectiveness in this task. However, how to
improve the model’s robustness to different human actions and
how to make effective use of features produced by the network
are main topics needed to be further explored. Human actions
are time series sequence, meaning that temporal information is
a key factor to model the representation of data. The ranges
of body parts involved in small actions (e.g. raise a glass or
shake head) and big actions (e.g. walking or jumping) are
diverse. It’s crucial for the model to generate and utilize more
features that can be adaptive to a wider range of actions.
Furthermore, feature channels are specific with the action class,
the model needs to weigh their importance and pay attention
to more related ones. To address these problems, in this work,
we propose a two-stream channel-wise dense connection GCN
(2s-CDGCN). Specifically, the skeleton data was extracted and
processed into spatial and temporal information for better feature
representation. A channel-wise attention module was used to
select and emphasize the more useful features generated by
the network. Moreover, to ensure maximum information flow,
dense connection was introduced to the network structure,
which enables the network to reuse the skeleton features and
generate more information adaptive and related to different
human actions. Our model has shown its ability to improve the
accuracy of human action recognition task on two large datasets,
NTU-RGB+D and Kinetics. Extensive evaluations were conducted
to prove the effectiveness of our model.

Channel-Wise Dense Connection Graph Convolutional Network For Skeleton-Based Action Recognition

Foreground segmentation in the dynamic scenes is an important task in video surveillance. The unsupervised background subtraction method based on
background statistics modeling has difficulties in updating. On the other hand, the supervised foreground segmentation method based on deep learning relies on the large-scale of accurately annotated training data, which limits its cross-scene performance. In this paper, we propose a foreground segmentation method from coarse to fine. First, an across-scenes trained Spatio-Temporal Attention Model (STAM) is used to achieve coarse segmentation, which does not require training on a specific scene. Then the coarse segmentation is used as a reference to help Co-occurrence Pixel-Block Model (CPB) complete the fine segmentation and at the same time help CPB to update its background model. This method is more flexible than those deep learning-based methods which depend on the specific scene training and realizes the accurate online dynamic update of the background model. Experimental results on WallFlower and LIMU validate our method outperforms STAM, CPB, and other methods of participating in comparison.

Coarse-to-fine Foreground Segmentation based on Co-occurrence Pixel-Block and Spatio-Temporal Attention Model

Beyond assigning the correct class, an activity recognition model should also be able to determine, how certain it is in its predictions. We present the first study of how well the confidence values of modern action recognition architectures indeed reflect the probability of the correct outcome and propose a learning-based approach for improving it. First, we extend two popular action recognition datasets with a reliability benchmark in form of the expected calibration error and reliability diagrams. Since our evaluation highlights that confidence values of standard action recognition architectures do not represent the uncertainty well, we introduce a new approach which learns to transform the model output into realistic confidence estimates through an additional calibration network. The main idea of our Calibrated Action Recognition with Input Guidance (CARING) model is to learn an optimal scaling parameter depending on the video representation. We compare our model with the native action recognition networks and the temperature scaling approach - a wide spread calibration method utilized in image classification. While temperature scaling alone drastically improves the reliability of the confidence values, our CARING method consistently leads to the best uncertainty estimates in all benchmark settings.

Premium content

InsideBias: Measuring Bias in Deep Networks and Application to Face Gender Biometrics

Downloads

Next from ICPR 2020

VSR++: Improving Visual Semantic Reasoning for Fine-Grained Image-Text Matching

Similar lecture

Human or Machine? It Is Not What You Write, But How You Write It