Italy


Manga character recognition is a key technology for manga character retrieval and verification. This task is very challenging since the manga character images have a long-tailed distribution and large quality variations. Training models with cross-entropy softmax loss on such imbalanced data would introduce biases to feature and class weight norms. To handle this problem, we propose a novel dual loss which is the sum of two losses: dual ring loss and dual adaptive re-weighting loss. Dual ring loss combines weight and feature soft normalization and serves as a regularization term to softmax loss. Dual adaptive re-weighting loss re-weights softmax loss according to the norm of both feature and class weight. With the proposed losses, we have achieved encouraging results on the Manga109 benchmark. Specifically, compared with the baseline softmax loss, our method improves the character retrieval mAP from 35.72% to 38.88% and the character verification accuracy from 87.00% to 88.50%.

ICPR 2020

Dual Loss for Manga Character Recognition with Imbalanced Training Data

imbalanced data

cross-entropy softmax loss

manga character

recognition

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster

Towards multi-class novelty detection, we propose an end-to-end trainable Parallel Network (PN) using no additional data but only the training set. Our key idea is to first divide the training set into successive subtasks of pseudo-novelty detection to simulate real scenarios. We then design PN which consists of multiple branches to address the pseudo-tasks. This yields a more compact and discriminative classification space and forms a natural ensemble. In practice, we sample a random subset of the training classes as pseudo-novel and regard the extra as known. The sampling result forms a sub-task fed to one branch of PN. All training classes are equally divided as pseudo-novel to PN's branches for better data balance and model diversity. By distinguishing between the known and the diverse pseudo-novel, PN extracts the concept of novelty in a compact classification space. This provides PN with generalization ability to real novel classes which are absent during training. During online inference, this ability is further strengthened with the ensemble of PN's multiple branches. Experiments on three public datasets show our method's superiority to the mainstream methods.

Parallel Network to Learn Novelty from the Known


This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

Rethinking Experience Replay


In video analysis and public surveillance, information from multiple modalities are used to jointly determine the identity of a person. We propose a correlation-based multimodal person recognition framework that is relatively simple but can efficaciously learn supervised information in multimodal data fusion and resist noise.


Person Recognition with HGR Maximal Correlation on Multimodal data

Clustering is a crucial but challenging task in pattern analysis and machine learning. Recent many deep clustering methods combining representation learning with cluster techniques emerged. These deep clustering methods mainly focus on the correlation among samples and ignore the relationship between samples and their representations. In this paper, we propose a novel end-to-end clustering framework, namely variational deep embedding clustering by augmented mutual information maximization (VCAMI). From the perspective of VAE, we prove that minimizing reconstruction loss is equivalent to maximizing the mutual information of the input and its latent representation. This provides a theoretical guarantee for us to directly maximize the mutual information instead of minimizing reconstruction loss. Therefore we proposed the augmented mutual information which highlights the uniqueness of the representations while discovering invariant information among similar samples. Extensive experiments on several challenging image datasets show that the VCAMI achieves good performance. we achieve state-of-the-art ACC results for clustering on MNIST (99.5\%) and CIFAR-10 (65.4\%) to the best of our knowledge.

Variational Deep Embedding Clustering by Augmented Mutual Information Maximization

Given the wide diffusion of deep neural network architectures for computer vision tasks, several new applications are nowadays more and more feasible. 
Among them, a particular attention has been recently given to instance segmentation, by exploiting the results achievable by two-stage networks (such as Mask R-CNN or Faster R-CNN), derived from R-CNN. 
In these complex architectures, a crucial role is played by the Region of Interest (RoI) extraction layer, devoted to extracting a coherent subset of features from a single Feature Pyramid Network (FPN) layer attached on top of a backbone.

This paper is motivated by the need to overcome the limitations of existing RoI extractors which select only one (the best) layer from FPN. 
Our intuition is that all the layers of FPN retain useful information. 
Therefore, the proposed layer (called Generic RoI Extractor - GRoIE) introduces non-local building blocks and attention mechanisms to boost the performance.

A comprehensive ablation study at component level is conducted to find the best set of algorithms and parameters for the GRoIE layer. 
Moreover, GRoIE can be integrated seamlessly with every two-stage architecture for both object detection and instance segmentation tasks. 
Therefore, the improvements brought about by the use of GRoIE in different state-of-the-art architectures are also evaluated. 
The proposed layer leads up to gain a 1.1% AP improvement on bounding box detection and 1.7% AP improvement on instance segmentation.

The code is publicly available on GitHub repository at https://github.com/IMPLabUniPr/mmdetection/tree/groie_dev

A Novel Region of Interest Extraction Layer for Instance Segmentation


Visual tracking algorithms use cues like appearance, structure, motion etc. for locating an object in a video. We propose an ensemble tracker with two components. First, a Siamese tracker that learns object appearance from a static image. Second, motion information obtained from consecutive frames using a flow estimation network. The motion information is used to correct the predictions obtained by the appearance based tracking component of the ensemble. Complementary nature of the two components (appearance and motion) lead to performance improvement as observed in experiments performed on VOT2018 and VOT2019 datasets.

Siamese Fully Convolutional Tracker with Motion Correction

Generating images from the sound is a challenging task. This paper proposes a novel deep learning model that generates bird images from their corresponding sound information. Our proposed model includes a sound encoder in order to extract suitable feature representations from audio recordings, and then it generates bird images that correspond to its calls using conditional generative adversarial networks (GANs) with auxiliary classifiers. We demonstrate that our model produces better image generation results which outperform other state-of-the-art methods in a similar context.

S2I-Bird: Sound-to-Image Generation of Bird Species using Generative Adversarial Networks

Transceivers used for telecommunications transmit and receive specific modulation patterns that are represented as sequences of complex numbers. Classifying modulation patterns is challenging because noise and channel impairments affect the signals in complicated ways such that the received signal bears little resemblance to the transmitted signal. Although deep learning approaches have shown great promise over statistical methods in this problem space, deep learning frameworks continue to lag in support for complex-valued data. To address this gap, we study the implementation and use of complex convolutions in a series of convolutional neural network architectures. Replacement of data structure and convolution operations by their complex generalization in an architecture improves performance, with statistical significance, at recognizing modulation patterns in complex-valued signals with high SNR after being trained on low SNR signals. This suggests complex-valued convolutions enables networks to learn more meaningful representations. We investigate this hypothesis by comparing the features learned in each experiment by visualizing the inputs that results in one-hot modulation pattern classification for each network.

Modulation Pattern detection Using Complex Convolutions in Deep Learning


Visual patterns usually appear at different
scales/sizes in natural images. Multi-scale feature representation
is of great importance for the single-image super-resolution
(SISR) task to reconstruct image objects at different scales.
However, such characteristic has been rarely considered by CNNbased
SISR methods. In this work, we propose a novel building
block, i.e. hierarchically aggregated residual transformation
(HART), to achieve multi-scale feature representation in each
layer of the network. Within each HART block, we connect
multiple convolutions in a hierarchical residual-like manner,
which efficiently provides a wide range of effective receptive fields
at a more granular level to detect both local and global image
features. To theoretically understand the proposed HART block,
we recast SISR as an optimal control problem and show that
HART effectively approximates the classical 4th-order Runge-
Kutta method, which has the merit of small local truncation
error for solving numerical ordinary differential equation. By
cascading the proposed HART blocks, we establish our highperforming
HARTnet. Through extensive experiments on various
benchmark datasets under different degradation models, we
demonstrate that HARTnet compares favourably against existing
state-of-the-art methods (including those in the NTIRE 2019 SR
Challenge leaderboard) in terms of both quantitative metrics
and visual quality. Moreover, the same HARTnet architecture
achieves promising performance on such other image restoration
tasks as image denoising and low-light image enhancement.

Hierarchically Aggregated Residual Transformation
for Single Image Super Resolution


This presentation introduces our proposed paper "Label Self-Adaption Hashing for Image Retrieval". We will give the background and contribution of our work at first. Then, the related works are briefly reviewed. Motivated by previous works, we propose a novel method for unsupervised hashing and give its optimization objective. An iterative algorithm is well-designed to solve the resulting optimization problem. Extensive experiments demonstrate the effectiveness of the proposed method.

Premium content

Dual Loss for Manga Character Recognition with Imbalanced Training Data

Downloads

Next from ICPR 2020

Parallel Network to Learn Novelty from the Known

Similar lecture

ReADS: A Rectified Attentional Double Supervised Network for Scene Text Recognition