Italy


This presentation introduces a fully convolutional network for the document layout analysis task. While state-of-the-art methods are using models pre-trained on natural scene images, our method Doc-UFCN relies on a U-shaped model trained from scratch for detecting objects from historical documents. We show that Doc-UFCN outperforms a state-of-the-art method on various datasets and also demonstrate that the pre-trained parts on natural scene images are not required to reach good results. In addition, we show that pre-training on multiple document datasets can improve the performances.

ICPR 2020

Multiple Document Datasets Pre-training Improves Text Line Detection With Deep Neural Networks

text line segmentation

historical documents

document layout analysis

fully convolutional network

deep learning

pre-training

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


technical paper


We propose a CNN-based approach to classify ten genres of ballroom dances given audio recordings, five latin and five standard, namely Cha Cha Cha, Jive, Paso Doble, Rumba, Samba, Quickstep, Slow Foxtrot, Slow Waltz, Tango and Viennese Waltz. We utilize a spectrogram of an audio signal and we treat it as an image that is an input of the CNN. The classification is performed independently by 5-seconds spectrogram segments in sliding window fashion and the results are then aggregated. The method was tested on following datasets: Publicly available Extended Ballroom dataset collected by Marchand and Peeters, 2016 and two YouTube datasets collected by us, one in studio quality and the other, more challenging, recorded on mobile phones. The method achieved accuracy 93.9%, 96.7% and 89.8% respectively. The method runs in real-time. We implemented a web application to demonstrate the proposed method.

Ballroom Dance Recognition from Audio Recordings

Stochastic gradient descent (SGD) usually samples training data based on the uniform distribution, which may not be a good choice because of the high variance of its stochastic gradient. Thus, importance sampling methods are considered in the literature to improve the performance. Most previous work on SGD-based methods with importance sampling requires the knowledge of Lipschitz constants of all component gradients, which are in general difficult to estimate. In this paper, we study an adaptive importance sampling method for common SGD-based methods by exploiting the local first-order information without knowing any Lipschitz constants. In particular, we periodically changes the sampling distribution by only utilizing the gradient norms in the past few iterations. We prove that our adaptive importance sampling non-asymptotically reduces the variance of the stochastic gradients in SGD, and thus better convergence bounds than that for vanilla SGD can be obtained. We extend this sampling method to several other widely used stochastic gradient algorithms including SGD with momentum and ADAM. Experiments on common convex learning problems and deep neural networks illustrate notably enhanced performance using the adaptive sampling strategy.

Low-Cost Lipschitz-Independent Adaptive Importance Sampling of Stochastic Gradients


This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

Feature Pyramid Hierarchies for Multi-scale Temporal Action Detection


Manga character recognition is a key technology for manga character retrieval and verification. This task is very challenging since the manga character images have a long-tailed distribution and large quality variations. Training models with cross-entropy softmax loss on such imbalanced data would introduce biases to feature and class weight norms. To handle this problem, we propose a novel dual loss which is the sum of two losses: dual ring loss and dual adaptive re-weighting loss. Dual ring loss combines weight and feature soft normalization and serves as a regularization term to softmax loss. Dual adaptive re-weighting loss re-weights softmax loss according to the norm of both feature and class weight. With the proposed losses, we have achieved encouraging results on the Manga109 benchmark. Specifically, compared with the baseline softmax loss, our method improves the character retrieval mAP from 35.72% to 38.88% and the character verification accuracy from 87.00% to 88.50%.

Dual Loss for Manga Character Recognition with Imbalanced Training Data

Towards multi-class novelty detection, we propose an end-to-end trainable Parallel Network (PN) using no additional data but only the training set. Our key idea is to first divide the training set into successive subtasks of pseudo-novelty detection to simulate real scenarios. We then design PN which consists of multiple branches to address the pseudo-tasks. This yields a more compact and discriminative classification space and forms a natural ensemble. In practice, we sample a random subset of the training classes as pseudo-novel and regard the extra as known. The sampling result forms a sub-task fed to one branch of PN. All training classes are equally divided as pseudo-novel to PN's branches for better data balance and model diversity. By distinguishing between the known and the diverse pseudo-novel, PN extracts the concept of novelty in a compact classification space. This provides PN with generalization ability to real novel classes which are absent during training. During online inference, this ability is further strengthened with the ensemble of PN's multiple branches. Experiments on three public datasets show our method's superiority to the mainstream methods.

Parallel Network to Learn Novelty from the Known

Rethinking Experience Replay


In video analysis and public surveillance, information from multiple modalities are used to jointly determine the identity of a person. We propose a correlation-based multimodal person recognition framework that is relatively simple but can efficaciously learn supervised information in multimodal data fusion and resist noise.


Person Recognition with HGR Maximal Correlation on Multimodal data

Clustering is a crucial but challenging task in pattern analysis and machine learning. Recent many deep clustering methods combining representation learning with cluster techniques emerged. These deep clustering methods mainly focus on the correlation among samples and ignore the relationship between samples and their representations. In this paper, we propose a novel end-to-end clustering framework, namely variational deep embedding clustering by augmented mutual information maximization (VCAMI). From the perspective of VAE, we prove that minimizing reconstruction loss is equivalent to maximizing the mutual information of the input and its latent representation. This provides a theoretical guarantee for us to directly maximize the mutual information instead of minimizing reconstruction loss. Therefore we proposed the augmented mutual information which highlights the uniqueness of the representations while discovering invariant information among similar samples. Extensive experiments on several challenging image datasets show that the VCAMI achieves good performance. we achieve state-of-the-art ACC results for clustering on MNIST (99.5\%) and CIFAR-10 (65.4\%) to the best of our knowledge.

Variational Deep Embedding Clustering by Augmented Mutual Information Maximization

Given the wide diffusion of deep neural network architectures for computer vision tasks, several new applications are nowadays more and more feasible. 
Among them, a particular attention has been recently given to instance segmentation, by exploiting the results achievable by two-stage networks (such as Mask R-CNN or Faster R-CNN), derived from R-CNN. 
In these complex architectures, a crucial role is played by the Region of Interest (RoI) extraction layer, devoted to extracting a coherent subset of features from a single Feature Pyramid Network (FPN) layer attached on top of a backbone.

This paper is motivated by the need to overcome the limitations of existing RoI extractors which select only one (the best) layer from FPN. 
Our intuition is that all the layers of FPN retain useful information. 
Therefore, the proposed layer (called Generic RoI Extractor - GRoIE) introduces non-local building blocks and attention mechanisms to boost the performance.

A comprehensive ablation study at component level is conducted to find the best set of algorithms and parameters for the GRoIE layer. 
Moreover, GRoIE can be integrated seamlessly with every two-stage architecture for both object detection and instance segmentation tasks. 
Therefore, the improvements brought about by the use of GRoIE in different state-of-the-art architectures are also evaluated. 
The proposed layer leads up to gain a 1.1% AP improvement on bounding box detection and 1.7% AP improvement on instance segmentation.

The code is publicly available on GitHub repository at https://github.com/IMPLabUniPr/mmdetection/tree/groie_dev

A Novel Region of Interest Extraction Layer for Instance Segmentation


Visual tracking algorithms use cues like appearance, structure, motion etc. for locating an object in a video. We propose an ensemble tracker with two components. First, a Siamese tracker that learns object appearance from a static image. Second, motion information obtained from consecutive frames using a flow estimation network. The motion information is used to correct the predictions obtained by the appearance based tracking component of the ensemble. Complementary nature of the two components (appearance and motion) lead to performance improvement as observed in experiments performed on VOT2018 and VOT2019 datasets.

Premium content

Multiple Document Datasets Pre-training Improves Text Line Detection With Deep Neural Networks

Downloads

Next from ICPR 2020

Ballroom Dance Recognition from Audio Recordings

Similar lecture

Multiple Document Datasets Pre-training Improves Text Line Detection With Deep Neural Networks