Italy


Gaining control over the dynamics of multi-task learning should help to unlock the potential of the deep network to a great extent. In the existing multi-task learning (MTL) approaches of a deep network, all the parameters of its feature encoding part are subjected to adjustments corresponding to each of the underlying sub-tasks. On the other hand, different functional areas of the human brain are responsible for distinct functions such as the Broca&#39;s area of the cerebrum is responsible for speech formation whereas its Wernicke&#39;s area is related to the language development etc. Inspired by this fact, in the present study, we used here a Feature Representation Block (FRB) of connection weights spanned over a few successive layers of a deep multi-task learning architecture and stratify the same into distinct subsets for their adjustments exclusively corresponding to different sub-tasks. Additionally, we have introduced a novel regularization component for controlled training of this Feature Representation Block. The purpose of the development of this learning framework is efficient end-to-end recognition of scene texts.

ICPR 2020

Stratified Multi-Task Learning for Robust Spotting of Scene Texts

scene text spotting

deep learning

multi-task learning

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 



Gaining control over the dynamics of multi-task learning should help to unlock the potential of the deep network to a great extent. In the existing multi-task learning (MTL) approaches of a deep network, all the parameters of its feature encoding part are subjected to adjustments corresponding to each of the underlying sub-tasks. On the other hand, different functional areas of the human brain are responsible for distinct functions such as the Broca's area of the cerebrum is responsible for speech formation whereas its Wernicke's area is related to the language development etc. Inspired by this fact, in the present study, we used here a Feature Representation Block (FRB) of connection weights spanned over a few successive layers of a deep multi-task learning architecture and stratify the same into distinct subsets for their adjustments exclusively corresponding to different sub-tasks. Additionally, we have introduced a novel regularization component for controlled training of this Feature Representation Block. The purpose of the development of this learning framework is efficient end-to-end recognition of scene texts.

technical paper


We propose a method for Offline Handwritten Mathematical Expression recognition. The method is a fast and accurate thanks to its architecture, which include both a Convolutional Neural Network and a Recurrent Neural Network. The CNN extracts features from the image to recognize and its output is provided to the RNN which produces the mathematical expression encoded in the LaTeX language.
To process both sequential and non-sequential mathematical expressions we also included a deconvolutional module which, in a recursive way, segments the image for additional analysis trough a recursive process. The results obtained show a very high accuracy obtained on a large handwritten data set of 9100 samples of handwritten expressions.

Recursive Recognition of Offline Handwritten Mathematical Expressions

In the computational prediction of chemical compound properties, molecular descriptors and fingerprints encoded to low dimensional vectors are used. The selection of proper molecular descriptors and fingerprints is both important and challenging as the performance of such models is highly dependent on descriptors. To overcome this challenge, natural language processing models that utilize simplified molecular input lineentry system as input were studied, and several transformer variant models achieved superior results when compared with conventional methods. In this study, we explored the structural differences of the transformer-variant model and proposed a new self-attention based model. The representation learning performance of the self-attention module was evaluated in a multi-task learning environment using imbalanced chemical datasets. The experiment results showed that our model achieved competitive outcomes on several benchmark datasets. The source code of our experiment is available at https://github.com/arwhirang/samtl and the dataset is available from the same URL.

Predicting Chemical Properties using Self-Attention Multi-task Learning based on SMILES Representation


This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

Appearance Features and Graph Representation


Content-based classification of manuscripts is an important task that is generally performed in archives and libraries
by experts with a wealth of knowledge on the manuscript’s
contents. Unfortunately, many manuscript collections are so vast
that it is not feasible to rely solely on experts to perform this
task. Current approaches for textual-content-based manuscript
classification generally require the handwritten images to be
first transcribed into text – but achieving sufficiently accurate
transcripts are generally unfeasible for large sets of historical
manuscripts. We propose a new approach to perform automatically this classification task which does not rely on any explicit
image transcripts. It is based on “probabilistic indexing”, a
relatively novel technology which allows to effectively represent
the intrinsic word-level uncertainty generally exhibited by handwritten text images. We assess the performance of this approach
on a large collection of complex manuscripts from the Spanish
Archivo General de Indias, with promising results. To the best
of our knowledge, this is the first published work proposing,
developing and assessing a successful approach for content-based
classification of untranscribed manuscript images.

Textual-Content-Based Classification of Bundles of Untranscribed Manuscript Images

Accurate localization is a foundational capacity, required for autonomous vehicles to accomplish other tasks such as navigation or path planning. It is a common practice for vehicles to use GPS to acquire location information. However, the application of GPS can result in severe challenges when vehicles run within the inner city where different kinds of structures may shadow the GPS signal and lead to inaccurate location results. To address the localization challenges of urban settings, we propose a novel feature voting technique for visual localization. Different from the conventional front-view-based method, our approach employs views from three directions (front, left, and right) and thus significantly improves the robustness of location prediction. In our work, we craft the proposed feature voting method into three state-of-the-art visual localization networks and modify their architectures properly so that they can be applied for vehicular operation. Extensive field test results indicate that our approach can predict location robustly even in challenging inner-city settings. Our research sheds light on using the visual localization approach to help autonomous vehicles to find accurate location information in a city maze, within a desirable time constraint.

Visual Localization for Autonomous Driving: Mapping the Accurate Location in the City Maze


In this paper we introduce a novel end-to-end blind inpainting algorithm for masonry wall images, performing the automatic detection and virtual completion of occluded or damaged wall regions.

A GAN based Blind Inpainting Method for Masonry Wall Images

Visual pose regression models estimate the camera pose from a query image with a single forward pass. Current models learn pose encoding from an image using deep convolutional networks which are trained per scene. The resulting encoding is typically passed to a multi-layer perceptron in order to regress the pose. In this work, we propose that scene-specific pose encoders are not required for pose regression and that encodings trained for visual similarity can be used instead. In
order to test our hypothesis, we take a shallow architecture of several fully connected layers and train it with pre-computed encodings from a generic image retrieval model. We find that these encodings are not only sufficient to regress the camera pose, but that, when provided to a branching fully connected architecture, a trained model can achieve competitive results and even surpass current state-of-the-art pose regressors in some cases. Moreover, we show that for outdoor localization, the proposed architecture is the only pose regressor, to date, consistently localizing in under 2 meters and 5 degrees.

Do We Really Need Scene-specific Pose Encoders?


Electrocardiogram (ECG) signal is studied to obtain crucial information about the condition of a patient’s heart. Machine learning based automated medical diagnostic systems that may help to evaluate the condition of the heart from this signal are required to be trained using large volumes of labelled training samples and the same may increase the chance of compromising with the patients’ privacy. To solve this issue, generation of synthetic electrocardiogram signals by learning only from the general distributions of the available real training samples have been attempted in the literature. However, these studies did not pay necessary attention to the specific vital details of these signals, such as the P wave, the QRS complex, and the T wave. This shortcoming often results in the generation of unrealistic synthetic signals, such as a signal which does not contain one or more of the above components. In the present study, a novel deep generative architecture, termed as CardioGAN, based on generative adversarial network and powered by the effective attention mechanism has been designed which is capable of learning the intricate inter-dependencies among the various parts of real samples leading to the generation
of more realistic electrocardiogram signals. Also, it helps in reducing the risk of breaching the privacy of patients. Extensive experimentation performed by us establishes that the proposed method achieves a better performance in generating synthetic electrocardiogram signals in comparison to the existing methods.


CardioGAN: An Attention-based Generative Adversarial Network for Generation of Electrocardiograms

Moving object detection in a video plays an important role in many vision applications. Recently, moving object detection using appearance modeling based on a convolutional neural network has been actively developed. However, the CNN- based methods usually require the user’s supervision of the first frame so that it becomes highly dependent on the training dataset. In contrast, the method of finding a foreground, which models a background occupying a large proportion in an image, can detect a moving object efficiently in an unsupervised manner. However, existing methods based on background modeling in a pan-tilt-zoom (PTZ) camera suffer many false positives or loss of moving objects due to the estimation error of camera motion. To overcome the aforementioned limitations, we propose a moving object detection method for a PTZ camera through two background models. In an unsupervised way, our method builds the two background models that have different roles: 1) a coarse background model for detecting large changes, and 2) a fine background model for detecting small changes. In more detail, the coarse background model builds a block-based Gaussian model, and the fine model builds a sample consensus model. Both models are adaptively updated according to the estimated camera motion in the video recorded by a PTZ camera. Then, each foreground result from two background models is incorporated to fill the moving object region. Through experiments, the proposed method achieves better performance than the state-of-the-art methods and operates in real-time without parallel processing. In addition, we showed the effectiveness of the proposed model through improved results of moving object detection through combination with the latest supervised method.

Unsupervised Moving Object Detection through Background Models for PTZ Camera

Multi-scale feature fusion has been an effective way for improving the performance of semantic segmentation. How- ever, current methods generally fail to consider the semantic gaps between the shallow (low-level) and deep (high-level) features and thus the fusion methods may not be optimal. In this paper, to address the issues of the semantic gap between the feature from different layers, we propose a unified framework based on the U-shape encoder-decoder architecture, named Enhanced Feature Pyramid Network (EFPN). Specifically, the semantic enhancement module (SEM), edge extraction module (EEM), and context aggregation model (CAM) are incorporated into the decoder network to improve the robustness of the multi- level features aggregation. In addition, a global fusion model (GFM), which in the encoder branch is proposed to capture more semantic information in the deep layers and effectively transmit the high-level semantic features to each layer. Extensive experiments are conducted and the results show that the proposed framework achieves the state-of-the-art results on three public datasets, namely PASCAL VOC 2012, Cityscapes, and PASCAL Context. Furthermore, we also demonstrate that the proposed method is effective for other visual tasks that require frequent fusing features and upsampling.

Premium content

Stratified Multi-Task Learning for Robust Spotting of Scene Texts

Downloads

Next from ICPR 2020

Recursive Recognition of Offline Handwritten Mathematical Expressions

Similar lecture

Stratified Multi-Task Learning for Robust Spotting of Scene Texts