Italy

In this work we propose a deep learning pipeline to predict
the visual future appearance of an urban scene. Despite
recent advances, generating the entire scene in an
end-to-end fashion is still far from being achieved.
Instead, here we follow a two stages approach, where
interpretable information is included in the loop and each
actor is modelled independently. We leverage a per-object
novel view synthesis paradigm; i.e. generating a synthetic
representation of an object undergoing a geometrical
roto-translation in the 3D space. Our model can be easily
conditioned with constraints (e.g. input trajectories)
provided by state-of-the-art tracking methods or by the
user itself. This allows us to generate a set of diverse
realistic futures starting from the same input in a
multi-modal fashion. We visually and quantitatively show
the superiority of this approach over traditional
end-to-end scene-generation methods on CityFlow, a
challenging real world dataset.

ICPR 2020

Future Urban Scene Generation Through Vehicles Synthesis

image based modeling

scene understanding

applications of computer vision

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster

The development of computer-assisted diagnosis (CAD) algorithms for pathological image analysis constitutes an important research topic. Such systems are required to be not only accurate but also explainable for their decisions, to ensure reliability. In this paper, we present an explainable diagnosis method based on convolutional neural networks (CNNs). This method allows us to interpret the basis of the decisions made by CNN from two perspectives, namely statistics and visualization. For the statistical explanation, the method constructs a dictionary of representative pathological features. It performs diagnoses based on the occurrence and importance of learned features referred from its dictionary. To construct the dictionary, we introduce a vector quantization scheme to CNN. For the visual interpretation, the method provides images of learned features embedded in high-dimensional feature space as an index of the dictionary by generating them using a conditional autoregressive model. The experimental results showed that the proposed network learned pathological features, which contributed to the diagnosis and yielded approximately an area under the receiver operating curve (AUC) of 0.93 for detecting atypical tissues in pathological images of the uterine cervix. Moreover, the proposed method demonstrated that it could provide visually interpretable images to show the rationales behind its decisions. Thus, the proposed system can serve as a valuable tool for pathological image analysis.

Explainable Feature Embedding using Convolutional Neural Networks for Pathological Image Analysis


Despite the recent advances in optical character recognition (OCR), mathematical expressions still face a great challenge to recognize due to their two-dimensional graphical layout. In this paper, we propose a convolutional sequence modeling network, ConvMath, which converts the mathematical expression description in an image into a LaTeX sequence in an end-to-end way. The network combines an image encoder for feature extraction and a convolutional decoder for sequence generation.
Compared with other Long Short Term Memory(LSTM) based encoder-decoder models, ConvMath is entirely based on convolution, thus it is easy to perform parallel computation. Besides, the network adopts multi-layer attention mechanism in the decoder, which allows the model to align output symbols with source feature vectors automatically, and alleviates the problem of lacking coverage while training the model. The performance of ConvMath is evaluated on an open dataset named IM2LATEX-100K, including 103556 samples. The experimental results demonstrate that the proposed network achieves state-of-the-art accuracy and much better efficiency than previous methods

ConvMath: A Convolutional Sequence Network for Mathematical Expression Recognition

Recognizing an activity with a single reference sample using metric learning approaches is a promising research field. The majority of few-shot methods focus on object recognition or face-identification. We propose a metric learning approach to reduce the action recognition problem to a nearest neighbor search in embedding space. We encode signals into images and extract features using a deep residual CNN. Using triplet loss, we learn a feature embedding. The resulting encoder transforms features into an embedding space in which closer distances encode similar actions while higher distances encode different actions. Our approach is based on a signal level formulation and remains flexible across a variety of modalities. It further outperforms the baseline on the large scale NTU RGB+D 120 dataset for the One-Shot action recognition protocol by 5.6%. With just 60% of the training data, our approach still outperforms the baseline approach by 3.7%. With 40% of the training data, our approach performs comparably well to the second follow up. Further, we show that our approach generalizes well in experiments on the UTD-MHAD dataset for inertial, skeleton and fused data and the Simitate dataset for motion capturing data. Furthermore, our inter-joint and inter-sensor experiments suggest good capabilities on previously unseen setups.

SL-DML: Signal Level Deep Metric Learning for Multimodal One-Shot Action Recognition


Most computer aided pathology detection systems rely on
large volumes of quality annotated data to aid diagnostics
and follow up procedures. However, quality assuring large
volumes of annotated medical image data can be subjective
and expensive. In this work we present a novel
standardization framework that implements three few-shot
learning (FSL) models that can be iteratively trained by
atmost 5 images per 3D stack to generate multiple regional
proposals (RPs) per test image. These FSL models include a
novel parallel echo state network framework and an
augmented U-net model. Additionally, we propose a novel
target label selection algorithm (TLSA) that measures
relative agreeability between RPs and the manually
annotated target labels to detect the ``best" quality
annotation per image. Using the FSL models, our system
achieves 0.28-0.64 Dice coefficient across vendor image
stacks for intra-retinal cyst segmentation. Additionally,
the TLSA is capable of correctly classifying high quality
target labels from their noisy counterparts with 70-97%
accuracy. The proposed system significantly automates high
quality annotation selection on an image level while
minimizing manual quality checking to 12-28% of the images
only. Thus, the proposed framework is flexible to
extensions for quality image annotation curation of other
image stacks as well.

Few Shot Learning Framework to ReduceInter-observer Variability in Medical Images

The prevalent adoptions of GPS-enabled devices have witnessed an explosion of various location-based services which produces a huge amount of trajectories monitoring the individuals' movements. In this paper, we tackle Trajectory-User Link (TUL) problem, which identifies humans' movement patterns and links trajectories to the users who generated them. Existing solutions on TUL problem employ recurrent neural networks and variational autoencoder methods, which face the bottlenecks in the case of excessively long trajectories and fragmentary users' movements. However, these are common characteristics of trajectory data in reality, leading to performance degradation of the existing models. In this paper, we propose an end-to-end attention recurrent neural learning framework, called TULAR (Trajectory-User Link with Attention Recurrent Networks), which focus on selected parts of the source trajectories when linking. TULAR introduces the Trajectory Semantic Vector (TSV) via unsupervised location representation learning and recurrent neural networks, by which to reckon the weight of parts of source trajectory. Further, we employ three attention scores for the weight measurements. Experiments are conducted on two real-world datasets and compared with several existing methods, and the results show that TULAR yields a new state-of-the-art performance. Source code is publicly available at GitHub: https://github.com/taos123/TULAR.


Trajectory-User Link with Attention Recurrent Networks

Networks of spiking neurons and Winner-Take-All spiking circuits (WTA-SNNs) can detect information encoded in spatio-temporal multi-valued events. These are described by the timing of events of interest, e.g., clicks, as well as by categorical numerical values assigned to each event, e.g., like or dislike. Other use cases include object recognition from data collected by neuromorphic cameras, which produce, for each pixel, signed bits at the times of sufficiently large brightness variations. Existing schemes for training WTA-SNNs are limited to rate-encoding solutions, and are hence able to detect only spatial patterns. Developing more general training algorithms for arbitrary WTA-SNNs inherits the challenges of training (binary) Spiking Neural Networks (SNNs). These amount, most notably, to the non-differentiability of threshold functions, to the recurrent behavior of spiking neural models, and to the difficulty of implementing backpropagation in neuromorphic hardware. In this paper, we develop a variational online local training rule for WTA-SNNs, referred to as VOWEL, that leverages only local pre- and post-synaptic information for visible circuits, and an additional common reward signal for hidden circuits. The method is based on probabilistic generalized linear neural models, control variates, and variational regularization. Experimental results on real-world neuromorphic datasets with multi-valued events demonstrate the advantages of WTA-SNNs over conventional binary SNNs trained with state-of-the-art methods, especially in the presence of limited computing resources.

VOWEL: A Local Online Learning Rule for Recurrent Networks of Probabilistic Spiking Winner-Take-All Circuits

A smile is a representative expression of happiness or high quality-of-life; however, automatic recognition of a smile according to happiness remains a challenging task.
Because expressions of happiness are strongly dependent upon physical condition and occurrence of other emotions, and similar facial expression often occur under different emotions, we consider that there is no absolute visual pattern of a smile corresponding to happiness.
Therefore, in this study, we assumed that a ``smile with happiness'' is observed as the temporal ascent in the degree of smiling and attempted to recognize this by capturing changes in facial expression within temporally sequential images.
As an implementation of this scheme, we proposed a Siamese-structure deep neural network to compare facial expressions in two input images and estimate the existence of smile ascension or descension.
For primal analysis of the proposed network, we developed a unique smiling dataset containing image pairs with various changes in smiling degree, including slight changes.
The results demonstrated that the proposed method achieved nearly perfect recognition with $>0.95$ accuracy when recognizing changes in the degree of smiling that humans certainly recognize.
Attention regions that contributed to the predicted labels were concentrated on the mouth, cheeks, and tail of the eyes, which indicates a reasonable function for recognizing changes in smiling degree was constructed by the proposed method.

Siamese-structure DNN Recognizing Changes in Facial Expression According to the Degree of Smiling

Retinal fundus images reveal the condition of retina, blood vessels and optic nerve, and is becoming widely adopted in clinical work because any subtle changes to the structures at the back of the eyes can affect the eyes and indicate the overall health. Recently, machine learning, in particular deep learning by convolutional neural network (CNN), has been increasingly adopted for computer-aided detection (CAD) of retinal lesions. However, a significant barrier to the high performance of CNN based CAD approach is the lack of sufficient labeled image samples for training. Unlike the fully-supervised learning which relies on pixel-level annotation of pathology in fundus images, this paper presents a new approach to discriminate the location of various lesions based on image-level labels via weakly learning. More specifically, our proposed method leverages the multi-level feature maps and classification score to cope with both bright and red lesions in fundus images. To enhance capability of learning less discriminative parts of objects (e.g. small blobs of microaneurysms opposed to bulk of exudates), the classifier is regularized by refining images with corresponding labels. The experimental results of the performance evaluation and benchmarking at both image-level and pixel-level on the public DIARETDB1 dataset demonstrate the feasibility and excellent potentials of our method in practical usage.

Robust Localization of Retinal Lesions via Weakly-supervised Learning

We propose a learning framework named Feature Fusion Learning (FFL) that efficiently trains a powerful classifier through a fusion module which combines the feature maps generated from parallel neural networks and generates meaningful feature maps. Specifically, we train a number of parallel neural networks as sub-networks, then we combine the feature maps from each sub-network using a fusion module to create a more meaningful feature map. The fused feature map is passed into the fused classifier for overall classification. Unlike existing feature fusion methods, in our framework, an ensemble of sub-network classifiers transfers its knowledge to the fused classifier and then the fused classifier delivers its knowledge back to each subnetwork, mutually teaching one another in an online-knowledge
distillation manner. This mutually teaching system not only improves the performance of the fused classifier but also obtains performance gain in each sub-network. Moreover, our model is more beneficial than other alternative methods because different types of network can be used for each sub-network. We have
performed a variety of experiments on multiple datasets such as CIFAR-10, CIFAR-100 and ImageNet and proved that our method is more effective than other alternative methods in terms of performances of both sub-networks and the fused classifier, and the aspect of generating meaningful feature maps.

Feature Fusion for Online Mutual Knowledge Distillation


This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

Premium content

Future Urban Scene Generation Through Vehicles Synthesis

Downloads

Next from ICPR 2020

Explainable Feature Embedding using Convolutional Neural Networks for Pathological Image Analysis

Similar lecture

Delivering Meaningful Representation for Monocular Depth Estimation