Italy

We introduce a new deep architecture for closing the semantic gaps between the multiscale feature maps. The proposed ScarfNet generates new multiscale feature maps with deeply fused and redistributed semantics by using the combination of biLSTM and the channel-wise attention model. Our evaluation results show that our method offers significant improvement over the baseline detectors as well as other competitive detectors.


ICPR 2020

ScarfNet: Multi-scale Features with Deeply Fused and Redistributed Semantics for Enhanced Object Detection

feature pyramid

object detection

computer vision

deep learning

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster


Automatic facial expression recognition (FER) has gained much attention due to its applications in human-computer interaction. Among the approaches to improve FER tasks, this paper focuses on deep architecture with the attention mechanism. We propose a novel Masking Idea to boost the performance of CNN in facial expression task. It uses a segmentation network to refine feature maps, enabling the network to focus on relevant information to make correct decisions. In experiments, we combine the ubiquitous Deep Residual Network and Unetlike architecture to produce a Residual Masking Network. The proposed method holds state-of-the-art (SOTA) accuracy on the well-known FER2013 and private VEMO datasets

Facial Expression Recognition Using Residual Masking Network


Photo-to-caricature translation aims to synthesize the caricature as a rendered image exaggerating the features through sketching, pencil strokes, or other artistic drawings. Style rendering and geometry deformation are the most important aspects in photo-to-caricature translation task. To take both into consideration, we propose an unsupervised contrastive photo-to-caricature translation architecture. Considering the intuitive artifacts in the existing methods, we propose a contrastive style loss for style rendering to enforce the similarity between the style of rendered photo and the caricature, and simultaneously enhance its discrepancy to the photos. To obtain an exaggerating deformation in an unpaired/unsupervised fashion, we propose a Distortion Prediction Module (DPM) to predict a set of displacements vectors for each input image while fixing some controlling points, followed by the thin plate spline interpolation for warping. The model is trained on unpaired photo and caricature while can offer bidirectional synthesizing via inputting either a photo or a caricature. Extensive experiments demonstrate that the proposed model is effective to generate hand-drawn like caricatures compared with existing competitors.

Unsupervised Contrastive Photo-to-Caricature Translation based on Auto-distortion

Within the context of eye-gaze tracking, the capability of permitting the user to move naturally is an important step towards allowing for more natural user interaction in less constrained scenarios. Natural movement can be characterised by changes in head pose, as well as non-rigid face deformations as the user performs different facial expressions. While the estimation of head pose within the domain of eye-gaze tracking is being increasingly considered, the face is most often regarded as a rigid body. The few methods that factor the challenge of handling face deformations into the head pose estimation problem, often require the availability of a pre-defined face model or a considerable amount of training data. In this paper, we direct our attention towards the application of shape-and- motion factorisation for head pose estimation, since this does not generally rely on the availability of an initial face model. Over the years, various shape-and-motion factorisation methods have been proposed to address the challenges of rigid and non-rigid shape and motion recovery, in a batch or sequential manner. However, the real-time recovery of non-rigid shape and motion by factorisation remains, in general, an open problem. Our work addresses this open problem by proposing a sequential factorisation method for non-rigid shape and motion recovery, which does not rely on the availability of a pre-defined face deformation model or training data. Quantitative and qualitative results show that our method can handle various non-rigid face deformations without deterioration of the head pose estimation accuracy.

Sequential Non-Rigid Factorisation for Head Pose Estimation


This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

Distilling Spikes: Knowledge Distillation in Spiking Neural Networks


Cardiac left ventricular (LV) segmentation is a paramount essential step for both diagnosis and treatment of cardiac pathologies such as ischemia, myocardial infarction, arrhythmia, and myocarditis. However, this segmentation is challenging due to high variability across patients and the potential lack of contrast between structures. In this work, we propose and evaluate a (2.5D) SegU-Net model based on the fusion of two deep learning segmentation techniques (U-Net and Seg-Net) for automated LGE-MRI (Late gadolinium enhanced magnetic resonance imaging) myocardial disease (infarct core and no-reflow region) quantification in a new multifield expert annotated dataset. Given that the scar tissue represents a small part of the whole MRI slices, we focused on the myocardium area. Segmentation results show that this preprocessing step facilitates the learning procedure. In order to solve the class imbalance problem, we propose to apply the Jaccard loss and the Focal Loss as optimization loss function and to integrate a class weights strategy into the objective function. Late combination has been used to merge the output of the best trained models on a different set of hyperparameters. The final network segmentation performances will be useful for future comparison of new methods to the current related work for this task. A total number of 2237 of slices (320 cases) were used for training/validation and 210 slices (35 cases) were used for testing. Experiments on our proposed dataset, using several evaluation metrics such as Jaccard distance (IOU), Accuracy, and Dice similarity coefficient (DSC), demonstrate efficiency performance in quantifying different zones of myocardium infarction across various patients. As compared to the second intra-observer study, our testing results showed that the SegU-Net prediction model leads to these average Dice coefficients overall segmented tissue classes, respectively: 'Back-ground': 0.99999, 'Myocardium': 0.99434, 'Infarctus': 0.95587, 'Noreflow': 0.78187 outperforming seven previously proposed methods.

A deep learning approach for the segmentation of myocardial diseases

In this work we propose a deep learning pipeline to predict
the visual future appearance of an urban scene. Despite
recent advances, generating the entire scene in an
end-to-end fashion is still far from being achieved.
Instead, here we follow a two stages approach, where
interpretable information is included in the loop and each
actor is modelled independently. We leverage a per-object
novel view synthesis paradigm; i.e. generating a synthetic
representation of an object undergoing a geometrical
roto-translation in the 3D space. Our model can be easily
conditioned with constraints (e.g. input trajectories)
provided by state-of-the-art tracking methods or by the
user itself. This allows us to generate a set of diverse
realistic futures starting from the same input in a
multi-modal fashion. We visually and quantitatively show
the superiority of this approach over traditional
end-to-end scene-generation methods on CityFlow, a
challenging real world dataset.

Future Urban Scene Generation Through Vehicles Synthesis

The development of computer-assisted diagnosis (CAD) algorithms for pathological image analysis constitutes an important research topic. Such systems are required to be not only accurate but also explainable for their decisions, to ensure reliability. In this paper, we present an explainable diagnosis method based on convolutional neural networks (CNNs). This method allows us to interpret the basis of the decisions made by CNN from two perspectives, namely statistics and visualization. For the statistical explanation, the method constructs a dictionary of representative pathological features. It performs diagnoses based on the occurrence and importance of learned features referred from its dictionary. To construct the dictionary, we introduce a vector quantization scheme to CNN. For the visual interpretation, the method provides images of learned features embedded in high-dimensional feature space as an index of the dictionary by generating them using a conditional autoregressive model. The experimental results showed that the proposed network learned pathological features, which contributed to the diagnosis and yielded approximately an area under the receiver operating curve (AUC) of 0.93 for detecting atypical tissues in pathological images of the uterine cervix. Moreover, the proposed method demonstrated that it could provide visually interpretable images to show the rationales behind its decisions. Thus, the proposed system can serve as a valuable tool for pathological image analysis.

Explainable Feature Embedding using Convolutional Neural Networks for Pathological Image Analysis


Despite the recent advances in optical character recognition (OCR), mathematical expressions still face a great challenge to recognize due to their two-dimensional graphical layout. In this paper, we propose a convolutional sequence modeling network, ConvMath, which converts the mathematical expression description in an image into a LaTeX sequence in an end-to-end way. The network combines an image encoder for feature extraction and a convolutional decoder for sequence generation.
Compared with other Long Short Term Memory(LSTM) based encoder-decoder models, ConvMath is entirely based on convolution, thus it is easy to perform parallel computation. Besides, the network adopts multi-layer attention mechanism in the decoder, which allows the model to align output symbols with source feature vectors automatically, and alleviates the problem of lacking coverage while training the model. The performance of ConvMath is evaluated on an open dataset named IM2LATEX-100K, including 103556 samples. The experimental results demonstrate that the proposed network achieves state-of-the-art accuracy and much better efficiency than previous methods

ConvMath: A Convolutional Sequence Network for Mathematical Expression Recognition

Recognizing an activity with a single reference sample using metric learning approaches is a promising research field. The majority of few-shot methods focus on object recognition or face-identification. We propose a metric learning approach to reduce the action recognition problem to a nearest neighbor search in embedding space. We encode signals into images and extract features using a deep residual CNN. Using triplet loss, we learn a feature embedding. The resulting encoder transforms features into an embedding space in which closer distances encode similar actions while higher distances encode different actions. Our approach is based on a signal level formulation and remains flexible across a variety of modalities. It further outperforms the baseline on the large scale NTU RGB+D 120 dataset for the One-Shot action recognition protocol by 5.6%. With just 60% of the training data, our approach still outperforms the baseline approach by 3.7%. With 40% of the training data, our approach performs comparably well to the second follow up. Further, we show that our approach generalizes well in experiments on the UTD-MHAD dataset for inertial, skeleton and fused data and the Simitate dataset for motion capturing data. Furthermore, our inter-joint and inter-sensor experiments suggest good capabilities on previously unseen setups.

SL-DML: Signal Level Deep Metric Learning for Multimodal One-Shot Action Recognition


Most computer aided pathology detection systems rely on
large volumes of quality annotated data to aid diagnostics
and follow up procedures. However, quality assuring large
volumes of annotated medical image data can be subjective
and expensive. In this work we present a novel
standardization framework that implements three few-shot
learning (FSL) models that can be iteratively trained by
atmost 5 images per 3D stack to generate multiple regional
proposals (RPs) per test image. These FSL models include a
novel parallel echo state network framework and an
augmented U-net model. Additionally, we propose a novel
target label selection algorithm (TLSA) that measures
relative agreeability between RPs and the manually
annotated target labels to detect the ``best" quality
annotation per image. Using the FSL models, our system
achieves 0.28-0.64 Dice coefficient across vendor image
stacks for intra-retinal cyst segmentation. Additionally,
the TLSA is capable of correctly classifying high quality
target labels from their noisy counterparts with 70-97%
accuracy. The proposed system significantly automates high
quality annotation selection on an image level while
minimizing manual quality checking to 12-28% of the images
only. Thus, the proposed framework is flexible to
extensions for quality image annotation curation of other
image stacks as well.

Premium content

ScarfNet: Multi-scale Features with Deeply Fused and Redistributed Semantics for Enhanced Object Detection

Downloads

Next from ICPR 2020

Facial Expression Recognition Using Residual Masking Network

Similar lecture

FeatureNMS: Non-Maximum Suppression by Learning Feature Embeddings