Italy


Manipulating images of complex scenes to reconstruct, insert and/or remove specific object instances is a challenging task.
Complex scenes contain multiple semantics and objects, which are frequently cluttered or ambiguous, thus hampering the performance of inpainting models.
Conventional techniques often rely on structural information such as object contours in multi-stage approaches that generate unreliable results and boundaries.
In this work, we propose a novel deep learning model to alter a complex urban scene by removing a user-specified portion of the image and coherently inserting a new object (e.g. car or pedestrian) in that scene.
Inspired by recent works on image inpainting, our proposed method leverages the semantic segmentation to model the content and structure of the image, and learn the best shape and location of the object to insert.
To generate reliable results, we design a new decoder block that combines the semantic segmentation and generation task to guide better the generation of new objects and scenes, which have to be semantically consistent with the image.

Our experiments, conducted on two large-scale datasets of urban scenes (Cityscapes and Indian Driving), show that our proposed approach successfully address the problem of semantically-guided inpainting of complex urban scene.

ICPR 2020

Semantic-Guided Inpainting Network for Complex Urban Scenes Manipulation

image manipulation

image inpainting

gans

deep learning

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster

A plethora of research in the literature shows how human eye fixation pattern varies depending on different factors, including genetics, age, social functioning, cognitive functioning, and so on. Analysis of these variations in visual attention has already elicited two potential research avenues: 1) determining the physiological or psychological state of the subject and 2) predicting the tasks associated with the act of viewing from the recorded eye-fixation data. To this end, this paper proposes a visual saliency based novel feature extraction method for automatic and quantitative classification of eye-tracking data, which is applicable to both of the research directions. Instead of directly extracting features from the fixation data, this method
employs several well-known computational models of visual attention to predict eye fixation locations as saliency maps. Comparing the saliency amplitudes, similarity and dissimilarity of saliency maps with the corresponding eye fixations maps gives an extra dimension of information which is effectively utilized to generate discriminative features to classify the eye-tracking data. Extensive experimentation using Saliency4ASD, Age Prediction, and Visual Perceptual Task dataset show that
our saliency-based feature can achieve superior performance, outperforming the previous state-of-the-art methods by a considerable margin. Moreover, unlike the existing application-specific solutions, our method demonstrates performance improvement across three distinct problems from the real-life domain: Autism Spectrum Disorder screening, toddler age prediction, and human visual perceptual task classification, providing a general paradigm that utilizes the extra-information
inherent in saliency maps for a more accurate classification

Classifying Eye-Tracking Data Using Saliency Maps

Time series clustering is a challenging task due to the specificities of this type of data. Temporal correlation and invariance to transformations such as shifting, warping or noise prevent the use of standard data mining methods. Time series clustering has been mostly studied under the angle of finding efficient algorithms and distance metrics adapted to the specific nature of time series data. Much less attention has been devoted to the general problem of model selection. Clustering stability has emerged as a universal and model-agnostic principle for clustering model selection. This principle can be stated as follows: an algorithm should find a structure in the data that is resilient to perturbation by sampling or noise. We propose to apply stability analysis to time series by leveraging prior knowledge on the nature and invariances of the data. These invariances determine the perturbation process used to assess stability. Based on a recently introduced criterion combining between-cluster and within-cluster stability, we propose an invariance-guided method for model selection, applicable to a wide range of clustering algorithms. Experiments conducted on artificial and benchmark data sets demonstrate the ability of our criterion to discover structure and select the correct number of clusters, whenever data invariances are known beforehand.

An Invariance-guided Stability Criterion for Time Series Clustering Validation

Acquiring information about the road lane structure
is a crucial step for autonomous navigation. To this end, several
approaches tackle this task from different perspectives such as
lane marking detection or semantic lane segmentation. However,
to the best of our knowledge, there is yet no purely vision
based end-to-end solution to answer the precise question: How
to estimate the relative number or "ID" of the current driven
lane within a multi-lane road or a highway?
In this work, we propose a real-time, vision-only (i.e. monocular
camera) solution to the problem based on a dual left-right
convention. We interpret this task as a classification problem
by limiting the maximum number of lane candidates to eight.
Our approach is designed to meet low-complexity specifications
and limited runtime requirements. It harnesses the temporal
dimension inherent to the input sequences to improve upon high complexity
state-of-the-art models. We achieve more than 95%
accuracy on a challenging test set with extreme conditions and
different routes.


Real-time End-to-End Lane ID Estimation Using
Recurrent Networks



In this paper, a new 2D transform named Discrete Mirror Transform (DMT) is presented.
The DMT is computed by decomposing a signal into its even and odd parts around an optimal location in a given direction so that the signal energy is maximally split between the two components.
After minimizing the information required to regenerate the original signal by removing redundant structures,
the process is iterated leading the signal energy to distribute into a continuously smaller set of coefficients.
The DMT can be displayed as a binary tree, where each node represents the 
single (even or odd) signal derived from the decomposition in the previous level.
An optimized version of the DMT (ODMT) is also introduced, by exploiting the possibility to choose different directions at which performing the decomposition.
Experimental simulations have been carried out in order to test the sparsity
properties of the DMT and ODMT when applied on images: referring to both transforms, the results show a superior performance with respect to the popular Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT) in terms of non-linear approximation.

2D - Discrete Mirror Transform for Image Non-Linear Approximation

Interpretable Machine Learning (IML) methods are used to gain insight into the relevance of a feature of interest for the performance of a model. Commonly used IML methods differ in whether they consider features of interest in isolation, e.g., Permutation Feature Importance (PFI), or in relation to all remaining feature variables, e.g., Conditional Feature Importance (CFI). As such, the perturbation mechanisms inherent to PFI and CFI represent extreme reference points. We introduce Relative Feature Importance (RFI), a generalization of PFI and CFI that allows for a more nuanced feature importance computation beyond the PFI versus CFI dichotomy. With RFI, the importance of a feature relative to any other subset of features can be assessed, including variables that were not available at training time. We derive general interpretation rules for RFI based on a detailed theoretical analysis of the implications of relative feature relevance, and demonstrate the method's usefulness on simulated examples.

Relative Feature Importance


A number of methods based on deep learning have been applied to medical image segmentation and have achieved state-of-the-art performance. Due to the importance of chest x-ray data in studying COVID-19, there is a demand for state-of-the-art models capable of precisely segmenting soft tissue on the chest x-rays. The dataset for exploring the best segmentation model is from Montgomery and Shenzhen hospital which had opened in 2014. The most famous technique is U-Net which has been used for many medical datasets including the Chest X-rays. However, most variant U-Nets mainly focus on the extraction of contextual information and skip connections. There is still a large space for improving the extraction of spatial features. In this paper, we propose a dual encoder fusion U-Net framework for Chest X-rays based on Inception Convolutional Neural Network with dilation, Densely Connected Recurrent Convolutional Neural Network, which is named DEFU-Net. The densely connected recurrent path extends the network deeper for facilitating contextual feature extraction. In order to increase the width of the network and enrich the representation of features, the inception blocks with dilation are adopted. The inception blocks can capture globally and locally spatial information from various receptive fields. At the same time, the two paths are fused by summing features, thus preserving the contextual and spatial information for the decoding part. This multi-learning-scale model is benefiting in the Chest X-ray dataset from two different manufacturers (Montgomery and Shenzhen hospital). The DEFU-Net achieves better performance than basic U-Net, residual U-Net, BCDU-Net, R2U-Net and attention R2U-Net. This model has proved the feasibility for the mixed datasets and approaches state-of-the-art.

Dual Encoder Fusion U-Net (DEFU-Net) for Cross-manufacturer Chest X-ray Segmentation

In feature embedding, the recovery of associated
discriminative information in the reduced subspace is critical
for downstream classifiers. In this study, a supervised feature
embedding method is proposed inspired by the well-known word
embedding technique, word2vec. Proposed embedding method
is implemented as representative learning of rank-based neighborhoods.
The notion of context words in word2vec is extended
into neighboring instances within a given window. Neighborship
is defined using ranks of instances rather than their values so
that regions with different densities are captured properly. Each
sample is represented by a unique one-hot vector whereas its
neighbors are encoded by several two-hot vectors. The two-hot
vectors are identical for neighboring samples of the same class. A
feed-forward neural network with a continuous projection layer,
then learns the mapping from one-hot vectors to multiple two-hot
vectors. The hidden layer determines the reduced subspace for
the train samples. The obtained transformation is then applied
on test data to find a lower-dimensional representation. Proposed
method is tested in classification problems on 10 UCI data
sets. Experimental results confirm that the proposed method is
effective in finding a discriminative representation of the features
and outperforms several supervised

Supervised Feature Embedding for Classification by Learning Rank-based Neighborhoods

Preventive conservation is an important practice in Cultural Heritage. The constant monitoring of the state of conservation of an artwork helps us reduce the risk of damage and number of necessary interventions. In this work, we propose a probabilistic approach for the detection of alterations on the surface of historical violins based on an a-contrario framework. Our method is a one step NFA clustering solution which considers grey-level and spatial density information in one background model. The proposed method is robust to noise and avoids parameter tuning and any assumption about the quantity of the worn-out areas. We have used as input UV induced fluorescence (UVIFL) images for considering details not perceivable with visible light. Tests were conducted on image sequences included in the ``Violins UVIFL imagery'' dataset. Results illustrate the ability of the algorithm to distinguish the worn area from the surrounding regions. Comparisons with state-of-the-art clustering methods show improved overall precision and recall.

One step clustering based on a-contrario framework for detection of alterations in historical violins


Speech emotion recognition (SER) is an important and challenging task. It requires that a machine learning model process a person’s speech signals and to judge his emotional state accurately. Due to the high dimensionality of the audio data, the extracted features are always noisy. Besides, the abstraction of audio features makes it impossible to fully use the inherent relationship among audio features. This paper proposes a model that combines a convolutional neural network (CNN) and a capsule network (CapsNet), named as CapCNN. The advantage of CapCNN lies in that it provides a solution for time sensitivity, and gives the overall characteristics. In this study, it is found that CapCNN can well handle the SER task. Compared with other state-of-the-art methods, our algorithm shows high performances on the CASIA and EMODB datasets. The detailed analysis confirms that our method provides balanced results on various classes.

The Application of Capsule Neural Network Based CNN for Speech emotion recognition


Digital era has changed the way music is produced and propagated creating new needs for automated and more effective management of music tracks in big volumes. Automated music mood detection constitutes an active task in the field of MIR (Music Information Retrieval) and connected with many research papers in the past few years. In order to approach the task of mood detection, we faced separately the analysis of musical lyrics and the analysis of musical audio signals. Then we applied a uniform multichannel analysis to classify our data in mood classes. The available data we will use to train and evaluate our models consists of a total of 2.000 song titles, classified in four mood classes {happy, angry, sad, relaxed}. The result of this process leads to a uniform prediction for emotional arousal that a music track can cause to a listener and show the way to develop many applications.

Premium content

Semantic-Guided Inpainting Network for Complex Urban Scenes Manipulation

Downloads

Next from ICPR 2020

Classifying Eye-Tracking Data Using Saliency Maps

Similar lecture

Free-Form Image Inpainting via Contrastive Attention Network