Italy

An advanced simulation framework has recently been introduced for exploring human perception and visuomotor control. In this context, we investigate locally-connected, irregular deep neural networks (liNets) for biomimetic active vision. Like commonly used CNNs, liNets are locally-connected, forming receptive fields, but unlike CNNs, they are suitable for spatially irregular photoreceptor distributions inspired by those found in foveated biological retinas. Compared to fully connected
deep neural networks, liNets accommodate a much greater number of retinal photoreceptors to enhance visual acuity without intractable memory consumption. LiNets serve well in the biomimetic active vision system embodied in a simulated human that learns active visuomotor control and active appearance based recognition.

ICPR 2020

Locally-Connected, Irregular Deep Neural Networks for Biomimetic Active Vision in a Simulated Human

biomimetic vision

biomechanical human simulation

virtual human

biomimetic active vision

deep learning

deep neural networks

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster

In this paper, we propose an occlusion-robust pose estimation method of an unknown object instance in an object category from a depth image. In a cluttered environment, objects are often occluded mutually. For estimating the pose of an object in such a situation, a method that de-occludes the unobservable area of the object would be effective. However, there are two difficulties; occlusion causes the offset between the center of the actual object and its observable area, and different instances in a category may have different shapes. To cope with these difficulties, we propose a two-stage Encoder-Decoder model to extract features with objects whose centers are aligned to the image center. In the model, we also propose the Median-shape Reconstructor as the second stage to absorb shape variations in a category. By evaluating the method with both a large-scale virtual dataset and a real dataset, we confirmed the proposed method achieves good performance on pose estimation of an occluded object from a depth image.

Median-shape Representation Learning for Category-level Object Pose Estimation in Cluttered Environments

Graphical models allow to describe the interplayamong variables of a system through a compact representation,suitable when relations evolve over time. For example, in abiological setting, genes interact differently depending on externalenvironmental or metabolic factors. To incorporate this dynamicsa viable strategy is to estimate a sequence of temporally relatedgraphs assuming similarity among samples in different timepoints. While adjacent time points may direct the analysistowards a robust estimate of the underlying graph, the resultingmodel will not incorporate long-term or recurrent temporalrelationships. In this work we propose a dynamical networkinference model that leverages on kernels to consider generaltemporal patterns (such as circadian rhythms or seasonality). Weshow how our approach may also be exploited when the recurrentpatterns are unknown, by coupling the network inference with aclustering procedure that detects possibly non-consecutive similarnetworks. Such clusters are then used to build similarity kernels.The convexity of the functional is determined by whether weimpose or infer the kernel. In the first case, the optimisationalgorithm exploits efficiently proximity operators with closed-form solutions. In the other case, we resort to an alternatingminimisation procedure which jointly learns the temporal kerneland the underlying network. Extensive analysis on syntheticdata shows the efficacy of our models compared to state-of-the-art methods. Finally, we applied our approach on two real-world applications to show how considering long-term patternsis fundamental to have insights on the behaviour of a complexsystem.

Temporal Pattern Detection in Time-Varying Graphical Models


We present a method for exploiting weakly annotated images to improve text extraction pipelines. The approach uses an arbitrary end-to-end text recognition system to obtain text region proposals and their, possibly erroneous, transcriptions. The method includes matching of imprecise transcriptions to weak annotations and an edit distance guided neighbourhood search. It produces nearly error-free, localised instances of scene text, which we treat as "pseudo ground truth" (PGT).

The method is applied to two weakly-annotated datasets. Training with the extracted PGT consistently improves the accuracy of a state of the art recognition model, by 3.7% on average, across different benchmark datasets (image domains) and 24.5% on one of the weakly annotated datasets.

Text Recognition: Real World Data and Where to Find Them

Multi-camera sports players' 3D localization is always a challenging task due to heavy occlusions in crowded sports scenes. Traditional methods can only provide players’ locations without identifying information. Existing methods of localization may cause ambiguous detection and unsatisfactory precision and recall, especially when heavy occlusions occur. To solve this problem, we propose a generic localization method by providing distinguishable results that have the probabilities of locations being occupied by players with unique ID labels. We design the algorithms with a multi-dimensional Bayesian model to create a Probabilistic and Identified Occupancy Map (PIOM). By using this model, we jointly apply deep learning-based object segmentation and identification to obtain sports players’ probable positions and their likely identification labels. This approach not only provides players 3D locations but also gives their ID information that is distinguishable from others. Experimental results demonstrate that our method outperforms the previous localization approaches with reliable and distinguishable outcomes.

Multi-camera Sports Players 3D Localization with Identification Reasoning

We introduce a new deep architecture for closing the semantic gaps between the multiscale feature maps. The proposed ScarfNet generates new multiscale feature maps with deeply fused and redistributed semantics by using the combination of biLSTM and the channel-wise attention model. Our evaluation results show that our method offers significant improvement over the baseline detectors as well as other competitive detectors.


ScarfNet: Multi-scale Features with Deeply Fused and Redistributed Semantics for Enhanced Object Detection


Automatic facial expression recognition (FER) has gained much attention due to its applications in human-computer interaction. Among the approaches to improve FER tasks, this paper focuses on deep architecture with the attention mechanism. We propose a novel Masking Idea to boost the performance of CNN in facial expression task. It uses a segmentation network to refine feature maps, enabling the network to focus on relevant information to make correct decisions. In experiments, we combine the ubiquitous Deep Residual Network and Unetlike architecture to produce a Residual Masking Network. The proposed method holds state-of-the-art (SOTA) accuracy on the well-known FER2013 and private VEMO datasets

Facial Expression Recognition Using Residual Masking Network


Photo-to-caricature translation aims to synthesize the caricature as a rendered image exaggerating the features through sketching, pencil strokes, or other artistic drawings. Style rendering and geometry deformation are the most important aspects in photo-to-caricature translation task. To take both into consideration, we propose an unsupervised contrastive photo-to-caricature translation architecture. Considering the intuitive artifacts in the existing methods, we propose a contrastive style loss for style rendering to enforce the similarity between the style of rendered photo and the caricature, and simultaneously enhance its discrepancy to the photos. To obtain an exaggerating deformation in an unpaired/unsupervised fashion, we propose a Distortion Prediction Module (DPM) to predict a set of displacements vectors for each input image while fixing some controlling points, followed by the thin plate spline interpolation for warping. The model is trained on unpaired photo and caricature while can offer bidirectional synthesizing via inputting either a photo or a caricature. Extensive experiments demonstrate that the proposed model is effective to generate hand-drawn like caricatures compared with existing competitors.

Unsupervised Contrastive Photo-to-Caricature Translation based on Auto-distortion

Within the context of eye-gaze tracking, the capability of permitting the user to move naturally is an important step towards allowing for more natural user interaction in less constrained scenarios. Natural movement can be characterised by changes in head pose, as well as non-rigid face deformations as the user performs different facial expressions. While the estimation of head pose within the domain of eye-gaze tracking is being increasingly considered, the face is most often regarded as a rigid body. The few methods that factor the challenge of handling face deformations into the head pose estimation problem, often require the availability of a pre-defined face model or a considerable amount of training data. In this paper, we direct our attention towards the application of shape-and- motion factorisation for head pose estimation, since this does not generally rely on the availability of an initial face model. Over the years, various shape-and-motion factorisation methods have been proposed to address the challenges of rigid and non-rigid shape and motion recovery, in a batch or sequential manner. However, the real-time recovery of non-rigid shape and motion by factorisation remains, in general, an open problem. Our work addresses this open problem by proposing a sequential factorisation method for non-rigid shape and motion recovery, which does not rely on the availability of a pre-defined face deformation model or training data. Quantitative and qualitative results show that our method can handle various non-rigid face deformations without deterioration of the head pose estimation accuracy.

Sequential Non-Rigid Factorisation for Head Pose Estimation


This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

Distilling Spikes: Knowledge Distillation in Spiking Neural Networks


Cardiac left ventricular (LV) segmentation is a paramount essential step for both diagnosis and treatment of cardiac pathologies such as ischemia, myocardial infarction, arrhythmia, and myocarditis. However, this segmentation is challenging due to high variability across patients and the potential lack of contrast between structures. In this work, we propose and evaluate a (2.5D) SegU-Net model based on the fusion of two deep learning segmentation techniques (U-Net and Seg-Net) for automated LGE-MRI (Late gadolinium enhanced magnetic resonance imaging) myocardial disease (infarct core and no-reflow region) quantification in a new multifield expert annotated dataset. Given that the scar tissue represents a small part of the whole MRI slices, we focused on the myocardium area. Segmentation results show that this preprocessing step facilitates the learning procedure. In order to solve the class imbalance problem, we propose to apply the Jaccard loss and the Focal Loss as optimization loss function and to integrate a class weights strategy into the objective function. Late combination has been used to merge the output of the best trained models on a different set of hyperparameters. The final network segmentation performances will be useful for future comparison of new methods to the current related work for this task. A total number of 2237 of slices (320 cases) were used for training/validation and 210 slices (35 cases) were used for testing. Experiments on our proposed dataset, using several evaluation metrics such as Jaccard distance (IOU), Accuracy, and Dice similarity coefficient (DSC), demonstrate efficiency performance in quantifying different zones of myocardium infarction across various patients. As compared to the second intra-observer study, our testing results showed that the SegU-Net prediction model leads to these average Dice coefficients overall segmented tissue classes, respectively: 'Back-ground': 0.99999, 'Myocardium': 0.99434, 'Infarctus': 0.95587, 'Noreflow': 0.78187 outperforming seven previously proposed methods.

Premium content

Locally-Connected, Irregular Deep Neural Networks for Biomimetic Active Vision in a Simulated Human

Downloads

Next from ICPR 2020

Median-shape Representation Learning for Category-level Object Pose Estimation in Cluttered Environments

Similar lecture

MFPP: Morphological Fragmental Perturbation Pyramid for Black-Box Model Explanations