Italy


Reconstruction of 3D human bodies, from images, faces many challenges, due to it generally being an ill-posed problem. In this paper we present a method to reconstruct 3D human bodies from multi-view images, through learning an
implicit function to represent 3D shape, based on multi-scale features extracted by multi-stage end-to-end neural networks. Our model consists of several end-to-end hourglass networks for extracting multi-scale features from multi-view images, and a
fully connected network for implicit function classification from these features. Given a 3D point, it is projected to multi-view images and these images are fed into our model to extract multi-scale features. The scales of features extracted by the hourglass networks decrease with the depth of our model, which represents
the information from local to global scale. Then, the multi-scale features as well as the depth of the 3D point are combined to a new feature vector and the fully connected network classifies the feature vector, in order to predict if the point lies inside or outside of the 3D mesh. The advantage of our method is that we use both local and global features in the fully connected network and represent the 3D mesh by an implicit function, which is more memory-efficient. Experiments on public datasets demonstrate that our method surpasses previous approaches in terms of the accuracy of 3D reconstruction of human bodies from images.

ICPR 2020

Learning to Implicitly Represent 3D Human Body From Multi-scale Features and Multi-view Images

multi-scale features

multi-view images

implicit function

hourglass network

human body

3d reconstruction

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster


Ghost-like artifact caused by ill-exposed and motion areas is one of the most challenging problems in high dynamic range (HDR) image reconstruction. When the motion range is small, previous methods based on optical flow or patch-match can suppress ghost-like artifacts by first aligning input images before merging them. However, they are not robust enough and still produce artifacts for challenging scenes where large foreground motions exist. To this end, we propose a deep network with an attention module and motion-emphasized loss function to produce ghost-free HDR images. In the attention module, we use the channel and spatial attention to guide the network to emphasize important components such as motion and saturated areas automatically. To be robust to images with different resolutions and objects with distinct scales, we adopt the selective kernel network as the basic framework for channel attention. In addition to the attention module, the motion-emphasized loss function based on the motion and ill-exposed areas mask is designed to help the network reconstruct motion areas. Experiments on the public dataset indicate that the proposed SK-AHDRNet produces ghost-free results where detail in ill-exposed areas is well recovered. The proposed method scores 43.17 with PSNR metric and 61.02 with HDR-VDP-2 metric on test which outperforms all conventional works. According to quantitative and qualitative evaluations, the proposed method can achieve state-of-the-art performance.

Selective Kernel and Motion-emphasized Loss Based Attention-guided Network for HDR Imaging of Dynamic Scenes


Video anomaly detection is a challenging task not only because it involves solving many sub-tasks such as motion representation, object localization and action recognition, but also because it is commonly considered as an unsupervised learning problem that involves detecting outliers. Traditionally, solutions to this task have focused on the mapping between video frames and their low-dimensional features, while ignoring the spatial connections of those features. Recent solutions focus on analyzing these spatial connections by using hard clustering techniques, such as K-Means, or applying neural networks to map latent features to a general understanding, such as action attributes. In order to solve video anomaly in the latent feature space, we propose a deep probabilistic model to transfer this task into a density estimation problem where latent manifolds are generated by a deep denoising autoencoder and clustered by expectation maximization. Evaluations on several benchmarks datasets show the strengths of our model, achieving outstanding performance on challenging datasets.

Video Anomaly Detection by Estimating Likelihood of Representations

Human beings not only have the ability to recognize novel unseen classes, but also can incrementally incorporate the new classes to existing knowledge preserved. However, zero- shot learning models assume that all seen classes should be known beforehand, while incremental learning models cannot recognize unseen classes. This paper introduces a novel and challenging task of Incrementally Zero-Shot Detection (IZSD), a practical strategy for both zero-shot learning and class- incremental learning in real-world object detection. An innovative end-to-end model – IZSD-EVer was proposed to tackle this task that requires incrementally detecting new classes and detecting the classes that have never been seen. Specifically, we propose a novel extreme value analyzer to detect objects from old seen, new seen, and unseen classes, simultaneously. Additionally and technically, we propose two innovative losses, i.e., background- foreground mean squared error loss alleviating the extreme imbalance of the background and foreground of images, and projection distance loss aligning the visual space and semantic spaces of old seen classes. Experiments demonstrate the efficacy of our model in detecting objects from both the seen and unseen classes, outperforming the alternative models on Pascal VOC and MSCOCO datasets.

Incrementally Zero-Shot Detection by an Extreme Value Analyzer

Knowledge distillation, which aims to train model under the supervision from another large model (teacher model) to the original model (student model), has achieved remarkable results in supervised learning. However, there are two major
problems with existing knowledge distillation methods. One is the teacher’s supervision is sometimes misleading, and the other is the student’s prediction is not accurate enough. To address the first issue, instead of learning a combination of
both teachers and ground truth, we apply knowledge adjustment to correct teachers’ supervision using ground truth. For the second problem, we use the selective classification framework to train the student model. In particular, the deep gambler loss is adopted to predict with reservation by explicitly introducing
the (m + 1)-th class. We consider two settings of knowledge distillation: (1) distillation across different network structures (AlexNet, ResNet), and (2) distillation across networks with different depths (ResNet18, ResNet50) to evaluate the effectiveness of our method. The experimental results on benchmark datasets
(i.e., Fashion-MNIST, SVHN, CIFAR10, CIFAR100) are reported with higher prediction accuracies and lower coverage errors.

Knowledge Distillation with a Precise Teacher and Prediction with Abstention


A breif introduction of the proposed method for diagnosis of COVID-19 on ct images

Unsupervised Detection of Pulmonary Opacities for Computer-Aided Diagnosis of COVID-19 on CT Images

Distributed Gaussian processes (DGPs) are prominent
local approximation methods to scale Gaussian processes
(GPs) to large datasets. Instead of a global estimation, they
train local experts by dividing the training set into subsets,
thus reducing the time complexity. This strategy is based on the
conditional independence assumption, which basically means that
there is a perfect diversity between the local experts. In practice,
however, this assumption is often violated, and the aggregation
of experts leads to sub-optimal and inconsistent solutions. In
this paper, we propose a novel approach for aggregating the
Gaussian experts by detecting strong violations of conditional
independence. The dependency between experts is determined
by using a Gaussian graphical model, which yields the precision
matrix. The precision matrix encodes conditional dependencies
between experts and is used to detect strongly dependent experts
and construct an improved aggregation. Using both synthetic
and real datasets, our experimental evaluations illustrate that
our new method outperforms other state-of-the-art (SOTA) DGP
approaches while being substantially more time-efficient than
SOTA approaches, which build on independent experts.

Aggregating Dependent Gaussian Experts in Local Approximation

We develop a novel multiple instance SVM based on maximizing the minimum witness rate (WR) among positive bags. We solve this nonlinear integer programming problem by the multiple instance SVM formulation with a convex loss function, where unknown integer labels of instances in positive bags are relaxed to real numbers between -1 and 1 using the tanh(.) to estimate WR of a positive bag. The results of WR-SVM outperform the state-of-the-art for MIL-based video classification models.


A New Convex Loss Function For Multiple Instance Support Vector Machines


Situation in Image compression:  Although existing methods based on a fully convolutional network have greatly improved the performance, the reconstructed results still do not show satisfactory on the structural similarity and statistical distribution similarity of the whole image.
CNN's Shortcoming: Overdownsampling thus losing details
Proposal: In this work, we propose an GAN based dense Auto-encoder (GDAE) and Semantic-perceptual residual compensation method (SPRC) to improve the coarse results generated by front CNN and compensate the details of reconstructed image.

Adaptive Image Compression Method Using GAN based Semantic-Perceptual Residual Compensation

In this paper we propose a novel method for location recognition and prediction in smart homes based on semi-supervised learning. We use data collected from low-resolution depth video cameras installed in four apartments with older adults over 70 years of age, and collected during a period of one to seven weeks. The location of the person in the depth images is detected by a person detection algorithm adapted from YOLO (You Only Look Once). The locations extracted from the videos are then clustered using K-means clustering. Sequence prediction algorithms are used to predict the next cluster (location) based on the previous clusters (locations). The accuracy of predicting the next location is up to 91\%, a significant improvement compared to the case where binary sensors are placed in the apartment based on human intuition. The paper presents an analysis on the effect of the memory length (i.e. the number of previous clusters used to predict the next one), and on the amount of recorded data required to converge.

Location Prediction in Real Homes of Older Adults based on K-Means in Low-Resolution Depth Videos

This paper focuses on the problem of visual saliency prediction, predicting regions of an image that tend to attract human visual attention, under a constrained computational budget. We modify and test various recent efficient convolutional neural network architectures like EfficientNet and MobileNetV2 and compare them with existing state-of-the-art saliency models such as SalGAN and DeepGaze II both in terms of standard accuracy metrics like Area Under Curve (AUC) and Normalized Scanpath Saliency (NSS), and in terms of the computational complexity and model size. We find that MobileNetV2 makes an excellent backbone for a visual saliency model and can be effective even without a complex decoder. We also show that knowledge transfer from a more computationally expensive model like DeepGaze II can be achieved via pseudo-labelling an unlabelled dataset, and that this approach gives result on-par with many state-of-the-art algorithms with a fraction of the computational cost and model size.

Premium content

Learning to Implicitly Represent 3D Human Body From Multi-scale Features and Multi-view Images

Downloads

Next from ICPR 2020

Selective Kernel and Motion-emphasized Loss Based Attention-guided Network for HDR Imaging of Dynamic Scenes

Similar lecture

P2D: a self-supervised method for depth estimation from polarimetry