Italy

We tackle depth completion task. Based on our assumption, color image and depth map share some structural information, we proposed DEN network that disentangle and exchange the structural information of color and depth domain to accomplish depth completion. In this presentation, we&#39;ll show the mechanism and design ideas of our method.

ICPR 2020

DEN: Disentangling and Exchanging Network for Depth Completion

depth completion

style transfer

domain adaptation

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


We tackle depth completion task. Based on our assumption, color image and depth map share some structural information, we proposed DEN network that disentangle and exchange the structural information of color and depth domain to accomplish depth completion. In this presentation, we'll show the mechanism and design ideas of our method.

technical paper

Non-negative matrix factorization (NMF) is an important method in learning latent data representation. The local geometrical structure can make the learned representation more effectively and significantly improve the performance of NMF. However, most of existing graph-based learning methods are determined by a predefined similarity graph which may be not optimal for specific tasks. To solve the above problem, we propose the Double Manifolds Regularized NMF (DMR-NMF) model which jointly learns an adaptive affinity matrix with the non-negative matrix factorization. The learned affinity matrix can guide the NMF to fit the clustering task. Moreover, we develop the iterative updating optimization schemes for DMR-NMF, and provide the strict convergence proof of our optimization strategy. Empirical experiments on four different real-world data sets demonstrate the state-of-the-art performance of DMR-NMF in comparison with the other related algorithms.

Double Manifolds Regularized Non-negative Matrix Factorization for Data Representation

Clustering high dimension multi-view data with complex intrinsic properties and nonlinear manifold structure is a challenging task since these data are always embedded in low dimension manifolds. Inspired by Low Rank Representation (LRR), some researchers extended classic LRR on Grassmann manifold or Product Grassmann manifold to represent data with non-linear metrics. However, most of these methods utilized convex nuclear norm to leverage a low-rank structure, which was over-relaxation of true rank and would lead to the results deviated from the true underlying ones. And, the computational complexity of singular value decomposition of matrix is high for nuclear norm minimization. In this paper, we propose a new low rank model for high-dimension multi-view data clustering on Product Grassmann Manifold with the matrix tri-factorization which is used to control the upper bound of true rank of representation matrix. And, the original problem can be transformed into the nuclear norm minimization with smaller scale matrices. An effective solution and theoretical analysis are also provided. The experimental results show
that the proposed method obviously outperforms other state-of-the-art methods on several multi-source human/crowd action video datasets.

Low Rank Representation on Product Grassmann Manifolds for Multi-view Subspace Clustering

Face synthesis is an interesting yet challenging task in computer vision. It is even much harder to generate a portrait video than a single image. In this paper, we propose a novel video generation framework for synthesizing arbitrarylength face videos without any face exemplar or landmark. To overcome the synthesis ambiguity of face video, we propose a divide-and-conquer strategy to separately address the video face synthesis problem from two aspects, face identity synthesis and rearrangement. To this end, we design a cascaded network which contains three components, Identity-aware GAN (IA-GAN), Face Coherence Network, and Interpolation Network. IA-GAN is proposed to synthesize photorealistic faces with the same identity from a set of noises. Face Coherence Network is designed to re-arrange the faces generated by IA-GAN while keeping the inter-frame coherence. Interpolation Network is introduced to eliminate the discontinuity between two adjacent frames and improve the smoothness of the face video. Experimental results demonstrate that our proposed network is able to generate face video with high visual quality while preserving the identity. Statistics show that our method outperforms state-of-the-art unconditional face video generative models in multiple challenging datasets.

Coherence and Identity Learning for Arbitrary-length Face Video Generation


Deep clustering aims to cluster unlabeled real-world samples by mining deep feature representation. Most of existing methods remain challenging when handling high-dimensional data and simultaneously exploring the complementarity of deep
feature representation and clustering. In this paper, we propose a novel Deep Variational Attention Encoder-decoder for Clustering (DVAEC). Our DVAEC improves the representation learning ability by fusing variational attention. Specifically, we design a feature-aware automatic clustering module to mitigate the
unreliability of similarity calculation and guide network learning. Besides, to further boost the performance of deep clustering from a global perspective, we define a joint optimization objective to promote feature representation learning and automatic clustering synergistically. Extensive experimental results show the promising
performance achieved by our DVAEC on six datasets comparing with several popular baseline clustering methods.

Feature-aware unsupervised learning with joint variational attention and automatic clustering


Motivated by the following two observations: 1)
people are aging differently under different conditions for
changeable facial attributes, e.g., skin color may become darker
when working outside, and 2) it needs to keep some unchanged
facial attributes during the aging process, e.g., race and gender,
we propose a controllable face aging method via attribute
disentanglement generative adversarial network. To offer fine
control over the synthesized face images, first, an individual
embedding of the face is directly learned from an image that
contains the desired facial attribute. Second, since the image may
contain other unwanted attributes, an attribute disentanglement
network is used to separate the individual embedding and learn
the common embedding that contains information about the
face attribute (e.g., race). With the common embedding, we can
manipulate the generated face image with the desired attribute
in an explicit manner. Experimental results on two common
benchmarks demonstrate that our proposed generator achieves
comparable performance on the aging effect with state-of-theart baselines while gaining more flexibility for attribute control.

Controllable Face Aging

In this paper, we propose a novel network training mechanism called ”dynamic channel propagation” to prune the neural networks during the training period. In particular, we pick up a specific group of channels in each convolutional layer to participate in the forward propagation in training time according to the significance level of channel, which is defined as channel utility. The utility values with respect to all selected channels are updated simultaneously with the error back-propagation process and will adaptively change. Furthermore, when the training ends, channels with high utility values are retained whereas those with low utility values are discarded. Hence, our proposed scheme trains and prunes neural networks simultaneously. We empirically evaluate our novel training scheme on various representative benchmark datasets and advanced convolutional neural network (CNN) architectures, including VGGNet and ResNet. The experiment results verify the superior performance and robust effectiveness of our approach.

Learning to Prune in Training via Dynamic Channel Propagation


In the relationship extraction task of NLP, how to effective use of the rich structural information on the dependency tree is a challenging research problem. To better learn the dependency relationship between nodes, we address the relationship extraction task by capturing rich contextual dependencies based on the attention mechanism, and using distributional reinforcement learning to generate optimal relation information representation. Unlike using an attention mechanism to effectively make use of relevant information, we propose a Dual Attention Graph Convolutional Network (DAGCN) to adaptively integrate local features with their global dependencies. Specifically, we append two types of attention modules on top of GCN, which model the semantic interdependencies in spatial and relational dimensions respectively. The position attention module selectively aggregates the feature at each position by a weighted sum of the features at all positions of nodes internal features. Similar features would be related to each other regardless of their distances. Meanwhile, the relation attention module selectively emphasizes interdependent node relations by integrating associated features among all nodes. We sum the outputs of the two attention modules and use reinforcement learning to predict the classification of nodes relationship to further improve feature representation which contributes to more precise extraction results. The results on the TACRED and SemEval datasets show that the model can obtain more useful information for relational extraction tasks, and achieve better performances on various evaluation indexes.

Adaptive Graph Convolutional Networks with Attention Mechanism for Relation Extraction

In order to improve the performance of two-stage object detection and consider the importance of scene and semantic information for visual recognition, the neural network of object detection algorithm is studied and analyzed in this paper. The main research work of this paper includes: A scene level region proposal self-attention object detection model based on depth separable convolution is proposed. In order to obtain stronger semantic information and context information of the target scene, the scene-level region proposal self-attention module is reconstructed based on the process of region proposal recognition. The feature map of the output feature pyramid network is sent into three parallel branches: semantic segmentation module, candidate area network module and region proposal self-attention module. At the same time, for the overall performance of the model, a deep separable convolutional network module is constructed on the backbone network, which includes six stages. In the fifth to sixth stage of the network, the separable convolutional network module is integrated respectively. Finally, a object detection method based on border regression network enhancement is proposed to achieve accurate target location. In order to verify the effectiveness of each model, the experimental results of each model are analyzed.

Object Detection Model Based on Scene-Level Region Proposal Self-Attention


With the increasing health concerns on diet, it’s worthwhile to develop an intelligent assistant that can help users eat healthier. Such assistants can automatically give personal advice for the users’ diet and generate health reports about eating on a regular basis. To boost the research on such diet assistant, we establish a real-world foodlog database using various methods such as filter, cluster and graph convolutional network. This database is built based on real-world lifelog and medical data, which is named as Real-World Multimodal Foodlog (RWMF). It contains 7500 multimodal pairs, and each pair consists of a food image paired with a line of personal biometrics data (such as Blood Glucose) and a textual food description of food composition paired with a line of food nutrition data. In this paper, we present the detailed procedures for setting up the database. We evaluate the performance of RWMF using different food classification and cross-modal retrieval approaches. We also test the performance of multimodal fusion on RWMF through ablation experiments. The experimental results show that the RWMF database is quite challenging and can be widely used to evaluate the performance of food analysis methods based on multimodal data.

RWMF: A Real-World Multimodal Foodlog Database

In many applications we want to find the number of clusters in a dataset. A common approach is to use a penalized k-means algorithm with an additive penalty term linear in the number of clusters. Obviously, the number of discovered clusters depends critically on the value of the coefficient of the penalty term, and an open problem is estimating the value of the coefficient in a principled manner. In this paper, we derive rigorous bounds for the coefficient of the additive penalty in k-means for ideal clusters. Although in practice clusters typically deviate from the ideal assumption, the ideal case serves as a useful guideline. Furthermore, we investigate
k-means with multiplicative penalty, which generally produces a more reliable signature, compared to additive penalty, for the correct number of clusters in cases where the ideal cluster assumption holds. We also empirically investigate certain types of deviations from ideal cluster assumption. In such cases both types of penalties may suggest multiple, ambiguous solutions. We present a consensus-based approach to resolving these ambiguous solutions by combining the results of additive and multiplicative penalties.

Premium content

DEN: Disentangling and Exchanging Network for Depth Completion

Downloads

Next from ICPR 2020

Double Manifolds Regularized Non-negative Matrix Factorization for Data Representation

Similar lecture

DEN: Disentangling and Exchanging Network for Depth Completion