Italy


6D object pose estimation is widely applied in robotic tasks such as grasping and manipulation. Prior methods using RGB-only images are vulnerable to heavy occlusion and poor illumination, so it is important to complement them with depth information. However, existing methods using RGB-D data cannot adequately exploit consistent and complementary information between RGB and depth modalities. In this paper, we present a novel method to effectively consider the correlation within and across both modalities with attention mechanism to learn discriminative and compact multi-modal features. Then, effective fusion strategies for intra- and inter-correlation modules are explored to ensure efficient information flow between RGB and depth. To our best knowledge, this is the first work to explore effective intra- and inter-modality fusion in 6D pose estimation. The experimental results show that our method can achieve the state-of-the-art performance on LineMOD and YCB-Video dataset. We also demonstrate that the proposed method can benefit a real-world robot grasping task by providing accurate object pose estimation.

ICPR 2020

6D Pose Estimation with Correlation Fusion


rgb -d

6d object pose estimation

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster


We propose a optimized framework for accurately predicting crick through rate of online video ads by taking advantage of the multimodal nature of online video ads including video, text, and metadata features. Experimental results show that our approach can achieve a correlation coefficient as high as 0.695, which is a significant improvement from the baseline (0.487).

Predicting Online Video Advertising Effects with Multimodal Deep Learning

Practical face image quality assessment (FIQA) models are trained under the supervision of labeled data, which requires more or less human labor. The human labeled quality scores are consistent with perceptual intuition but laborious. On the other hand, models can be trained with data generated automatically by the recognition models with artificially selected references. However, the recognition scores are sometimes inaccurate, which may give wrong quality scores during FIQA training. In this paper, we propose a labour-saving method for quality scores generation. For the first time, we conduct systematic investigations to show that there exist severe contradictions between different types of target quality, namely distribution gap (DG). To bridge the gap, we propose a novel framework for training FIQA models by combining the merits of data from different sources. In order to make the target score from multiple sources compatible, we design a method called quality distribution alignment (QDA). Meanwhile, to correct the wrong target by recognition models, contradictory samples selection (CSS) is adopted to select samples from the human labeled dataset adaptively. Extensive experiments and analysis on public benchmarks including MegaFace has demonstrated the superiority of our in terms of effectiveness and efficiency.

Face Image Quality Assessment for Model and Human Perception


In this paper, we propose a simple and effective framework
for detecting arbitrary shaped scene text, which detects text
regions and kernels of text-like regions respectively. Moreover,
we introduce a decay learning weight to compress the influence
of the unclear boundary of text instances. The experiments have
verified that our method consistently achieves the satisfying
results on four standard scene text benchmarks, in terms of
accuracy and speed. In the future, we will work on improving
its ability to detect small text instances and extending our
method for end-to-end text spotting.

An Accurate Threshold Insensitive Kernel Detectorfor Arbitrary Shaped Text

We proposed a smart way to inference for the task of barcoding by leveraging the checksum of barcode and probabilistic outcomes of CNN models.

Smart Inference for Multidigit Convolutional Neural Network based Barcode Decoding

Airwriting using mobile phones has many applications in human-computer interaction. However, the recognition of airwriting character needs a lot of training data from user, which brings great difficulties to the pratical application. The model learnt from a specific person often cannot yield satisfied results when used on another person. The data gap between people is mainly caused by the following factors: personal writing styles, mobile phone sensors, and ways to hold mobile phones. To address the cross-people problem, we propose a deep neural network (DNN) that combines convolutional neural network (CNN) and bilateral long short-term memory (BLSTM). In each layer of the network, we also add an AdaBN layer which is able to increase the generalization ability of the DNN. Different
from the original AdaBN method, we explore the feasibility for semi-supervised learning. We implement it to our design and conduct comprehensive experiments. The evaluation results show that our system can achieve an accuracy of 99% for recognition and an improvement of 10% on average for transfer learning between various factors such as people, devices and postures. To the best of our knowledge, our work is the first to implement cross-people airwriting recognition via motion sensor signal, which is a fundamental step towards ubiquitous sensing.

Cross-People Mobile-Phone Based Airwriting Character Recognition

Remote Sensing has become an enabling factor for a broad spectrum of applications such as disaster prevention, vector-borne disease, and climate change. These applications benefit from a higher number of satellitary images, however, acquiring a huge amount of ground truth data is expensive. 
A common solution exploits models that are pre-trained on the ImageNet dataset. This approach is limited only to the tasks involving RGB images as input.
We propose a novel representation learning procedure closely relates to colorization.
Once the model has reached good capabilities on tile colorization, we use its encoder as a feature extractor, finetuned on a remote sensing task. 


The color out of space: learning self-supervised representations for Earth Observation imagery

A more stationary and discriminative embedding is necessary for robust classification of images. We focus our attention on the newel CapsNet model and we propose the angular margin loss function in composition with margin loss.
We define a fixed classifier implemented with fixed weights vectors obtained by the vertex coordinates of a simplex polytope. Each weight vector is to be considered as the centroid of a class in the dataset. The embedding of an image is obtained through the capsule network encoding phase, that is identified as digitcaps matrix. Based on the centroids from the simplex coordinates and the embedding from the model, we compute the angular distance between the image embedding and the centroid of the
correspondent class of the image. We take this angular distance as angular margin loss. We keep the computation proposed for margin loss in the original architecture of CapsNet. We propose PolyCapsNet, as CapsNet model trained to minimise the angular between the embedding and the centroid of the class and maximise the magnitude of the embedding for the predicted class. The experiments on different
datasets demonstrate that the angular margin loss improves the capability of capsule networks with complex datasets.

Fixed Simplex Coordinates for Angular Margin Loss in CapsNet


Compressed sensing allows signal reconstruction from a few measurements. This work proposes a complete pipeline for digit recognition applied to audio reconstructed signals. The reconstruction procedure exploits the assumption that the original signal lies in the range of a generator. A pretrained generator of a Generative Adversarial Network generates audio digits. A new method for reconstruction is proposed, using only the most active segment of the signal, i.e., the segment with the highest energy. The underlying assumption is that such segment offers a more compact representation, preserving the meaningful content of signal. Cases when the reconstruction produces noise, instead of digit, are treated as outliers. In order to detect and reject them, three unsupervised indicators are used, namely, the total energy of reconstructed signal, the predictions of an one-class Support Vector Machine, and the confidence of a pretrained classifier used for recognition. This classifier is based on neural networks architectures and is pretrained on original audio recordings, employing three input representations, i.e., raw audio, spectrogram, and gammatonegram. Experiments are conducted, analyzing both the quality of reconstruction and the performance of classifiers in digit recognition, demonstrating that the proposed method yields higher performance in both the quality of reconstruction and digit recognition accuracy.

Digit Recognition Applied to Reconstructed Audio Signals Using Deep Learning

This presentation introduces a highly efficient image retrieval technique called DCNN-vForest (Decomposition Convolution Neural Network and vocabulary Forest), which aims to retrieve the relevant images to the given image query by studying the correlation between images in the image database based on decomposition. The regional and global features of the image database are first extracted using the convolution neural network, and then divided into clusters of similar images using the Kmeans algorithm. We propose a new structure called vForest (vocabulary Forest), by calculating the vocabulary tree on each cluster of images. The retrieval process benefits from the knowledge provided by the vForest, and instead of considering the whole image database, only the most similar cluster to the image query is explored. To demonstrate the usefulness of our approach, intensive experiments have been carried out on ground-truth image databases, the results reveal the superiority of DCNN-vForest against the baseline image retrieval solutions, in terms of runtime and accuracy.

Hybrid Decomposition Convolution Neural Network and Vocabulary Forest for Image Retrieval

Dear sir or madam:
My name is chenshannan, a master student in the Key Lab of Advanced Design and Intelligent Computing of Dalian University. My recent research interest include image classification and deep learning. I feel very honored to accepted by ICPR. The following i will simply Introduce my related work in four aspects.

Recently, modeling deep convolutional activations by the global second-order pooling has shown great advance on visual recognition tasks. However, most of the existing deep second-order statistical models mainly compute second-order statistics of activations of the last convolutional layer as image representations, and they seldom introduce second-order statistics into earlier layers to better fit network topology, thus limiting the representational ability to a certain extent. 

The main contributions of our paper are summarized as follows: 
A novel Second-order Attention Guided Network (SoAG-Net) is proposed for visual recognition in an end-to-end.Ours model involves a group of SoAG modules seemingly inserted into intermediate layers of deep convolutional networks.Ours model captures second-order statistics of activations to predict attention map used for guiding the learning of networks.

We also use cifar and The Street View House Numbers dataset in our Experiments.
we presented a novel second-order attention guided network (SoAG-Net), which contains conceptually simple yet effective modules conveniently plugged into earlier residual stages of ConvNet. Ours module non-trivially guides the learning of convolutional activations by attention map computed from second-order statistics of activations themselves. With such module throughout a network, SoAG-Net simultaneously gets stronger representation power and provides more nonlinearity.

Premium content

6D Pose Estimation with Correlation Fusion

Downloads

Next from ICPR 2020

Predicting Online Video Advertising Effects with Multimodal Deep Learning

Similar lecture

MixedFusion: 6D Object Pose Estimation from Decoupled RGB-Depth Features