Italy

Clustering is a crucial but challenging task in pattern analysis and machine learning. Recent many deep clustering methods combining representation learning with cluster techniques emerged. These deep clustering methods mainly focus on the correlation among samples and ignore the relationship between samples and their representations. In this paper, we propose a novel end-to-end clustering framework, namely variational deep embedding clustering by augmented mutual information maximization (VCAMI). From the perspective of VAE, we prove that minimizing reconstruction loss is equivalent to maximizing the mutual information of the input and its latent representation. This provides a theoretical guarantee for us to directly maximize the mutual information instead of minimizing reconstruction loss. Therefore we proposed the augmented mutual information which highlights the uniqueness of the representations while discovering invariant information among similar samples. Extensive experiments on several challenging image datasets show that the VCAMI achieves good performance. we achieve state-of-the-art ACC results for clustering on MNIST (99.5\%) and CIFAR-10 (65.4\%) to the best of our knowledge.

ICPR 2020

Variational Deep Embedding Clustering by Augmented Mutual Information Maximization

variational embedding

deep clustering

mutual information

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster

Given the wide diffusion of deep neural network architectures for computer vision tasks, several new applications are nowadays more and more feasible. 
Among them, a particular attention has been recently given to instance segmentation, by exploiting the results achievable by two-stage networks (such as Mask R-CNN or Faster R-CNN), derived from R-CNN. 
In these complex architectures, a crucial role is played by the Region of Interest (RoI) extraction layer, devoted to extracting a coherent subset of features from a single Feature Pyramid Network (FPN) layer attached on top of a backbone.

This paper is motivated by the need to overcome the limitations of existing RoI extractors which select only one (the best) layer from FPN. 
Our intuition is that all the layers of FPN retain useful information. 
Therefore, the proposed layer (called Generic RoI Extractor - GRoIE) introduces non-local building blocks and attention mechanisms to boost the performance.

A comprehensive ablation study at component level is conducted to find the best set of algorithms and parameters for the GRoIE layer. 
Moreover, GRoIE can be integrated seamlessly with every two-stage architecture for both object detection and instance segmentation tasks. 
Therefore, the improvements brought about by the use of GRoIE in different state-of-the-art architectures are also evaluated. 
The proposed layer leads up to gain a 1.1% AP improvement on bounding box detection and 1.7% AP improvement on instance segmentation.

The code is publicly available on GitHub repository at https://github.com/IMPLabUniPr/mmdetection/tree/groie_dev

A Novel Region of Interest Extraction Layer for Instance Segmentation


Visual tracking algorithms use cues like appearance, structure, motion etc. for locating an object in a video. We propose an ensemble tracker with two components. First, a Siamese tracker that learns object appearance from a static image. Second, motion information obtained from consecutive frames using a flow estimation network. The motion information is used to correct the predictions obtained by the appearance based tracking component of the ensemble. Complementary nature of the two components (appearance and motion) lead to performance improvement as observed in experiments performed on VOT2018 and VOT2019 datasets.

Siamese Fully Convolutional Tracker with Motion Correction

Generating images from the sound is a challenging task. This paper proposes a novel deep learning model that generates bird images from their corresponding sound information. Our proposed model includes a sound encoder in order to extract suitable feature representations from audio recordings, and then it generates bird images that correspond to its calls using conditional generative adversarial networks (GANs) with auxiliary classifiers. We demonstrate that our model produces better image generation results which outperform other state-of-the-art methods in a similar context.

S2I-Bird: Sound-to-Image Generation of Bird Species using Generative Adversarial Networks

Transceivers used for telecommunications transmit and receive specific modulation patterns that are represented as sequences of complex numbers. Classifying modulation patterns is challenging because noise and channel impairments affect the signals in complicated ways such that the received signal bears little resemblance to the transmitted signal. Although deep learning approaches have shown great promise over statistical methods in this problem space, deep learning frameworks continue to lag in support for complex-valued data. To address this gap, we study the implementation and use of complex convolutions in a series of convolutional neural network architectures. Replacement of data structure and convolution operations by their complex generalization in an architecture improves performance, with statistical significance, at recognizing modulation patterns in complex-valued signals with high SNR after being trained on low SNR signals. This suggests complex-valued convolutions enables networks to learn more meaningful representations. We investigate this hypothesis by comparing the features learned in each experiment by visualizing the inputs that results in one-hot modulation pattern classification for each network.

Modulation Pattern detection Using Complex Convolutions in Deep Learning


Visual patterns usually appear at different
scales/sizes in natural images. Multi-scale feature representation
is of great importance for the single-image super-resolution
(SISR) task to reconstruct image objects at different scales.
However, such characteristic has been rarely considered by CNNbased
SISR methods. In this work, we propose a novel building
block, i.e. hierarchically aggregated residual transformation
(HART), to achieve multi-scale feature representation in each
layer of the network. Within each HART block, we connect
multiple convolutions in a hierarchical residual-like manner,
which efficiently provides a wide range of effective receptive fields
at a more granular level to detect both local and global image
features. To theoretically understand the proposed HART block,
we recast SISR as an optimal control problem and show that
HART effectively approximates the classical 4th-order Runge-
Kutta method, which has the merit of small local truncation
error for solving numerical ordinary differential equation. By
cascading the proposed HART blocks, we establish our highperforming
HARTnet. Through extensive experiments on various
benchmark datasets under different degradation models, we
demonstrate that HARTnet compares favourably against existing
state-of-the-art methods (including those in the NTIRE 2019 SR
Challenge leaderboard) in terms of both quantitative metrics
and visual quality. Moreover, the same HARTnet architecture
achieves promising performance on such other image restoration
tasks as image denoising and low-light image enhancement.

Hierarchically Aggregated Residual Transformation
for Single Image Super Resolution


This presentation introduces our proposed paper "Label Self-Adaption Hashing for Image Retrieval". We will give the background and contribution of our work at first. Then, the related works are briefly reviewed. Motivated by previous works, we propose a novel method for unsupervised hashing and give its optimization objective. An iterative algorithm is well-designed to solve the resulting optimization problem. Extensive experiments demonstrate the effectiveness of the proposed method.

Label Self-Adaption Hashing for Image Retrieval


Semantic segmentation is a challenging and computationally heavy task in computer vision. Recent works have shown promising results applied to real-time semantic segmentation tasks such as autonomous driving cars, scene recognition, and robot navigation systems. In order to maintain fast inference speed, most of the existing networks for real-time semantic segmentation make use of light decoders, or they simply do not use them at all. This strategy helps to maintain their fast inference speed; however, their accuracy performance is significantly lower in comparison to non-real-time semantic segmentation networks. In order to reduce the accuracy gap between real-time and non-real-time segmentation networks, we introduce two key modules aimed to improve the decoder of a high-performance baseline network for real-time semantic segmentation. Our first module, Dilated Asymmetric Pyramidal Fusion (DAPF), is designed to substantially increase the receptive field on the top of the last stage of the encoder, obtaining richer contextual features. Our second module, Multi-resolution Dilated Asymmetric (MDA) module, fuses multi-resolution feature maps coming from both, early and deeper stages of the network, simultaneously refining detail and contextual information in multiple stages of the decoder. Both modules exploit contextual information without excessively increasing the computational complexity of the network. Our proposed network entitled “FASSDNet” reaches 78.8% of mIoU accuracy on Cityscapes validation dataset at 41.1 FPS on full resolution images (1024x2048). In addition, with a lite version of our network, we reach 74.1% of mIoU at 133.1 FPS (full resolution) on a single NVIDIA GTX 1080Ti card with no additional acceleration techniques.

Fast and Accurate Real-Time Semantic Segmentation with Dilated Asymmetric Convolutions

The edge computing paradigm places compute-capable devices - edge servers - at the network edge to assist mobile devices in executing data analysis tasks. Intuitively, offloading compute-intense tasks to edge servers can reduce their execution time. However, poor conditions of the wireless channel connecting the mobile devices to the edge servers may degrade the overall capture-to-output delay achieved by edge offloading. Herein, we focus on edge computing supporting remote object detection by means of Deep Neural Networks (DNNs), and develop a framework to reduce the amount of data transmitted over the wireless link. The core idea we propose builds on recent approaches splitting DNNs into sections - namely head and tail models - executed by the mobile device and edge server, respectively. The wireless link, then, is used to transport the output of the last layer of the head model to the edge server, instead of the DNN input. Most prior work focuses on classification tasks and leaves the DNN structure unaltered. Herein, our focus is on DNNs for three different object detection tasks, which present a much more convoluted structure, and modify the architecture of the network to: (i) achieve in-network compression by introducing a bottleneck layer in the early layers on the head model, and (ii) prefilter pictures that do not contain objects of interest using a convolutional neural network. Results show that the proposed technique represents an effective intermediate option between local and edge computing in a parameter region where these extreme point solutions fail to provide satisfactory performance. The code and trained models are available at https://github.com/yoshitomo-matsubara/hnd-ghnd-object-detectors .

Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks

Human-object interaction recognition aims for identifying the relationship between a human subject and an object. Researchers incorporate global scene context into the early layers of deep Convolutional Neural Networks as a solution. They report a significant increase in the performance since generally interactions are correlated with the scene (\ie riding bicycle on the city street). However, this approach leads to the following problems. It increases the network size in the early layers, therefore not efficient. It leads to noisy filter responses when the scene is irrelevant, therefore not accurate. 
It only leverages scene context whereas human-object interactions offer a multitude of contexts, therefore incomplete. To circumvent these issues, in this work, we propose Self-Selective Context (SSC). SSC operates on the joint appearance of human-objects and context to bring the most discriminative context(s) into play for recognition. We devise novel contextual features that model the locality of human-object interactions and show that SSC can seamlessly integrate with the State-of-the-art interaction recognition models. Our experiments show that SSC leads to an important increase in interaction recognition performance, while using much fewer parameters.

Self-Selective Context for Interaction Recognition

In this paper, we study the task of facial expression recognition under strong occlusion. We are particularly interested in cases where 50\% of the face is occluded, e.g. when the subject wears a Virtual Reality (VR) headset. While previous studies show that pre-training convolutional neural networks (CNNs) on fully-visible (non-occluded) faces improves the accuracy, we propose to employ knowledge distillation to achieve further improvements. First of all, we employ the classic teacher-student training strategy, in which the teacher is a CNN trained on fully-visible faces and the student is a CNN trained on occluded faces. Second of all, we propose a new approach for knowledge distillation based on triplet loss. During training, the goal is to reduce the distance between an anchor embedding, produced by a student CNN that takes occluded faces as input, and a positive embedding (from the same class as the anchor), produced by a teacher CNN trained on fully-visible faces, so that it becomes smaller than the distance between the anchor and a negative embedding (from a different class than the anchor), produced by the student CNN. Third of all, we propose to combine the distilled embeddings obtained through the classic teacher-student strategy and our novel teacher-student strategy based on triplet loss into a single embedding vector.

We conduct experiments on two benchmarks, FER+ and AffectNet, with two CNN architectures, VGG-f and VGG-face, showing that knowledge distillation can bring significant improvements over the state-of-the-art methods designed for occluded faces in the VR setting. Furthermore, we obtain accuracy rates that are quite close to the state-of-the-art models that take as input fully-visible faces. For example, on the FER+ data set, our VGG-face based on concatenated distilled embeddings attains an accuracy rate of 82.75\% on lower-half-visible faces, which is only 2.24\% below the accuracy rate of a state-of-the-art VGG-13 that is evaluated on fully-visible faces. Given that our model sees only the lower-half of the face, we consider this to be a remarkable achievement. In conclusion, we consider that our distilled CNN models can provide useful feedback for the task of recognizing the facial expressions of a person wearing a VR headset.

Premium content

Variational Deep Embedding Clustering by Augmented Mutual Information Maximization

Downloads

Next from ICPR 2020

A Novel Region of Interest Extraction Layer for Instance Segmentation

Similar lecture

Constrained Spectral Clustering Network with Self-Training