Italy

Generating images from the sound is a challenging task. This paper proposes a novel deep learning model that generates bird images from their corresponding sound information. Our proposed model includes a sound encoder in order to extract suitable feature representations from audio recordings, and then it generates bird images that correspond to its calls using conditional generative adversarial networks (GANs) with auxiliary classifiers. We demonstrate that our model produces better image generation results which outperform other state-of-the-art methods in a similar context.

ICPR 2020

S2I-Bird: Sound-to-Image Generation of Bird Species using Generative Adversarial Networks

audio-visual generation

cross-modal learning

generative adversarial networks

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster

Transceivers used for telecommunications transmit and receive specific modulation patterns that are represented as sequences of complex numbers. Classifying modulation patterns is challenging because noise and channel impairments affect the signals in complicated ways such that the received signal bears little resemblance to the transmitted signal. Although deep learning approaches have shown great promise over statistical methods in this problem space, deep learning frameworks continue to lag in support for complex-valued data. To address this gap, we study the implementation and use of complex convolutions in a series of convolutional neural network architectures. Replacement of data structure and convolution operations by their complex generalization in an architecture improves performance, with statistical significance, at recognizing modulation patterns in complex-valued signals with high SNR after being trained on low SNR signals. This suggests complex-valued convolutions enables networks to learn more meaningful representations. We investigate this hypothesis by comparing the features learned in each experiment by visualizing the inputs that results in one-hot modulation pattern classification for each network.

Modulation Pattern detection Using Complex Convolutions in Deep Learning


Visual patterns usually appear at different
scales/sizes in natural images. Multi-scale feature representation
is of great importance for the single-image super-resolution
(SISR) task to reconstruct image objects at different scales.
However, such characteristic has been rarely considered by CNNbased
SISR methods. In this work, we propose a novel building
block, i.e. hierarchically aggregated residual transformation
(HART), to achieve multi-scale feature representation in each
layer of the network. Within each HART block, we connect
multiple convolutions in a hierarchical residual-like manner,
which efficiently provides a wide range of effective receptive fields
at a more granular level to detect both local and global image
features. To theoretically understand the proposed HART block,
we recast SISR as an optimal control problem and show that
HART effectively approximates the classical 4th-order Runge-
Kutta method, which has the merit of small local truncation
error for solving numerical ordinary differential equation. By
cascading the proposed HART blocks, we establish our highperforming
HARTnet. Through extensive experiments on various
benchmark datasets under different degradation models, we
demonstrate that HARTnet compares favourably against existing
state-of-the-art methods (including those in the NTIRE 2019 SR
Challenge leaderboard) in terms of both quantitative metrics
and visual quality. Moreover, the same HARTnet architecture
achieves promising performance on such other image restoration
tasks as image denoising and low-light image enhancement.

Hierarchically Aggregated Residual Transformation
for Single Image Super Resolution


This presentation introduces our proposed paper "Label Self-Adaption Hashing for Image Retrieval". We will give the background and contribution of our work at first. Then, the related works are briefly reviewed. Motivated by previous works, we propose a novel method for unsupervised hashing and give its optimization objective. An iterative algorithm is well-designed to solve the resulting optimization problem. Extensive experiments demonstrate the effectiveness of the proposed method.

Label Self-Adaption Hashing for Image Retrieval


Semantic segmentation is a challenging and computationally heavy task in computer vision. Recent works have shown promising results applied to real-time semantic segmentation tasks such as autonomous driving cars, scene recognition, and robot navigation systems. In order to maintain fast inference speed, most of the existing networks for real-time semantic segmentation make use of light decoders, or they simply do not use them at all. This strategy helps to maintain their fast inference speed; however, their accuracy performance is significantly lower in comparison to non-real-time semantic segmentation networks. In order to reduce the accuracy gap between real-time and non-real-time segmentation networks, we introduce two key modules aimed to improve the decoder of a high-performance baseline network for real-time semantic segmentation. Our first module, Dilated Asymmetric Pyramidal Fusion (DAPF), is designed to substantially increase the receptive field on the top of the last stage of the encoder, obtaining richer contextual features. Our second module, Multi-resolution Dilated Asymmetric (MDA) module, fuses multi-resolution feature maps coming from both, early and deeper stages of the network, simultaneously refining detail and contextual information in multiple stages of the decoder. Both modules exploit contextual information without excessively increasing the computational complexity of the network. Our proposed network entitled “FASSDNet” reaches 78.8% of mIoU accuracy on Cityscapes validation dataset at 41.1 FPS on full resolution images (1024x2048). In addition, with a lite version of our network, we reach 74.1% of mIoU at 133.1 FPS (full resolution) on a single NVIDIA GTX 1080Ti card with no additional acceleration techniques.

Fast and Accurate Real-Time Semantic Segmentation with Dilated Asymmetric Convolutions

The edge computing paradigm places compute-capable devices - edge servers - at the network edge to assist mobile devices in executing data analysis tasks. Intuitively, offloading compute-intense tasks to edge servers can reduce their execution time. However, poor conditions of the wireless channel connecting the mobile devices to the edge servers may degrade the overall capture-to-output delay achieved by edge offloading. Herein, we focus on edge computing supporting remote object detection by means of Deep Neural Networks (DNNs), and develop a framework to reduce the amount of data transmitted over the wireless link. The core idea we propose builds on recent approaches splitting DNNs into sections - namely head and tail models - executed by the mobile device and edge server, respectively. The wireless link, then, is used to transport the output of the last layer of the head model to the edge server, instead of the DNN input. Most prior work focuses on classification tasks and leaves the DNN structure unaltered. Herein, our focus is on DNNs for three different object detection tasks, which present a much more convoluted structure, and modify the architecture of the network to: (i) achieve in-network compression by introducing a bottleneck layer in the early layers on the head model, and (ii) prefilter pictures that do not contain objects of interest using a convolutional neural network. Results show that the proposed technique represents an effective intermediate option between local and edge computing in a parameter region where these extreme point solutions fail to provide satisfactory performance. The code and trained models are available at https://github.com/yoshitomo-matsubara/hnd-ghnd-object-detectors .

Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks

Human-object interaction recognition aims for identifying the relationship between a human subject and an object. Researchers incorporate global scene context into the early layers of deep Convolutional Neural Networks as a solution. They report a significant increase in the performance since generally interactions are correlated with the scene (\ie riding bicycle on the city street). However, this approach leads to the following problems. It increases the network size in the early layers, therefore not efficient. It leads to noisy filter responses when the scene is irrelevant, therefore not accurate. 
It only leverages scene context whereas human-object interactions offer a multitude of contexts, therefore incomplete. To circumvent these issues, in this work, we propose Self-Selective Context (SSC). SSC operates on the joint appearance of human-objects and context to bring the most discriminative context(s) into play for recognition. We devise novel contextual features that model the locality of human-object interactions and show that SSC can seamlessly integrate with the State-of-the-art interaction recognition models. Our experiments show that SSC leads to an important increase in interaction recognition performance, while using much fewer parameters.

Self-Selective Context for Interaction Recognition

In this paper, we study the task of facial expression recognition under strong occlusion. We are particularly interested in cases where 50\% of the face is occluded, e.g. when the subject wears a Virtual Reality (VR) headset. While previous studies show that pre-training convolutional neural networks (CNNs) on fully-visible (non-occluded) faces improves the accuracy, we propose to employ knowledge distillation to achieve further improvements. First of all, we employ the classic teacher-student training strategy, in which the teacher is a CNN trained on fully-visible faces and the student is a CNN trained on occluded faces. Second of all, we propose a new approach for knowledge distillation based on triplet loss. During training, the goal is to reduce the distance between an anchor embedding, produced by a student CNN that takes occluded faces as input, and a positive embedding (from the same class as the anchor), produced by a teacher CNN trained on fully-visible faces, so that it becomes smaller than the distance between the anchor and a negative embedding (from a different class than the anchor), produced by the student CNN. Third of all, we propose to combine the distilled embeddings obtained through the classic teacher-student strategy and our novel teacher-student strategy based on triplet loss into a single embedding vector.

We conduct experiments on two benchmarks, FER+ and AffectNet, with two CNN architectures, VGG-f and VGG-face, showing that knowledge distillation can bring significant improvements over the state-of-the-art methods designed for occluded faces in the VR setting. Furthermore, we obtain accuracy rates that are quite close to the state-of-the-art models that take as input fully-visible faces. For example, on the FER+ data set, our VGG-face based on concatenated distilled embeddings attains an accuracy rate of 82.75\% on lower-half-visible faces, which is only 2.24\% below the accuracy rate of a state-of-the-art VGG-13 that is evaluated on fully-visible faces. Given that our model sees only the lower-half of the face, we consider this to be a remarkable achievement. In conclusion, we consider that our distilled CNN models can provide useful feedback for the task of recognizing the facial expressions of a person wearing a VR headset.

Teacher-Student Training and Triplet Loss for Facial Expression Recognition under Occlusion

In this presentation, we mainly show the proposed method in our paper which deal with consistent geolocalization problem. Specifically, we learn motion trajectory with recurrent neural network to achieve geolocalization and propose two strategies solve inconsistent prediction.

Map-Based Temporally Consistent Geolocalization through Learning Motion Trajectories


We present an unsupervised deep learning method for text line segmentation that is inspired by the relative variance between text lines and spaces among text lines. Handwritten text line segmentation is important for the efficiency of further processing. A common method is to train a deep learning network for embedding the document image into an image of blob lines that are tracing the text lines. Previous methods learned such embedding in a supervised manner, requiring the annotation of many document images. This paper presents an unsupervised embedding of document image patches without a need for annotations. The number of foreground pixels over the text lines is relatively different from the number of foreground pixels over the spaces among text lines. Generating similar and different pairs relying on this principle definitely leads to outliers. However, as the results show, the outliers do not harm the convergence and the network learns to discriminate the text lines from the spaces between text lines. Remarkably, with a challenging Arabic handwritten text line segmentation dataset, VML-AHTE, we achieved superior performance over the supervised methods. Additionally, the proposed method was evaluated on the ICDAR 2017 and ICFHR 2010 handwritten text line segmentation datasets.

Unsupervised deep learning for text line segmentation

Transit of wasteful materials within the European
Union is highly regulated through a system of permits. Waste
processing costs vary greatly depending on the waste category of
a permit. Therefore, companies may have a financial incentive to
allege transporting waste with erroneous categorisation.
Our goal is to assist inspectors in selecting potentially manipulated
permits for further investigation, making their task
more effective and efficient. Due to data limitations, a supervised
learning approach based on historical cases is not possible.
Standard unsupervised approaches, such as outlier detection
and data quality-assurance techniques, are not suited since we
are interested in targeting non-random modifications in both
category and category-correlated features.
For this purpose we (1) introduce the concept of crosslier:
an anomalous instance of a category which lies across other
categories; (2) propose eXPose: a novel approach to crosslier
detection based on supervised category modelling; and (3) present
the crosslier diagram: a visualisation tool specifically designed for
domain experts to easily assess crossliers. We compare eXPose
against traditional outlier detection methods in various benchmark
datasets with synthetic crossliers and show the superior
performance of our method in targeting these instances.

Premium content

S2I-Bird: Sound-to-Image Generation of Bird Species using Generative Adversarial Networks

Downloads

Next from ICPR 2020

Modulation Pattern detection Using Complex Convolutions in Deep Learning

Similar lecture

Improving Mix-and-Separate Training in Audio-Visual Sound Source Separation with an Object Prior