Italy


The deep learning technique has recently led to significant improvement in object detection accuracy. In many applications, object detection is performed on video data consisting of a sequence of two-dimensional (2D) image frames. Numerous object detection schemes have been designed to detect objects independently in each video frame. Though temporal information within adjacent image frames can be exploited in subsequent object tracking stage, it has been shown that the object detection accuracy can be significantly improved by exploiting the temporal structure in the image sequence in the object detection stage. In this paper, we propose a novel video object detection method that exploits both the motion context inferred from the adjacent frames and the spatio-temporal features aggregated over the image sequence. First, correlation between the spatial feature maps over two adjacent frames are computed and the embedding vector, representing the motion context, is obtained by encoding the N correlation maps using long short term memory (LSTM). In addition to utilizing the motion context, the spatial feature maps for N+1 consecutive frames are aggregated to boost the quality of the feature map. The gated attention network is employed to selectively combine the temporal feature maps based on their relevance to the feature map in the present image frame. While most video object detectors have been developed for two-stage object detectors, our proposed idea applies to one-stage detectors with the advantage of low computational complexity in practical real-time applications. Our numerical evaluation conducted on the ImageNet object detection from video (VID) dataset demonstrates that our proposed network achieves significant performance gain over the baseline algorithms and outperforms the existing one-stage video object detectors.

ICPR 2020

Video Object Detection Using Object’s Motion Context and Spatio-Temporal Feature Aggregation

motion and tracking

video analysis

object detection

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster


Learning-based stereo matching has recently achieved promising results, yet still suffers difficulties in establishing reliable matches in weakly matchable regions that are textureless, non-Lambertian, or occluded. In this paper, we address this challenge by proposing a stereo matching network that considers pixel-wise matchability. Specifically, the network jointly regresses disparity and matchability maps from 3D probability volume through expectation and entropy operations. Next, a learned attenuation is applied as the robust loss function to alleviate the influence of weakly matchable pixels in the training. Finally, a matchability-aware disparity refinement is introduced to improve the depth inference in weakly matchable regions. The proposed deep stereo matchability (DSM) framework can
improve the matching result or accelerate the computation while still guaranteeing the quality. Moreover, the DSM framework is portable to many recent stereo networks. Extensive experiments are conducted on Scene Flow and KITTI stereo datasets to demonstrate the effectiveness of the proposed framework over the state-of-the-art learning-based stereo methods.

Learning Stereo Matchability in Disparity Regression Networks

Facial actions are spatio-temporal signals by nature, and therefore their modeling is crucially dependent on the availability of temporal information. In this paper, we focus on inferring such temporal dynamics of facial actions when no explicit temporal information is available, i.e. from still images. We present a novel self-supervised learning approach to capture multiple scales of temporal dynamics, with an application to facial Action Unit (AU) intensity estimation and dimensional affect estimation. In particular: 1. We propose a framework that infers a dynamic representation (DR) from a still image, capturing the bi-directional flow of time within a short time-window centered at the input image; 2. We show that the proposed rank loss can apply facial temporal evolution to self-supervise the training process without using target representations, allowing the network to represent dynamics more broadly; 3. We propose a multiple temporal scale approach that infers DRs for different window lengths (MDR) from a still image. We empirically validate the value of our approach on the task of frame ranking, and show how our proposed MDR attains state of the art results on BP4D for AU intensity estimation and on SEMAINE for dimensional affect estimation, using only still images at test time.

Self-supervised learning of Dynamic Representations for Static Images

This paper presents a proposal of a faster Wasserstein k-means algorithm for histogram data by reducing Wasserstein distance computations and exploiting sparse simplex projection. We shrink data samples, centroids, and the ground cost matrix, which leads to considerable reduction of the computations used to solve optimal transport problems without loss of clustering quality. Furthermore, we dynamically reduced the computational complexity by removing lower-valued data samples and harnessing sparse simplex projection while keeping the degradation of clustering quality lower. We designate this proposed algorithm as sparse simplex projection based Wasserstein k-means, or SSPW k-means. Numerical evaluations conducted with comparison to results obtained using Wasserstein k-means algorithm demonstrate the effectiveness of the proposed SSPW k-means for real-world datasets.

Wasserstein k-means with sparse simplex projection

Pretraining Image Encoders without Reconstruction via Feature Prediction Loss


In the past, super-resolution networks based on the attention mechanism seldom paid attention to the relationship of the internal feature maps of the network. The attention module we designed can refine the feature map information, improve the nonlinear expression ability of the network, and achieve better reconstruction effects.

Pay more attention to network feature maps

Thermodynamic characterisations or analogies have proved to provide powerful tools for the statistical analysis of network populations or time series, together with the identification of structural anomalies that occur within them. For instance, classical Boltzmann statistics together with the corresponding partition function have been used to apply the tools of statistical physics to the analysis of variations in network structure. However, the physical analogy adopted in this analysis, together with the interpretation of the resulting system of particles is sometimes vague and remains an open question. This, in turn, has implications concerning the definition of quantities such as temperature and energy. In this paper, we take a novel view of the thermal characterisation where we regard the edges in a network as the particles of the thermal system. By considering networks with a fixed number of nodes we obtain a conservation law which applies to the particle occupation configuration. Using this interpretation, we provide a physical meaning for the temperature which is related to the number of network nodes and edges. This provides a fundamental description of a network as a thermal system. If we further interpret the elements of the adjacency matrix as the binary microstates associated with edges, this allows us to further extend the analysis to systems with edge-weights. We thus introduce the concept of the canonical ensemble into the thermal network description and the corresponding partition function and then use this to compute the thermodynamic quantities. Finally, we provide numerical experiments on synthetic and real-world data-sets to evaluate the thermal characterisations for both unweighted and weighted networks.

Thermal Characterisation of Unweighted and Weighted Networks


CTC and Attn are the two most popular methods incorporated with CNNs, RNNs or some other basic deep learning components. Attn based approaches are more accurate in most scenarios while CTC based ones achieve better efficiency and are easier to train. However, some drawbacks still exist in these two mainstream methods respectively. In this paper, to deal with the problems mentioned above, a Recified Attentional Double Supervised network named ReADS is proposed.

ReADS: A Rectified Attentional Double Supervised Network for Scene Text Recognition

Recent studies have shown remarkable success in face image generations. However, most of the existing methods only generate face images from random noise, and cannot generate face images according to the specific attributes. In this paper, we focus on the problem of face synthesis from attributes, which aims at generating faces with specific characteristics corresponding to the given attributes. To this end, we propose a novel attributes aware face image generator method with generative adversarial networks called AFGAN. Specifically, we firstly propose a two-path embedding layer and self-attention mechanism to convert binary attribute vector to rich attribute features. Then three stacked generators generate 64*64, 128*128 and 256*256 resolution face images respectively by taking the attribute features as input. In addition, an image-attribute matching loss is proposed to enhance the correlation between the generated images and input attributes. Extensive experiments on CelebA demonstrate the superiority of our AFGAN in terms of both qualitative and quantitative evaluations.

Attributes Aware Face Generation with Generative Adversarial Networks

Deep feedforward convolutional neural networks (CNNs) perform well in the saliency prediction of omnidirectional images (ODIs), and have become the leading class of candidate models of the visual processing mechanism in the primate ventral stream. These CNNs have evolved from shallow network architecture to extremely deep and branching architecture to achieve superb performance in various vision tasks, yet it is unclear how brain-like they are. In particular, these deep feedforward CNNs are difficult to mapping to ventral stream structure of the brain visual system due to their vast number of layers and missing biologically-important connections, such as recurrence. To tackle this issue, some brain-like shallow neural networks are introduced. In this paper, we propose a novel brain-like network model for saliency prediction of head fixations on ODIs. Specifically, our proposed model consists of three modules: a CORnet-S module, a template feature extraction module and a ranking attention module (RAM). The CORnetS module is a lightweight artificial neural network (ANN) with four anatomically mapped areas (V1, V2, V4 and IT) and it can simulate the visual processing mechanism of ventral visual stream in the human brain. The template features extraction module is introduced to extract attention maps of ODIs and provide guidance for the feature ranking in the following RAM module. The RAM module is used to rank and select features that are important for fine-grained saliency prediction. Extensive experiments have validated the effectiveness of the proposed model in predicting saliency maps of ODIs.

Saliency Prediction on Omnidirectional Images with Brain-Like Shallow Neural Network


In this paper, we propose a graph-based feature aggregation network (G-FAN) for video face recognition. 
Compared with the still image, video face recognition exhibits great challenges due to huge intra-class variability and high inter-class ambiguity. 
To address this problem, our G-FAN first uses a Convolutional Neural Network to extract deep features for every input face of a subject. Then, we build an affinity graph based on the relationship between facial features and apply Graph Convolutional Network to generate fine-grained quality vectors for each frame. Finally, the features among multiple frames are adaptively aggregated into a discriminative vector to represent a video face. Different from previous works that take a single image as input, our G-FAN could utilize the correlation information between image pairs and aggregate a template of face images simultaneously. The experiments on video face recognition benchmarks, including YTF, IJB-A, and \IJB-C show that: i) G-FAN automatically learns to advocate high-quality frames while repelling low-quality ones. ii) G-FAN significantly boosts recognition accuracy and outperforms other state-of-the-art aggregation methods.

Premium content

Video Object Detection Using Object’s Motion Context and Spatio-Temporal Feature Aggregation

Downloads

Next from ICPR 2020

Learning Stereo Matchability in Disparity Regression Networks

Similar lecture

IPT: A Dataset for Identity Preserved Tracking in Closed Domains