Italy

We present a new lightweight CNN-based algorithm for multi-frame optical flow estimation.
Our solution introduces a double recurrence over spatial scale and time through repeated use of a generic &quot;STaR&quot; (SpatioTemporal Recurrent) cell. It includes (i) a temporal recurrence based on conveying learned features rather than optical flow estimates; (ii) an occlusion detection process which is coupled with optical flow estimation and therefore uses a very limited number of extra parameters. The resulting STaRFlow algorithm gives state-of-the-art performances on MPI Sintel and Kitti2015 and involves significantly less parameters than all other methods with comparable results.

ICPR 2020

STaRFlow: A SpatioTemporal Recurrent Cell for Lightweight Multi-Frame Optical Flow Estimation

multi-frame

optical flow

deep learning

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


We present a new lightweight CNN-based algorithm for multi-frame optical flow estimation.
Our solution introduces a double recurrence over spatial scale and time through repeated use of a generic "STaR" (SpatioTemporal Recurrent) cell. It includes (i) a temporal recurrence based on conveying learned features rather than optical flow estimates; (ii) an occlusion detection process which is coupled with optical flow estimation and therefore uses a very limited number of extra parameters. The resulting STaRFlow algorithm gives state-of-the-art performances on MPI Sintel and Kitti2015 and involves significantly less parameters than all other methods with comparable results.

poster

Topic modeling is widely studied for the dimension reduction and analysis of documents. However, it is formulated as a difficult optimization problem. Current approximate solutions also suffer from inaccurate model- or data-assumptions. To deal with the above problems, we propose a polynomial-time deep topic model with no model and data assumptions. Specifically, we first apply multilayer bootstrap network (MBN), which is an unsupervised deep model, to reduce the dimension of documents, and then use the low-dimensional data representations or their clustering results as the target of supervised Lasso for topic word discovery. To our knowledge, this is the first time that MBN and Lasso are applied to unsupervised topic modeling. Experimental comparison results with five representative topic models on the 20-newsgroups and TDT2 corpora illustrate the effectiveness of the proposed algorithm.

Deep Topic Modeling by Multilayer Bootstrap Network and Lasso

We present a deep transformation model for probabilistic regression. Deep learning is known for outstandingly accurate predictions on complex data but in regression tasks it is predominantly used to just predict a single number. This
ignores the non-deterministic character of most tasks. Especially
if crucial decisions are based on the predictions, like in medical
applications, it is essential to quantify the prediction uncertainty.
The presented deep learning transformation model estimates the
whole conditional probability distribution, which is the most
thorough way to capture uncertainty about the outcome. We
combine ideas from a statistical transformation model (most
likely transformation) with recent transformation models from
deep learning (normalizing flows) to predict complex outcome
distributions. The core of the method is a parameterized transformation
function which can be trained with the usual maximum
likelihood framework using gradient descent. The method can
be combined with existing deep learning architectures. For small
machine learning benchmark datasets, we report state of the art
performance for most dataset and partly even outperform it. Our
method works for complex input data, which we demonstrate by
employing a CNN architecture on image data.

Deep transformation models for tackling complex probabilistic regression problems

We propose a novel lightweight network for stereo estimation. Our network consists of a fully-convolutional densely connected neural network (FC-DCNN) that computes matching costs between rectified image pairs. Our FC-DCNN method learns expressive features and performs some simple but effective post-processing steps. The densely connected layer structure connects the output of each layer to the input of each subsequent layer. This network structure and the fact that we do not use any fully-connected layers or 3D convolutions leads to a very lightweight network. The output of this network is used in order to calculate matching costs and create a cost-volume. Instead of using time and memory-inefficient cost-aggregation methods such as semi-global matching or conditional random fields in order to improve the result, we rely on filtering techniques, namely median filter and guided filter. By computing a left-right consistency check we get rid of inconsistent values. Afterwards we use a watershed foreground-background segmentation on the disparity image with removed inconsistencies. This mask is then used to refine the final prediction. We show that our method works well for both challenging indoor and outdoor scenes by evaluating it on the Middlebury, KITTI and ETH3D benchmarks respectively. Our full framework is available at https://github.com/thedodo/FC-DCNN

FC-DCNN: A densely connected neural network for stereo estimation

In this work, we perform a wide variety of experiments with different deep learning architectures on datasets of limited size.
According to our study, we show that model complexity is a critical factor when only a few samples per class are available.
Differently from the literature, we show that in some configurations, the state of the art can be improved using low complexity models.
For instance, in problems with scarce training samples and without data augmentation, low-complexity convolutional neural networks perform comparably well or better than state-of-the-art architectures.
Moreover, we show that even standard data augmentation can boost recognition performance by large margins.
This result suggests the development of more complex data generation/augmentation pipelines for cases when data is limited.
Finally, we show that dropout, a widely used regularization technique, maintains its role as a good regularizer even when data is scarce.
Our findings are empirically validated on the sub-sampled versions of popular CIFAR-10, Fashion-MNIST and, SVHN benchmarks.

A Close Look at Deep Learning with Small Data

As a promising step, the performance of data analysis and feature learning are able to be improved if certain pattern matching mechanism is available. One of the feasible solutions can refer to the importance estimation of instances, and consequently, kernel mean matching (KMM) has become an important method for knowledge discovery and novelty detection in kernel machines. Furthermore, the existing KMM methods have focused on concrete learning frameworks. In this work, a novel approach to adaptive matching of kernel means is proposed, and selected data with high importance are adopted to achieve calculation efficiency with optimization. In addition, scalable learning can be conducted in proposed method as a generalized solution to matching of appended data. The experimental results on a wide variety of real-world data sets demonstrate the proposed method is able to give outstanding performance compared with several state-of-the-art methods, while calculation efficiency can be preserved.

Adaptive Matching of Kernel Means

Affinity measure in object tracking outputs a similarity or distance score for given detections. As an affinity measure is typically imperfect, it generally has an uncertain region in which regarding two groups of detections as the same object or different objects based on the score can be wrong. How to reduce the uncertain region is a major challenge for most similarity-based tracking methods. Early mistakes often result in distribution drifts for tracked objects and this is another major issue for object tracking. In this paper, we propose a new offline tracking method called agglomerative hierarchical clustering with ensemble of tracking experts (AHC_ETE), to tackle the uncertain region and early mistake issues. We conduct tracking from certain to uncertain to reduce early mistakes. Meanwhile, we ensemble multiple tracking experts to reduce the uncertain region as the final uncertain region is the intersection of those of all tracking experts. Experiments on the MOT15 and MOT16 datasets demonstrated the effectiveness of our method. The code is publicly available at https://github.com/cyoukaikai/ahc_ete.

From Certain to Uncertain: Toward Optimal Solution for Offline Multiple Object Tracking

In recent years, Discriminative Correlation Filters (DCFs) have gained popularity due to their superior performance in visual object tracking. However, existing DCF trackers usually learn filters using fixed attention mechanisms that focus on the centre of an image and suppresses filter amplitudes in surroundings. In this paper, we propose an Adaptive Context-Aware Discriminative Correlation Filter (ACA-DCF) that is able to improve the existing DCF formulation with complementary attention mechanisms. Our ACA-DCF integrates foreground attention and background attention for complementary context-aware filter learning. More importantly, we ameliorate the design using an adaptive weighting strategy that takes complex appearance variations into account. The experimental results obtained on several well-known benchmarks demonstrate the effectiveness and superiority of the proposed method over the state-of-the-art approaches.

Adaptive Context-Aware Discriminative Correlation Filters for Robust Visual Object Tracking


Foreground segmentation technologies play an important role in applications such as free-viewpoint video (FVV) and sports video analysis. In this situation, we propose a new method that achieves accurate foreground silhouette extraction method using dynamic object presence probability (DOPP). Our main contributions are as follows. 1) Object presence probability for each pixel is calculated from the object recognition results based on deep learning. After that, background subtraction is implemented by changing the threshold and the update rate of the background model in response to the object presence probability. Parameter tuning of background subtraction is executed by using the object recognition results to improve the silhouette extraction quality. 2) To calculate more accurate silhouette images, parameters of background subtraction are adjusted by monitoring optical flows between consecutive frames. The object presence probability of the current frame is dynamically updated by using the object presence probability of the previous frame with optical flows. In the experiments, we confirmed that the proposed method achieved more accurate silhouette extraction than conventional methods in three sports sequences.

Accurate Background Subtraction Using Dynamic Object Presence Probability in Sports Scenes




In this work, we aim to calculate accurate indoor location with WiFi fingerprints. This task is tricky because the input are high dimensional and noisy. To deal with these issues, we propose a Variational Information Bottleneck-base model. The proposed model consists of an encoder used to find a representation of the input and a predictor used to predict the final output using the latent representation. Finally, we test the proposed model on a real-word dataset and the obtained results show the effectiveness of our method.

Variational Information Bottleneck Model for Accurate Indoor Position Recognition

Meetings are a common activity that provides
certain challenges when creating systems that assist them. Such
is the case of the Active speaker detection, which can provide
useful information for human interaction modeling, or human-robot interaction. Active speaker detection is mostly done using
speech, however, certain visual and contextual information can
provide additional insights. In this paper we propose an active
speaker detection framework that integrates audiovisual features
with social information, from the meeting context. Visual cue
is processed using a Convolutional Neural Network (CNN) that
captures the spatio-temporal relationships. We analyze several
CNN architectures with both cues: raw pixels (RGB images)
and motion (estimated with optical flow). Contextual reasoning
is done with an original methodology, based on the gaze of all
participants. We evaluate our proposal with a public benchmark
in state-of-art: AMI corpus. We show how the addition of visual
and context information improves the performance of the active
speaker detection.

Premium content

STaRFlow: A SpatioTemporal Recurrent Cell for Lightweight Multi-Frame Optical Flow Estimation

Downloads

Next from ICPR 2020

Deep Topic Modeling by Multilayer Bootstrap Network and Lasso

Similar lecture

Estimation of Clinical Tremor using Spatio-Temporal Adversarial AutoEncoder