Italy

Beyond assigning the correct class, an activity recognition model should also be able to determine, how certain it is in its predictions. We present the first study of how well the confidence values of modern action recognition architectures indeed reflect the probability of the correct outcome and propose a learning-based approach for improving it. First, we extend two popular action recognition datasets with a reliability benchmark in form of the expected calibration error and reliability diagrams. Since our evaluation highlights that confidence values of standard action recognition architectures do not represent the uncertainty well, we introduce a new approach which learns to transform the model output into realistic confidence estimates through an additional calibration network. The main idea of our Calibrated Action Recognition with Input Guidance (CARING) model is to learn an optimal scaling parameter depending on the video representation. We compare our model with the native action recognition networks and the temperature scaling approach - a wide spread calibration method utilized in image classification. While temperature scaling alone drastically improves the reliability of the confidence values, our CARING method consistently leads to the best uncertainty estimates in all benchmark settings.

ICPR 2020

Uncertainty-sensitive Activity Recognition: a Reliability Benchmark and the CARING Models

model calibration

model confidence

activity recognition

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


technical paper


Spatio-temporal feature descriptors are of great
importance for characterizing the local changes of 3D deformable
shapes. In this study, we propose a method for characterizing
3D shapes from point clouds and we show a direct application
on a study of organ temporal deformations. As an example,
we characterize the behavior of the bladder during forced
respiratory motion with a reduced number of 3D surface points:
first, a set of geodesically equidistant points representing the
vertices of quadrilateral mesh for the organ surface are tracked
throughout a long dynamic MRI sequence using a large deformation diffeomorphic metric mapping (LDDMM) framework.
Second, a novel 3D shape descriptor invariant to translation,
scale and rotation is proposed for characterizing the temporal
organ deformations by employing an Eulerian Partial Differential
Equations (PDEs) methodology. We demonstrate the robustness
of our feature on both synthetic 3D shapes and realistic dynamic
Magnetic Resonance Imaging (MRI) data sequences portraying
the bladder deformation during a forced breathing exercise.
Promising results are obtained, showing that the proposed feature
may be useful for several computer vision applications such as
medical imaging, aerodynamics and robotics.

A new geodesic-based feature for characterization of 3D shapes

Supervised deep learning methods have recently
achieved remarkable performance in person re-identification.
Unsupervised domain adaptation (UDA) approaches have also
been proposed for application scenarios where only unlabelled
data are available from target camera views. We consider a
more challenging scenario when even collecting a suitable amount
of representative, unlabelled target data for offline training or
fine-tuning is infeasible. In this context we revisit the human-in-
the-loop (HITL) approach, which exploits online the operator’s
feedback on a small amount of target data. We argue that HITL
is a kind of online domain adaptation specifically suited to person
re-identification. We then reconsider relevance feedback methods
for content-based image retrieval that are computationally much
cheaper than state-of-the-art HITL methods for person re-
identification, and devise a specific feedback protocol for them.
Experimental results show that HITL can achieve comparable or
better performance than UDA, and is therefore a valid alternative
when the lack of unlabelled target data makes UDA infeasible.

Online Domain Adaptation for Person Re-Identification with a Human in the Loop


Fast image retrieval is of great importance in many computer vision tasks and especially practical applications. Deep hashing, the state-of-the-art fast image retrieval scheme, introduces deep learning to learn the hash functions and generate binary hash codes, and outperforms the other image retrieval methods in terms of accuracy. However, all the existing deep hashing methods could only generate one level hash codes and require a linear traversal of all the hash codes to figure out the closest one when a new query arrives, which is very time-consuming and even intractable for large scale applications. In this work, we propose a Hierarchical Deep Hashing(HDHash) scheme to speed up the state-of-the-art deep hashing methods. More specifically, hierarchical deep hash codes of multiple levels can be generated and indexed with tree structures rather than linear ones, and pruning irrelevant branches can sharply decrease the retrieval time. To our best knowledge, this is the first work to introduce hierarchical indexed deep hashing for fast large scale image retrieval. Extensive experimental results on three benchmark datasets demonstrate that the proposed HDHash scheme achieves better or comparable accuracy with significantly improved efficiency and reduced memory as compared to state-of- the-art fast image retrieval schemes.

Hierarchical Deep Hashing for Fast Large Scale Image Retrieval


Detecting and segmenting individual cells from microscopy images is critical to various life science applications. Traditional cell segmentation tools are often ill-suited for applications in brightfield microscopy due to poor contrast and intensity heterogeneity, and only a small subset are applicable to segment cells in a cluster. In this regard, we introduce a novel supervised technique for cell segmentation in a multi- task learning paradigm. A combination of a multi-task loss, based on the region and cell boundary detection, is employed for an improved prediction efficiency of the network. The learning problem is posed in a novel min-max framework which enables adaptive estimation of the hyper-parameters in an automatic fashion. The region and cell boundary predictions are combined via morphological operations and active contour model to segment individual cells. The proposed methodology is particularly suited to segment touching cells from brightfield microscopy images without manual interventions. Quantitatively, we observe an overall Dice score of 0.93 on the validation set, which is an improvement of over 15.9% on a recent unsupervised method, and outperforms the popular supervised U-net algorithm by at least 5.8% on average

Learning to Segment Clustered Amoeboid Cells from Brightfield Microscopy Via Multi-Task Learning with Adaptive Weight Selection


Handwritten signature has long been a widely accepted
biometric and applied in many verification scenarios.
However, automatic signature verification remains an open research
problem, which is mainly due to three reasons. 1) Skilled
forgeries generated by persons who imitate the original writting
pattern are very difficult to be distinguished from genuine
signatures. It is especially so in the case of offline signatures,
where only the signature image is captured as a feature for
verification. 2) Most state-of-the-art models are writer-dependent,
requiring a specific model to be trained whenever a new user is
registered in verification, which is quite inconvenient. 3) Writerindependent
models often have unsatisfactory performance. To
this end, we propose a novel metric learning based method
for offline writer-independent signature verification. Specifically,
a dual triplet loss is used to train the model, where two
different triplets are constructed for random and skilled forgeries,
respectively. Experiments on three alphabet datasets — GPDS
Synthetic, MCYT and CEDAR—show that the proposed method
achieves competitive or superior performance to the state-of-theart
methods. Experiments are also conducted on a new offline
Chinese signature dataset — CSIG-WHU, and the results show
that the proposed method has a high feasibility on characterbased
signatures.

Learning Metric Features for Writer-Independent Signature Verification using Dual Triplet Loss


Image stitching describes the process of reconstruction of a high-resolution image by combining multiple images. Using a scanning electron microscope as the image source, individual images will show patterns in a nm dimension, whereas the combined image may cover an area of several mm$^2$. The recovery of the physical layout of modern semiconductor products manufactured in advanced technologies nodes down to 22~nm requires a perfect stitching process with no deviation with respect to the original design data, as any stitching error will result in failures during the reconstruction of the electrical design. In addition, the recovery of the complete design requires the acquisition of all individually layers of a semiconductor device. The layers represent a 3D structure with interconnections defining error limits on the stitching error for each individual scanned image mosaic. An advanced stitching and alignment process is presented, enabling a true geometrical layout recovery in nanoscale dimensions, which is also applied to and evaluated for other use cases from biological applications.

Recovery of 2D and 3D layout information through an advanced image stitching algorithm using scanning electron microscope images

In recent years, Artificial Neural Networks (ANNs) pruning has become the focal point of many researches, due to the extreme overparametrization of such models. This has urged the scientific world to investigate methods for the simplification of the structure of weights in ANNs, mainly in an effort to reduce time for both training and inference. Frankle and Carbin, and later Renda, Frankle, and Carbin introduced and refined an iterative pruning method which is able to effectively prune the network of a great portion of its parameters with little to no loss in performance. On the downside, this method requires a large amount of time for its application, since, for each iteration, the network has to be trained for (almost) the same amount of epochs of the unpruned network. In this work, we show that, for a limited setting, if targeting high overall sparsity rates, this time can be effectively reduced for each iteration, save for the last one, by more than 50%, while yielding a final product (i.e., final pruned network) whose performance is comparable to the ANN obtained using the existing method.

Speeding-up pruning for Artificial Neural Networks:
Introducing Accelerated Iterative Magnitude Pruning

Scene flow estimation based on stereo sequences is a comprehensive task relevant to disparity and optical flow. Some existing methods are time-consuming and often fail in the presence of reflective surfaces. In this paper, we propose a two-stage adaptive object scene flow estimation method using a hybrid CNN-CRF model (ACOSF), which benefits from high-quality features and the structured modelling capability. Meanwhile, in order to balance the computational efficiency and accuracy, we employ adaptive iteration for energy function optimization, which is flexible and efficient for various scenes. Besides, we utilize high-quality pixel selection to reduce the computation time with only a slight decrease in accuracy. Our method achieves competitive results with the state-of-the-art, which ranks second on the challenging KITTI 2015 scene flow benchmark.

Two-Stage Adaptive Object Scene Flow Using Hybrid CNN-CRF Model


Most of the existing methods to localize (sub) image relations – a subclass of near-duplicate retrieval techniques – rely on the distinctiveness of matched features of the images being compared. These sets of matching features usually include a proportion of outliers, i.e. features linking non-matching regions. In approaches that are designed for retrieval purposes only, these false matches usually have a minor impact on the final ranking. However, if also a localization of regions and corresponding image transformations should be computed, these false matches often have a more significant impact. We present a novel outlier filtering approach "ECOTA" to classify outliers and describe the correlation between image regions without any prior knowledge about the content of images. The comparison with the state of the art approaches indicates that our approach is more robust than those approaches and is able to detect correlation even when most matches are outliers. Moreover, ECOTA reduces significantly the pre-processing time to filter the matches.

Localization and Transformation Reconstruction of Image Regions: An Extended Congruent Triangles Approach

Any high-dimensional data arising from practical applications usually contains irrelevant features that may impact on the performance of existing subspace
clustering methods. This paper proposes a novel subspace clustering method which reconstructs the feature matrix by the means of unsupervised feature selection (UFS) to achieve a better dictionary for subspace clustering (SC). Different from most existing clustering methods, the proposed approach uses the reconstructed feature matrix as the dictionary rather than the original data matrix. As the feature matrix reconstructed by representative features is more discriminative and closer to the ground-truth, it results in improved performance. The corresponding non-convex optimization problem is effectively solved using the half-quadratic and augmented Lagrange multiplier methods. Extensive experiments on four real datasets demonstrate the effectiveness of the proposed method.

Premium content

Uncertainty-sensitive Activity Recognition: a Reliability Benchmark and the CARING Models

Downloads

Next from ICPR 2020

A new geodesic-based feature for characterization of 3D shapes

Similar lecture

Uncertainty-sensitive Activity Recognition: a Reliability Benchmark and the CARING Models