Italy


Audio-visual co-segmentation is a task to extract segments and regions corresponding to specific events on unlabeled audio and video signals. It is particularly important to accomplish it in an unsupervised way, since it is generally very difficult to manually label all the objects and events appearing in audio-visual signals for supervised learning. Here, we propose to take advantage of the temporal proximity of corresponding audio and video entities included in the signals. For this purpose, we newly employ a guided attention scheme to this task to efficiently detect and utilize temporal co-occurrences of audio and video information. Experiments using a real TV broadcasts of sumo wrestling, a sport event, with live commentaries show that our model can automatically extract specific athlete movements and its spoken descriptions in an unsupervised manner.

ICPR 2020

Unsupervised Co-Segmentation for Athlete Movements and Live Commentaries Using Crossmodal Temporal Proximity

guided attention

audio-visual co-segmentation

self-supervised learning

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster


This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

Kernel-based LIME with Feature Dependency Sampling

Activation functions and attention mechanisms are typically treated as having different purposes and have evolved differently.
However, both concepts can be formulated as a non-linear gating function. 
Inspired by their similarity, we propose a novel type of activation units called attentional activation (ATAC) units as a unification of activation functions and attention mechanisms.
In particular, we propose a local channel attention module for the simultaneous non-linear activation and element-wise feature refinement, which locally aggregates point-wise cross-channel feature contexts.
By replacing the well-known rectified linear units by 
such ATAC units in convolutional networks, we can construct fully attentional networks that perform significantly better with a modest number of additional parameters.
We conducted detailed ablation studies on the ATAC units using several host networks with varying network depths to empirically verify the effectiveness and efficiency of the units.
Furthermore, we compared the performance of the ATAC units against existing activation functions as well as other attention mechanisms on the CIFAR-10, CIFAR-100, and ImageNet datasets.
Our experimental results show that networks constructed with the proposed ATAC units generally yield performance gains over their competitors given a comparable number of parameters.

Attention as Activation

Evaluation of generative AI is a difficult problem, especially in artistic domains in which aesthetic qualities of generated samples are to an extent subjective, such as in music. The most widely accepted method for evaluating such models is to conduct a survey of users, which is a resource intensive process. In this work we propose a framework for cheaply evaluating generative models in the symbolic music domain by utilizing tools from music theory, such as the circle of fifths, with the goal of producing quantifiable metrics which reflect the "musicality" of a written score or MIDI file.

Heuristics for Evaluation of AI Generated Music

Despite the success of deep neural networks (DNN) in many applications, their ability to model uncertainty is still significantly limited. For example, in safety-critical applications such as autonomous driving, it is crucial to obtain a prediction that reflects different types of uncertainty to address life-threatening situations appropriately. In such cases, it is essential to be aware of the risk (i.e.,~aleatoric uncertainty) and the reliability (i.e.,~epistemic uncertainty) that comes with a prediction. We present AE-DNN, a model allowing the separation of aleatoric and epistemic uncertainty while maintaining a proper generalization capability. AE-DNN is based on deterministic DNN, which can determine the respective uncertainty measures in a single forward pass. In analyses with synthetic and image data, we show that our method improves the modeling of epistemic uncertainty while providing an intuitively understandable separation of risk and reliability.

Separation of Aleatoric and Epistemic Uncertainty in Deterministic Deep Neural Networks

We use feature attention and transformation to address the nighttime pedestrian detection issue.

Nighttime Pedestrian Detection Based on Feature Attention and Transformation


We present DeepScoresV2, an extended version of the DeepScores dataset for optical music recognition (OMR). We improve upon the original DeepScores dataset by providing much more detailed annotations, namely (a) annotations for135classes including fundamental symbols of non-fixed size and shape, increasing the number of annotated symbols by 23%; (b) oriented bounding boxes; (c) higher-level rhythm and pitch information (onset beat for all symbols and line position for noteheads); and (d) a compatibility mode for easy use in conjunction with the MUSCIMA++ dataset for OMR on handwritten documents. These additions open up the potential for future advancement in OMR research. Additionally, we release two state-of-the-art baselines for DeepScoresV2based on Faster R-CNN and the Deep Watershed Detector. An analysis of the baselines shows that regular orthogonal bounding boxes are unsuitable for objects which are long, small, and potentially rotated, such as ties and beams, which demonstrates the need for detection algorithms that naturally incorporate objects angles.

The DeepScoresV2 Dataset and Benchmark for Music Object Detection

We investigate whether it is possible to classify the gender of a standing person based on a video sequence containing body sway recorded by an overhead camera. Existing methods that extract a feature from the movement of a walking person for gender classification cannot detect the slight movements of a standing person. In this paper, we propose a method for extracting a feature from the body sway of a standing person. We design a spatio-temporal feature for representing body sway using the frequency analysis of time-series signals derived from the local movements of the upper body. To evaluate the accuracy of our method, we acquired video sequences of body sway from 30 females and 30 males using an overhead camera. We found that our method obtained 90.3% accuracy for the gender classification of a standing person. We compared the accuracy of our method with that of parameters based on medical data. We found that the proposed spatio-temporal feature extracted from body sway significantly improved gender classification accuracy.

Gender Classification Using Video Sequences of Body Sway Recorded by Overhead Camera

Ureteroscopy is becoming the first surgical treatment option for the majority of urinary affections. 
This procedure is performed using an endoscope which provides the surgeon with the visual information necessary to navigate inside the urinary tract. 
Having in mind the development of surgical assistance systems, that could enhance the performance of surgeon, the task of lumen segmentation is a fundamental part since this is the visual reference which marks the path that the endoscope should follow. 
This is something that has not been analyzed in ureteroscopy data before. 
However, this task presents several challenges given the image quality and the conditions itself of ureteroscopy procedures. 
In this paper, we study the implementation of a Deep Neural Network which exploits the advantage of residual units in an architecture based on U-Net. 
For the training of these networks, we analyze the use of two different color spaces: gray-scale and RGB data images. 
We found that training on gray-scale images gives the best results obtaining mean values of Dice Score, Precision, and Recall of 0.73, 0.58, and 0.92 respectively. 
The results obtained shows that the use of residual U-Net could be a suitable model for further development for a computer-aided system for navigation and guidance through the urinary system.

A Lumen Segmentation Method in Ureteroscopy Images based on a Deep Residual U-Net architecture


The vascular structure of blood vessels is important in diagnosing retinal conditions such as glaucoma and diabetic retinopathy. Accurate segmentation of these vessels can help in detecting retinal objects such as the optic disc and optic cup and hence determine if there are damages to these areas. Moreover, the structure of the vessels can help in diagnosing glaucoma. The rapid development of digital imaging and computer-vision techniques has increased the potential for developing approaches for segmenting retinal vessels. In this paper, we propose an approach for segmenting retinal vessels that uses deep learning along with transfer learning. We adapted the U-Net structure to use a customized InceptionV3 as the encoder and used multiple skip connections to form the decoder. Moreover, we used a weighted loss function to handle the issue of class imbalance in retinal images. Furthermore, we contributed a new dataset to this field. We tested our approach on six publicly available datasets and a newly created dataset. We achieved an average accuracy of 95.60% and a Dice coefficient of 80.98%. The results obtained from comprehensive experiments demonstrate the robustness of our approach to the segmentation of blood vessels in retinal images obtained from different sources. Our approach results in greater segmentation accuracy than other approaches.

Transfer Learning Through Weighted Loss Function and Group Normalization for Vessel Segmentation from Retinal Images

Data is one of the most important keys for success when studying a simple or a complex phenomenon. With the use of deep learning exploding and its democratization, non-computer science experts may struggle to use highly complex deep learning architectures, even when straightforward models offer them suitable performances. In this article, we study the specific and common problem of data imbalance in real databases as most of the bad performance problems are due to the data itself. We cover two keys aspects. First, we propose ways to deal with the situation when a data set contains different levels of imbalance. Classical imbalanced learning strategies cannot be directly applied when using multi-attribute deep learning models, i.e., multi-task or multi-label architectures. Therefore, one of our contributions is a proposed adaptation to face each one of the problems derived from imbalance. Second, we demonstrate that with little to no imbalance, straightforward deep learning models work well. However, for non-experts, these models can be seen as black boxes, where all efforts are invested in pre-processing the data. To simplify the problem, we perform the classification task without features that are costly to extract, such as part localization which is widely used in the state of the art of attribute classification. We make use of three widely known attribute databases, CUB-200-2011 - CUB as our main use case due to its deeply imbalanced nature, along with two balanced databases: celebA and AwA2. All of them contain multi-attribute annotations. The results of very fine-grained attribute learning demonstrate that in the presence of imbalance, our proposed strategies make it possible to have competitive results against the state of the art while taking advantage of multi-attribute deep learning models. We also noticed an increase in performance while using a specialized loss function (Focal Loss). For CUB, we have competitive results, and for CelebA and AwA2 our strategies over-perform the state of the art.

Premium content

Unsupervised Co-Segmentation for Athlete Movements and Live Commentaries Using Crossmodal Temporal Proximity

Downloads

Next from ICPR 2020

Kernel-based LIME with Feature Dependency Sampling

Similar lecture

Self-supervised Learning for Astronomical Image Classification