Italy

Weakly Supervised Object Localization (WSOL) task aims to recognize the object position by using only image-level labels. Some previous techniques remove the most discriminative parts for all input images or random images to capture the entire object location. However, these methods can not perform the correct operation on different images such as hiding the data or feature maps that should not be hidden. In this case, both classification and localization accuracy will be affected. Meanwhile, just erasing the most important regions tends to make the model learn the less discriminative parts from outside of the objects. To address these limitations, we propose an Attention-based Selection Strategy (ASS) method to choose images that do need to be erased. Moreover, we use different threshold self-attention maps to reduce the impact of unhelpful information in one of the branches of our selection strategy. Based on our experiments, the proposed method is simple but effective to improve the performance of WSOL. In particular, ASS achieves new state-of-the-art accuracy on CUB-200-2011 dataset and works very well on ILSVRC 2016 dataset.

ICPR 2020

Attention-based Selection Strategy for Weakly Supervised Object Localization

attention-based selection strategy

weakly supervised object localization

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster


Tracking Fast Moving Objects (FMO), which appear
as blurred streaks in video sequences, is a difficult task for
standard trackers, as the object position does not overlap in
consecutive video frames and texture information of the objects
is blurred. Up-to-date approaches tuned for this task are based
on background subtraction with a static background and slow
deblurring algorithms. In this article, we present a tracking-by-segmentation
approach implemented using modern deep learning
methods that perform near real-time tracking on real-world
video sequences. We have developed a physically plausible FMO
sequence generator to be a robust foundation for our training
pipeline and demonstrate straightforward network adaptation for
different FMO scenarios with varying foreground.

Tracking Fast Moving Objects by SegmentationNetwork

Tone-mapping plays an essential role in high dynamic range (HDR) imaging. It aims to preserve visual information of HDR images in a medium with a limited dynamic
range. Although many works have been proposed to provide
tone-mapped results from HDR images, most of them can only
perform tone-mapping in a single pre-designed way. However,
the subjectivity of tone-mapping quality varies from person to
person, and the preference of tone-mapping style also differs
from application to application. In this paper, a learning-based
multimodal tone-mapping method is proposed, which not only
achieves excellent visual quality but also explores the style diversity. Based on the framework of BicycleGAN, the proposed
method can provide a variety of expert-level tone-mapped results
by manipulating different latent codes. Finally, we show that
the proposed method performs favorably against state-of-the-art
tone-mapping algorithms both quantitatively and qualitatively.

Explorable Tone Mapping Operators

Visual privacy concerns associated with image sharing is a critical issue that needs to be addressed to enable safe and lawful use of online social platforms. Users of social media platforms often suffer from no guidance in sharing sensitive images in public and often face social and legal consequences. Given the recent success of visual attention based deep learning methods in measuring abstract phenomena like image memorability, we are motivated to investigate whether visual attention based methods could be useful in measuring psychophysical phenomena like "privacy sensitivity". In this paper, we propose PrivAttNet - a visual attention based approach, that can be trained end-to-end to estimate the privacy sensitivity of images without explicitly detecting sensitive objects and attributes present in the image. We show that our model outperforms various SOTA and baseline strategies - a 1.6 fold reduction in L1-error over SOTA and 7% - 10% improvement in Spearman-rank correlation between the predicted and ground truth sensitivity scores. Additionally, the attention maps from PrivAttNet are found to be useful in directing the users to the regions that are responsible for generating the privacy risk score.

PrivAttNet: Predicting Privacy Risks in Images Using Visual Attention

Most recent successes on forecasting the people motion are based on LSTM models and all most recent progress has been achieved by modelling the social interaction among people and the people interaction with the scene. We question the use of the LSTM models and propose the novel use of Transformer Networks for trajectory forecasting. This is a fundamental switch from the sequential step-by-step processing of LSTMs to the only-attention-based memory mechanisms of Transformers. In particular, we consider both the original Transformer Network (TF) and the larger Bidirectional Transformer (BERT), state-of-the-art on all natural language processing tasks. Our proposed Transformers predict the trajectories of the individual people in the scene. These are simple models because each person is modelled separately without any complex human-human nor scene interaction terms. In particular, the TF model without bells and whistles yields the best score on the largest and most challenging trajectory forecasting benchmark of TrajNet. Additionally, its extension which predicts multiple plausible future trajectories performs on par with more engineered techniques on the 5 datasets of ETH+ UCY. Finally, we show that Transformers may deal with missing observations, as it may be the case with real sensor data.

Transformer Networks for Trajectory Forecasting

Increased awareness of the impact of food consumption on health and lifestyle today has given rise to novel data-driven food analysis systems. Although these systems may recognize the ingredients, a detailed analysis of their amounts in the meal, which is paramount for estimating the correct nutrition, is usually ignored. We study the novel and challenging problem of predicting the relative amount of each ingredient from a food image. We propose PITA, the Picture-to-Amount deep learning architecture to solve the problem. More specifically, we predict the ingredient amounts using a domain-driven Wasserstein loss from image-to-recipe cross-modal embeddings learned to align the two views of food data. Experiments on a dataset of recipes collected from the Internet show the model generates promising results and improves the baselines on this challenging task. A demo of our system and our data is availableat: foodai.cs.rutgers.edu

Picture-to-Amount (PITA): Predicting Relative Ingredient Amounts from Food Images

In this paper, we propose a new approach to understand actions in egocentric videos that exploits the semantics of object interactions at both frame and temporal levels. At the frame level, we use a region-based approach that takes as input a primary region roughly corresponding to the user hands and a set of secondary regions potentially corresponding to the interacting objects and calculates the action score through a CNN formulation. This information is then fed to a Hierarchical Long Short-Term Memory Network (HLSTM) that captures temporal dependencies between actions within and across shots. Ablation studies thoroughly validate the proposed approach, showing in particular that both levels of the HLSTM architecture contribute to performance improvement. Furthermore, quantitative comparisons show that the proposed approach outperforms the state-of-the-art in terms of action recognition on standard benchmarks, without relying on motion information.

Modeling Long-Term Interactions to Enhance Action Recognition

This paper considers three different classifier stacking algorithms: simple stacking, cascades of classifier ensembles and nonlinear version of classifier stacking based on classifier interactions. Classifier interactions can be expressed using classifier prediction pairwise matrix (CPPM). As a meta-learner for the last algorithm Convolutional Neural Networks (CNNs) and two other classifier stacking algorithms (simple classifier stacking and cascades of classifier ensembles) have been applied. This allows applying classical stacking and cascade-based recursive stacking in the Euclidean and the Riemannian geometries. The cascades of random forests (RFs) and extra trees (ETs) are considered as a forest-based alternative to deep neural networks \cite{zhou2017deep}. Our goal is to compare accuracies of the cascades of RFs and CNN-based stacking or deep multi-layer perceptrons (MLPs) for different classifications problems. We use gesture phase dataset from UCI repository \cite{frank2010uci} to compare and analyze cascades of RFs and extra trees (ETs) in both geometries and CNN-based version of classifier stacking. This data set was selected because generally motion is considered as a nonlinear process (patterns do no lie in Euclidean vector space) in computer vision applications. Thus we can assess how good are forest-based deep learning and the Riemannian manifolds (R-manifolds) when applied to nonlinear processes. Some more datasets from UCI repository were used to compare the aforementioned algorithms to some other well-known classifiers and their stacking-based versions in both geometries. Experimental results show that classifier stacking algorithms in Riemannian geometry (R-geometry) are less dependent on some properties of individual classifiers (e.g. depth of decision trees in RFs or ETs) in comparison to Euclidean geometry. More independent individual classifiers allow to obtain R-manifolds with better properties for classification. Generally, accuracy of classification using classifier stacking in R-geometry is higher than in Euclidean one.

Comparison of Stacking-based Classifier Ensembles using Euclidean and Riemannian Geometries

Large-scale highway traffic forecasting approaches are critical for intelligent transportation systems. Recently, deep-learning-based traffic forecasting methods have emerged as promising approaches for a wide range of traffic forecasting tasks. These methods are specific to a given traffic network, however, and consequently, they cannot be used for forecasting traffic on an unseen traffic network.
Previous work has identified diffusion convolutional recurrent neural networks, (DCRNN), as a state-of-the-art method for highway traffic forecasting. 
It models the complex spatial and temporal dynamics of a highway network using a graph-based diffusion convolution operation within a recurrent neural network. 
Currently, DCRNN cannot perform transfer learning because it learns location-specific traffic patterns, which cannot be used for unseen regions of a network or new geographic locations. To that end, we develop TL-DCRNN, a new transfer learning approach for DCRNN, where a single model trained on a highway network can be used to forecast traffic on unseen highway networks. Given a traffic network with a large amount of traffic data, our approach consists of partitioning the traffic network into a number of subgraphs and using a new training scheme that utilizes subgraphs to marginalize the location-specific information, thus learning the traffic as a function of network connectivity and temporal patterns alone. The resulting trained model can be used to forecast traffic on unseen networks. We demonstrate that TL-DCRNN can learn from San Francisco regional traffic data and can forecast traffic in the Los Angeles region and vice versa.


Transfer Learning with Graph Neural Networks for Short-Term Highway Traffic Forecasting


Purpose: Liver cancer is a major cause of morbidity and mortality in the world. The primary goals of this manuscript are the identification of novel imaging features (texture, functional, and shape), and development of a computer-aided diagnostic (CAD) system to accurately detect and grade liver tumors non-invasively. 
Methods: A total of 85 patients with liver tumors were enrolled in the study after consents were obtained. 34 patients had benign tumors, 17 patients had intermediate tumors, and 34 patients had hepatocellular carcinoma (HCC) malignant tumors. A multi-phase contrast-enhanced magnetic resonance imaging (CE-MRI) was collected to extract the imaging features. The proposed approach consists of three main steps. First, a pre-processing is applied to the CE-MRI scans to delineate the tumor lesions that will be used as a region of interest (ROI) across the four different phases of the CE-MRI. Second, a group of three features are modeled to provide a quantitative discrimination between the tumor lesions, namely: the tumor appearance that is modeled using a set of texture features, (namely; the first-order histogram features, second-order gray-level co-occurrence matrix (GLCM) features, and second-order gray-level run-length matrix (GLRLM) features), to capture any discrimination that may appear in the lesion texture; the spherical harmonics (SH) based shape features that have the ability to describe the shape complexity of the liver tumors; and the functional features that are based on the calculation of the wash-in/wash-out slopes to evaluate the intensity changes across different phases. Finally, these features were integrated together to obtain the combined features to be fed to a machine learning classifier towards getting the final diagnostic decision. 
Results: Using the Random Forests classifier with a leave-one-out (LOSO) cross-validation, the developed CAD system achieved an 87.1% accuracy in distinguishing between malignant, intermediate and benign tumors (i.e., First stage classification). LR-1 lesions were classified from LR-2 benign lesions with 91.2% accuracy, while 85.3% accuracy was achieved differentiating between LR-4 and LR-5 malignant tumors (i.e., Second stage classification).

A Novel Computer-Aided Diagnostic System for Early Assessment of Hepatocellular Carcinoma

This work makes a contribution to the problem of incremental class learning, the principle of which is to sequentially introduce batches of samples annotated with new classes during the learning phase. The main objective is to reduce the drop in classification performance on old classes, a phenomenon commonly called catastrophic forgetting. We propose a new method which exploits the availability of a large quantity of non-annotated images in addition to the annotated batches. These images are used to regularize the classifier and give the feature space a more stable structure. We demonstrate on two image data sets, MNIST and STL-10, that our approach is able to improve the global performance of classifiers learned using an incremental learning protocol, even with annotated batches of small size.

Premium content

Attention-based Selection Strategy for Weakly Supervised Object Localization

Downloads

Next from ICPR 2020

Tracking Fast Moving Objects by SegmentationNetwork

Similar lecture

Dual-attention Guided Dropblock Module for Weakly Supervised Object Localization