Italy

Errors in semantic segmentation could be classified into two types: the large area misclassification and inaccurate local boundaries. Previously attention-based methods typically capture rich global contextual information, which benefits the large area classification but cannot address the local errors of boundaries. In this paper, we propose a Global-Local Attention Network (GLANet) which can simultaneously consider the global context and local details. Specifically, our GLANet consists of two branches: (1) the global attention branch and (2) local attention branch. Furthermore, three different modules are embedded in GLANet for respectively modelling the semantic interdependencies in spatial, channel and boundary dimension. Lastly, we merge the outputs of different branches to enhance the feature representation further, resulting in more precise segmentation. Overall, the proposed method achieves the competitive segmentation accuracy on two public aerial image datasets, bringing significant improvements over the existing baselines.

ICPR 2020

Global-Local Attention Network for Semantic Segmentation in Aerial Images

aerial images

self attention

semantic segmentation

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster

Feature learning in the presence of a mixed type of variables, numerical and categorical types, is important for related modeling problems. In this work, we propose a novel strategy to explicitly model the probabilistic dependence structure among the mixed type of variables by an undirected graph. The dependence structure among different pairs of variables is encoded by a suitable mapping function to estimate the edges of the graph. Spectral decomposition of the graph Laplacian provides the desired feature transformation. We numerically validate the implications of the feature learning strategy on various datasets in terms of data clustering

Graph Spectral Feature Learning for Mixed Data of Categorical and Numerical Type

We present a simple but yet effective method for learning distinctive 3D local deep descriptors (DIPs) that can be used to register point clouds without requiring an initial alignment. Point cloud patches are extracted, canonicalised with respect to their estimated local reference frame and encoded into rotation-invariant compact descriptors by a PointNet-based deep neural network. DIPs can effectively generalise across different sensor modalities because they are learnt end-to-end from locally and randomly sampled points Because DIPs encode only local geometric information, they are robust to clutter, occlusions and missing regions. We evaluate and compare DIPs against alternative hand-crafted and deep descriptors on several indoor and outdoor datasets consisting of point clouds reconstructed using different sensors. Results show that DIPs (i) achieve comparable results to the state-of-the-art on RGB-D indoor scenes (3DMatch dataset), (ii) outperform state-of-the-art by a large margin on laser-scanner outdoor scenes (ETH dataset), and (iii) generalise to indoor scenes reconstructed with the Visual-SLAM system of Android ARCore.

Distinctive 3D local deep descriptors

The unprecedented growth of Internet users has
resulted in an abundance of unstructured information on social
media including health forums, where patients request health-related information or opinions from other users. Previous studies
have shown that online peer support has limited effectiveness
without expert intervention. Therefore, a system capable of assessing the severity of the health state from the patients’ social media
posts can help health professionals (HP) in prioritizing the users
post. In this study, we inspect the efficacy of different aspects of
Natural Language Understanding (NLU) to identify the severity
of the users health state in relation to two perspectives(tasks) (a)
Medical Condition (i.e., Recover, Exist, Deteriorate, Other) and
(b) Medication (i.e., Effective, Ineffective, Serious Adverse Effect,
Other) in online health communities. We propose a multiview
learning framework that models both the textual content as well
as contextual-information to assess the severity of the users health
state. Specifically, our model utilizes the NLU views such as
sentiment, emotions, personality, and use of figurative language
to extract the contextual information. The diverse NLU views
demonstrate its effectiveness on both the tasks and as well as on
the individual disease to assess a users health.

Assessing the Severity of Health States based on Social Media Posts

Parkinson's disease (PD) is an irreversible neurodegenerative disease that seriously affects patients' lives. To provide patients with accurate treatment in time and to reduce deterioration of the disease, it is critical to have an early diagnosis of PD and accurate clinical score predictions. Different from previous studies on PD, most of which only focus on feature selection methods, we propose a network combining joint learning from multiple modalities and relations (JLMMR) with sparse nonnegative autoencoder (SNAE) to further enhance the ability of feature expression. We first preprocess and extract features of the modal neuroimaging data with multiple time points. To extract discriminative and informative features from longitudinal data, we apply the JLMMR method for feature selection to avoid over-fitting issues. We further exploit SNAE to learn longitudinal discriminative features for joint disease diagnosis and obtain clinical score predictions. Extensive experiments on the publicly available Parkinson's Progression Markers Initiative (PPMI) dataset show the proposed method produces promising classification and prediction performance, which outperforms state-of-the-art methods as well.

Longitudinal Feature Selection and Feature Learning for Parkinson’s Disease Diagnosis and Prediction

Machine learning approaches have achieved remarkable advances over the last decades, especially in supervised learning tasks such as classification. Meanwhile, multimedia data and applications experienced an explosive growth, becoming ubiquitous in diverse domains. Due to the huge increase in multimedia data collections and the lack of labeled data in several scenarios, creating methods capable of exploit the unlabeled data and operate under weakly supervision is a crucial task. In this work, we propose a rank-based model to exploit contextual information encoded in the unlabeled data in order to perform weakly supervised classification. We employ different rank-based correlation measures to identify strong similarities relationships and expand the labeled set in an unsupervised way. Subsequently, the extended labeled set is used by a classifier to achieve better accuracy results. The proposed weakly supervised approach was evaluated on multimedia classification tasks, considering several combinations of rank correlation measures and classifiers. An experimental evaluation was conducted on 4 public image datasets and different features. Very positive gains were achieved in comparison with various semi-supervised and supervised classifiers taken as baselines when considering the same amount of labeled data.

Weakly Supervised Learning through Rank-Based Contextual Measures

We train a detection, classification and score analysis DNN to improve fencing referee decisions and provide trainers statistical analysis. The model classifies ‘Lunges’, ‘Counter-Attack’, and ‘Preparation to attack’ and introduces additional classes to reduce false positives. A scoring function evaluates, in real-time, who is awarded points.

Real-Time Fencing Move Classification and Detection at Touch Time during a Fencing Match

As an essential high-level task of the video-understanding topic, automatically describing a video with natural language has recently gained attention as a fundamental challenge in computer vision. Previous models for video captioning have several limitations, such as the existence of gaps in current semantic representations and the inexpressibility of the generated captions. To deal with these limitations, in this paper, we present a new architecture that we call Attentive Visual Semantic Specialized Network (AVSSN), which is an encoder-decoder model based on our Adaptive Attention Gate and Specialized LSTM layers. This architecture can selectively decide when to use visual or semantic information in the text generation process. The adaptive gate makes the decoder automatically select the relevant information for providing a better temporal state representation than the existing decoders. Besides, the model is capable of learning to improve the expressiveness of generated captions attending to their length, using a sentence-length-related loss function. We evaluate the effectiveness of the proposed approach on the Microsoft Video Description (MSVD) and the Microsoft Research Video-to-Text (MSR-VTT) datasets, achieving state-of-the-art performance with several popular evaluation metrics: BLEU-4, METEOR, CIDEr, and ROUGE-L.

Attentive Visual Semantic


Recent research on super-resolution (SR) has witnessed major developments with the advancements of deep convolutional neural networks. There is a need for information extraction from scenic text images or even document images on
device, most of which are low-resolution (LR) images. Therefore, SR becomes an essential pre-processing step as Bicubic Upsampling, which is conventionally present in smartphones, performs poorly on LR images. To give the user more control over his privacy, and to reduce the carbon footprint by reducing the
overhead of cloud computing and hours of GPU usage, executing SR models on the edge is a necessity in the recent times. There are various challenges in running and optimizing a model on resource-constrained platforms like smartphones. In this paper, we present a novel deep neural network that reconstructs sharper character edges and thus boosts OCR confidence. The proposed architecture not only achieves significant improvement in PSNR over bicubic upsampling on various benchmark datasets but also runs with an average inference time of 11.7 ms per image. We have outperformed state-of-the-art on the Text330 dataset. We also achieve an OCR accuracy of 75.89% on the ICDAR 2015 TextSR dataset, where ground truth has an accuracy of 78.10%.

On-Device Text Image Super Resolution

Recent advances in automated image analysis have
lead to an increased number of proposed datasets in remote
sensing applications. This permits the successful employment
of data hungry state-of-the-art deep neural networks. However,
the Earth is not covered equally by semantically meaningful
classes. Thus, many land cover datasets suffer from a severe
class imbalance. We show that by taking appropriate measures,
the performance in the minority classes can be improved by
up to 30 percent without affecting the performance in the
majority classes strongly. Additionally, we investigate the use
of an attribute encoding scheme to represent the inherent class
hierarchies commonly observed in land cover analysis.

Towards Tackling Multi-Label Imbalances in Remote Sensing Imagery


Crowd counting in still images is a challenging
problem in practice due to huge crowd-density variations, large
perspective changes, severe occlusion, and variable lighting conditions. The state-of-the-art patch rescaling module (PRM) based
approaches prove to be very effective in improving the crowd
counting performance. However, the PRM module requires an
additional and compromising crowd-density classification process. To address these issues and challenges, the paper proposes
a new multi-resolution fusion based end-to-end crowd counting
network. It employs three deep-layers based columns/branches,
each catering the respective crowd-density scale. These columns
regularly fuse (share) the information with each other. The
network is divided into three phases with each phase containing
one or more columns. Three input priors are introduced to serve
as an efficient and effective alternative to the PRM module,
without requiring any additional classification operations. Along
with the final crowd count regression head, the network also
contains three auxiliary crowd estimation regression heads, which
are strategically placed at each phase end to boost the overall
performance. Comprehensive experiments on three benchmark
datasets demonstrate that the proposed approach outperforms all
the state-of-the-art models under the RMSE evaluation metric.
The proposed approach also has better generalization capability
with the best results during the cross-dataset experiments.


Premium content

Global-Local Attention Network for Semantic Segmentation in Aerial Images

Downloads

Next from ICPR 2020

Graph Spectral Feature Learning for Mixed Data of Categorical and Numerical Type

Similar lecture

Fast and Accurate Real-Time Semantic Segmentation with Dilated Asymmetric Convolutions