Italy

Current research on embodied AI mainly focuses on stand-alone tasks. Instead, we aim at bridging recent findings on embodied exploration and image captioning.
We devise a novel setting involving the exploration of a photorealistic indoor environment while describing the current RGB observation, and we call it Explore and Explain.
We present a novel solution involving curiosity-driven exploration and we apply a captioning algorithm to describe relevant objects along the trajectory of the agent.

ICPR 2020

Explore and Explain: Self-Supervised Navigation and Recounting

navigation

curiosity

captioning

reinforcement

exploration

self-supervision

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


technical paper

Convolutional Neural Networks (CNNs) have recently been introduced for addressing copy-move forgery detection (CMFD). However, current CMFD CNN-based approaches have insufficient performance commitment regarding the 
localization of the positive class. In this paper, this issue is explored by considering both linear and nonlinear interactions between pixels. A nonlinear Inception module based on second-order Volterra kernels is proposed, in order to ameliorate the results of a state-of-the-art CMFD architecture. The outcome of this work shows that a combination of linear and nonlinear convolution kernels can make the input foreground and background pixels more separable. The proposed approach is evaluated on CASIA and CoMoFoD, two publicly available CMFD datasets, and
results to an improved positive class localization performance. Moreover, the findings of the proposed method imply that the nonlinear Inception module stimulates immense robustness against miscellaneous post processing attacks.

RobusterNet: Improving Copy-Move Forgery Detection with Volterra-based Convolutions

This presentation briefly introduces our motivation and method of utilizing temporal features in dynamic point cloud segmentation. It also presents the experiment results, including comparison to other methods and ablation study on the effect of temporal features.

Human Segmentation with Dynamic LiDAR Data

Visual Question Answering (VQA) is challenging due to the complex cross-modal relations. It has received extensive attention from the research community. From the human perspective, to answer a visual question, one needs to read the question and then refer to the image to generate an answer. This answer will then be checked against the question and image again for the final confirmation. In this paper, we mimic this process and propose a fully attention based VQA architecture. Moreover, an answer-checking module is proposed to perform a unified attention on the jointly answer, question and image representation to update the answer. This mimics the human answer checking process to consider the answer in the context. With answer-checking modules and transferred BERT layers, our model achieves the state-of-the-art accuracy 71.57% using fewer parameters on VQA-v2.0 test-standard split.

Answer-checking in Context: A Multi-modal Fully Attention Network for Visual Question Answering

The ability to handle large scale variations is crucial for many real world visual tasks. A straightforward approach for handling scale in a deep network is to process an image at several scales simultaneously in a set of scale channels. Scale invariance can then, in principle, be achieved by using weight sharing between the scale channels together with max or average pooling over the outputs from the scale channels. The ability of such scale channel networks to generalise to scales not present in the training set over significant scale ranges has, however, not previously been explored. We, therefore, present a theoretical analysis of invariance and covariance properties of scale channel networks and perform an experimental evaluation of the ability of different types of scale channel networks to generalise to previously unseen scales. We identify limitations of previous approaches and propose a new type of foveated scale channel architecture, where the scale channels process increasingly larger parts of the image with decreasing resolution. Our proposed FovMax and FovAvg networks perform almost identically over a scale range of 8, also when training on single scale training data, and do also give improvements in the small sample regime.

Exploring the ability of CNNs to generalise to previously unseen scales over wide scale ranges

Channel attention has shown its great performance recently when it was incorporated into deep convolutional neural networks. However, existing methods usually require extensive computing resources due to their involuted structure, which further increase the computational burden of 3D CNNs. In this paper, a lightweight sparse channel attention (SCA) module implemented by efficient group convolution is proposed, which adopts the idea of sparse channel connection and involves much fewer parameters but brings clear performance gain. Meanwhile, to solve the lack of local channel interaction brought by group convolution, a dominant function called Aggregate-Shuffle-Diverge (ASD) is leveraged to enhance information flow over each group with no additional parameters. We also adjust the existing mainstream 3D CNNs by employing 3D convolution factorization, so as to further reduce the parameters. Our SCA module can be flexibly incorporated into most existing 3D CNNs, all of which can achieve a perfect trade-off between performance and complexity on action recognition task with factorized I3D or 3D ResNext backbone networks. The experimental results also indicate that the resulting network, namely, SCA Net can achieve outstanding performance on UCF-101 and HMDB-51 datasets.

SCA Net: Sparse Channel Attention Module for Action Recognition


This paper studies the problem of Person Re-Identification (ReID)
for large-scale applications. Recent research efforts have been devoted
to building complicated part models, which introduce considerably
high computational cost and memory consumption, inhibiting its practicability
in large-scale applications. This paper aims to develop a novel learning
strategy to find efficient feature embeddings while maintaining the
balance of accuracy and model complexity. More specifically, we find
by enhancing the classical triplet loss together with cross-entropy
loss, our method can explore the hard examples and build a discriminant
feature embedding yet compact enough for large-scale applications.
Our method is carried out progressively using Bayesian optimization,
and we call it the Progressive Learning Algorithm (PLA). Extensive
experiments on three large-scale datasets show that our PLA is comparable
or better than the state-of-the-arts. Especially, on the challenging
Market-1501 dataset, we achieve Rank-1=94.7%/mAP=89.4% while saving
at least 30\% parameters than strong part models.

HMFlow: Hybrid Matching Optical Flow Network for Small and Fast-Moving Objects

In this paper we investigate the exploitation of non linear scaling of distances for advanced nearest neighbor classification. Starting from the recently found relation between the \emph{Hypersphere Classifier} (HC) and the \emph{Adaptive Nearest Neighbor rule} (ANN) , here we propose {\itshape PowerHC}, an improved version of HC in which distances are normalized using a non linear mapping; non linear scaling of data, whose usefulness for feature spaces has been already assessed, has been hardly investigated for distances. A thorough experimental evaluation, involving 24 datasets and a challenging real world scenario of seismic signal classification, confirms the suitability of the proposed approach.

PowerHC: non linear normalization of distances for advanced nearest neighbor classification

The joint understanding of vision and language has been recently gaining a lot of attention in both the Computer Vision and Natural Language Processing communities, with the emergence of tasks such as image captioning, image-text matching, and visual question answering. As both images and text can be encoded as sets or sequences of elements -- like regions and words -- proper reduction functions are needed to transform a set of encoded elements into a single response, like a classification or similarity score. In this paper, we propose a novel fully-attentive reduction method for vision and language. Specifically, our approach computes a set of scores for each element of each modality employing a novel variant of cross-attention, and performs a learnable and cross-modal reduction, which can be used for both classification and ranking. We test our approach on image-text matching and visual question answering, building fair comparisons with other reduction choices, on both COCO and VQA 2.0 datasets. Experimentally, we demonstrate that our approach leads to a performance increase on both tasks.

A Novel Attention based Aggregation Function to Combine Vision and Language

Rethinking deep active learning: Using unlabeled data at model training

Traffic accidents caused by distracted driving have gradually increased in recent years. In this work, we propose a novel multi-feature fusion network based on pose estimation, for image-based distracted driving detection. Since hand is the most important part of driver to infer the distracted actions, our proposed method firstly detects hands using the human body posture information. In addition to the features extracted from the whole image, our network also include the important information of hand and human body posture. The global feature, hand and pose features are finally fused by weighted combination of probability vectors and concatenation of feature maps. The experimental results show that our method achieves state-of-the- art performance on our own SZ Bus Driver dataset and the public AUC Distracted Driver dataset.


Premium content

Explore and Explain: Self-Supervised Navigation and Recounting

Downloads

Next from ICPR 2020

RobusterNet: Improving Copy-Move Forgery Detection with Volterra-based Convolutions

Similar lecture

Explore and Explain: Self-Supervised Navigation and Recounting