Italy




Recognizing human activities from multi-channel time series data collected from wearable sensors has become an important practical application of machine learning. A serious challenge comes from the presence of coherent activities or body movements, such as movements of the head while walking or sitting, since signals representing these movements are mixed and interfere with each other. Basic multi-label classification typically assumes independence within the multiple activities. This is over-simplified and reduces modeling power even when using state-of-the-art deep learning methods. In this paper, we investigate this new problem, which we name ``Coherent Human Activity Recognition (Co-HAR)&#39;&#39;, that keeps complete conditional dependency between the multiple labels. Additionally, we treat Co-HAR as a dense labeling problem that classifies each sample on a time step with multiple coherent labels to provide high-fidelity and duration-sensitive support to high-precision applications. To explicitly model conditional dependency, a novel condition-aware deep architecture ``Conditional-UNet&#39;&#39; is developed to allow for multiple dense labeling for Co-HAR. We also contribute a first-of-its-kind Co-HAR dataset for head gesture recognition associated with a user&#39;s activity, walking or sitting, to the research community. Extensive experiments on this dataset show that our model outperforms state-of-the-art deep learning methods and achieves up to 92% accuracy on context-based head gesture classification.

ICPR 2020

Conditional-UNet: A Condition-aware Deep Model for Coherent Human Activity Recognition From Wearables

bayesian model

human activity recognition

segmentation

deep learning

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 





Recognizing human activities from multi-channel time series data collected from wearable sensors has become an important practical application of machine learning. A serious challenge comes from the presence of coherent activities or body movements, such as movements of the head while walking or sitting, since signals representing these movements are mixed and interfere with each other. Basic multi-label classification typically assumes independence within the multiple activities. This is over-simplified and reduces modeling power even when using state-of-the-art deep learning methods. In this paper, we investigate this new problem, which we name ``Coherent Human Activity Recognition (Co-HAR)'', that keeps complete conditional dependency between the multiple labels. Additionally, we treat Co-HAR as a dense labeling problem that classifies each sample on a time step with multiple coherent labels to provide high-fidelity and duration-sensitive support to high-precision applications. To explicitly model conditional dependency, a novel condition-aware deep architecture ``Conditional-UNet'' is developed to allow for multiple dense labeling for Co-HAR. We also contribute a first-of-its-kind Co-HAR dataset for head gesture recognition associated with a user's activity, walking or sitting, to the research community. Extensive experiments on this dataset show that our model outperforms state-of-the-art deep learning methods and achieves up to 92% accuracy on context-based head gesture classification.

poster

This article proposes a novel attention-based body-pose encoding for human activity recognition. Most of the existing human activity recognition approaches based on 3D pose data often enrich the input data using additional handcrafted representations such as velocity, super-normal vectors, pairwise relations, and so on. The enriched data complements the 3Dbody joint position data and improves model performance. In this paper, we propose a novel approach that learns enhanced feature representations from a given sequence of 3D body joints.To achieve this encoding, the approach exploits two body pose streams: 1) a spatial stream which encodes the spatial relationship between various body joints at each time point to learn spatial structure involving the spatial distribution of different body joints 2) a temporal stream that learns the temporal variation of individual body joints over the entire sequence duration to present a temporally enhanced representation. Afterwards,these two pose streams are fused with a multi-head attention mechanism. We also capture the contextual information from the RGB video stream using a deep Convolutional NeuralNetwork (CNN) model combined with a multi-head attention and a bidirectional Long Short-Term Memory (LSTM) network.Finally, the RGB video stream is combined with the fused body-pose stream to give a novel end-to-end deep model for effective human activity recognition.

Attention-Driven Body Pose Encoding for Human Activity Recognition


Accurate and robust detection of objects from
monocular images is a fundamental vision task. This paper
describes a novel approach of holistic scene understanding that
can simultaneously achieve multiple tasks of scene reconstruction
and object detection from a single monocular image. Rather than
pursuing an independent solution for each individual task as
most existing work does, we seek a globally optimal solution that
holistically resolves the multiple perception and reasoning tasks
in an effective manner. The approach explores the complementary
properties of multimodal RGB images and depth data to improve
scene perception tasks. It uniquely combines the techniques
of canonical correlation analysis and deep learning to learn
the most correlated features to maximize the modal crosscorrelation
for improving performance and robustness of object
detection in complex environments. Extensive experiments have
been conducted to evaluate and demonstrate the performances
of the proposed approach.

Object Detection on Monocular Images with Two-Dimensional Canonical Correlation Analysis


Representing brain activities by networks is very crucial to understand various cognitive states. This study proposes a novel method to estimate static and dynamic brain networks using Kulback-Leibler divergence. The suggested brain networks are based on the probability distributions of voxel intensity values measured by functional Magnetic Resonance Images (fMRI) recorded while the subjects perform a predefined cognitive task, called complex problem solving. We investigate the validity of the estimated brain networks by modeling and analyzing the different phases of complex problem solving process of human brain, namely planning and execution phases. The suggested computational network model is tested by a classification schema using Support Vector Machines. We observe that the network models can successfully discriminate the planning and execution phases of complex problem solving process with more than 90% accuracy, when the estimated dynamic networks, extracted from the fMRI data, are classified by Support Vector Machines.

Estimating Static and Dynamic Brain Networks by Kulback-Leibler Divergence from fMRI Data

This work describes an intelligent price suggestion system for online second-hand listings. In contrast to conventional pricing strategies which are employed to a large number of identical products, or to non-identical but similar products such as homes on Airbnb, the proposed system provides price suggestions for online second-hand items which are non-identical and fall into numerous different categories. Moreover, simplifying the item listing process for users is taken into consideration when designing the price suggestion system. Specifically, we design a truncate loss to train a vision-based price suggestion module which mainly takes some vision-based features as input to first classify whether an uploaded item image is qualified for price suggestion, and then offer price suggestions for items with qualified images. For the items with unqualified images, we encourage users to input some text descriptions of the items, and with the text descriptions, we design a multimodal item retrieval module to offer price suggestions. Extensive experiments demonstrate the effectiveness of the proposed system.

Price Suggestion for Online Second-hand Items


This paper provides a comprehensive comparative study of traditional, hybrid, and deep learning (DL) methods for detecting marine species in echograms. Acoustic backscatter data obtained from multi-frequency echosounders is visualized as echograms and typically interpreted by marine biologists via manual or semi-automatic methods, which are time-consuming. Challenges related to automatic echogram interpretation are the variable size and acoustic properties of the biological targets (marine life), along with significant inter-class similarities. Our
study explores and compares three types of approaches that cover the entire range of machine learning methods. Based on our experimental results, we conclude that an end-to-end DL-based framework, that can be readily scaled to accommodate new species, is overall preferable to other learning approaches for echogram interpretation, even when only a limited number of annotated training samples is available.

Detecting Marine Species in Echograms via Traditional, Hybrid, and Deep Learning Frameworks


Layout analysis of historical handwritten documents
is a key pre-processing step in document image analysis that, by
segmenting the image into its homogeneous regions facilitates
subsequent procedures such as optical character recognition and
automatic transcription. Learning-based approaches have shown
promising performances in layout analysis, however, the majority
of them require tedious pixel-wise labelled training data to
achieve generalisation capabilities, this limitation preventing their
application due to the lack of large labelled datasets. This paper
proposes a novel unsupervised representation learning method
for documents’ layout analysis that reduces the need for labelled
data: a sparse autoencoder is first trained in an unsupervised
the manner on a historical text document’s image; representation
of image patches, computed by the sparse encoder, is then used
to classify pixels into various region categories of the document
using a feed-forward neural network. A new training method,
inspired by the ISTA algorithm, is also introduced here to train
the sparse encoder. Experimental results on DIVA-HisDB dataset
demonstrate that the proposed method outperforms previous
approaches based on unsupervised representation learning while
achieving performances comparable to the state-of-the-art fully
supervised methods.

Ancient Document Layout Analysis: Autoencoders meet Sparse Coding


In recent years, deep neural networks have achieved
state-of-the-art performance in a variety of recognition and
segmentation tasks in medical imaging including brain tumor
segmentation. We investigate that segmenting a brain tumor is
facing to the imbalanced data problem where the number of pixels
belonging to the background class (non tumor pixel) is much
larger than the number of pixels belonging to the foreground
class (tumor pixel). To address this problem, we propose a multitask
network which is formed as a cascaded structure. Our model
consists of two targets, i.e., (i) effectively differentiate the brain
tumor regions and (ii) estimate the brain tumor mask. The first
objective is performed by our proposed contextual brain tumor
detection network, which plays a role of an attention gate and
focuses on the region around brain tumor only while ignoring the
far neighbor background which is less correlated to the tumor.
Different from other existing object detection networks which
process every pixel, our contextual brain tumor detection network
only processes contextual regions around ground-truth instances
and this strategy aims at producing meaningful regions proposals.
The second objective is built upon a 3D atrous residual network
and under an encode-decode network in order to effectively segment
both large and small objects (brain tumor). Our 3D atrous
residual network is designed with a skip connection to enables the
gradient from the deep layers to be directly propagated to shallow
layers, thus, features of different depths are preserved and used
for refining each other. In order to incorporate larger contextual
information from volume MRI data, our network utilizes the
3D atrous convolution with various kernel sizes, which enlarges
the receptive field of filters. Our proposed network has been
evaluated on various datasets including BRATS2015, BRATS2017
and BRATS2018 datasets with both validation set and testing
set. Our performance has been benchmarked by both regionbased
metrics and surface-based metrics. We also have conducted
comparisons against state-of-the-art approaches

A Multi-task Contextual Atrous Residual Network for Brain Tumor Detection & Segmentation


Multimedia Forensics experts work consists in providing answers about integrity of a specific media content and from where it comes from. Exploitation of any traces from JPEG double compressed images is often one of the main investigative path to be used for these purposes. Thus it is fundamental to have tools and algorithms able to safely estimate the first quantization matrix to further proceed with camera model identification and related tasks. In this paper, a technique based on extensive simulation is proposed, with the aim to infer the first quantization for a certain numbers of Discrete Cosine Transform (DCT) coefficients exploiting local image statistics without using any a-priori knowledge. The method provides also a reliable confidence value for the estimation which is of great importance for forensic purposes. Experimental results w.r.t. the state-of-the-art demonstrate the effectiveness of the proposed technique both in terms of precision and overall reliability

Computational Data Analysis for First Quantization Estimation on JPEG Double Compressed Images

3D human pose estimation from a single 2D video is an extremely difficult task because computing 3D geometry from 2D images is an ill-posed problem. Recent popular solutions adopt fully-supervised learning strategy, which requires to train a deep network on a large-scale ground truth dataset of 3D poses and 2D images. However, such a large-scale dataset with natural images does not exist, which limits the usability of existing methods. While building a complete 3D dataset is tedious and expensive, abundant 2D in-the-wild data is already publicly available. As a consequence, there is a growing interest in the computer vision community to design efficient techniques that use the unsupervised learning strategy, which does not require any ground truth 3D data. Such methods can be trained with only natural 2D images of humans. In this paper we propose an unsupervised method for estimating 3D human pose in videos. The standard approach for unsupervised learning is to use the Generative Adversarial Network (GAN) framework. To improve the performance of 3D human pose estimation in videos, we propose a new GAN network that enforces body consistency over frames in a video. We evaluate the efficiency of our proposed method on a public 3D human body dataset.

Unsupervised 3D Human Pose Estimation in Multi-view-multi-pose Video

This paper proposes a method based on visual motion primitives to address the problem of action understanding. The approach builds in an unsupervised way a dictionary of kinematic primitives from a set of sub-movements obtained by segmenting the velocity profile of an action on the basis of local minima derived directly from the optical flow. The dictionary is then used to describe each sub-movement as a linear combination of atoms using sparse coding. The descriptive capability of the proposed motion representation is experimentally validated on the MoCA dataset, a collection of synchronized multi-view videos and motion capture data of cooking activities. 
The results show that the approach, despite its simplicity, has a good performance in action classification, especially when the motion primitives are combined over time. Also, the method is proved to be tolerant to view point changes, and can thus support cross-view action recognition. 
Overall, the method may be seen as a backbone of a general approach to action understanding, with potential applications in robotics.

Premium content

Conditional-UNet: A Condition-aware Deep Model for Coherent Human Activity Recognition From Wearables

Downloads

Next from ICPR 2020

Attention-Driven Body Pose Encoding for Human Activity Recognition

Similar lecture

Machine-learned Regularization and Polygonization of Building Segmentation Masks