Italy

An empirical comparison between different skeleton-based methods for both pose estimation and action recognition speaker actions in lecture videos.

ICPR 2020

Skeleton-based Methods for Speaker ActionClassification on Lecture Videos

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


technical paper

Machine learning techniques and algorithms are employed in many application domains such as financial applications, recommendation systems, medical diagnosis systems, and self-driving cars. They play a crucial role in harnessing the power of Big Data being produced every day in our digital world. In general, building a well-performing machine learning pipeline is an iterative and complex process that requires a solid understanding of various techniques that can be used in each component of the machine learning pipeline. Feature engineering(FE) is one of the most time-consuming steps in building machine learning pipelines. It requires a deep understanding of the domain and data exploration to discover relevant hand-crafted features from raw data. In this work, we empirically evaluate the impact of integrating an automated feature extraction tool (AutoFeat) into two automated machine learning frameworks, namely, Auto-Sklearn and TPOT, on their predictive performance. Besides, we discuss the limitations of AutoFeat that need to be addressed in order to improve the predictive performance of the automated machine learning frameworks on real-world datasets.

An Empirical Analysis of Integrating Feature Extraction to Automated Machine Learning Pipeline

Knowledge Tracing (KT) is the task of tracing evolving knowledge state of each student as (s)he engages with a sequence of learning activities and can provide personalized instructions. However, exiting methods such as Bayesian Knowledge Tracing (BKT) and Deep Knowledge Tracing (DKT) ei-ther cannot capture the relationship among different concepts or lack of in-terpretability. Although Knowledge Tracing Machines (KTM) makes up for these shortcomings, it only uses a linear function to model students' knowledge states, which cannot capture more information contained in each feature. To solve these problems, this work introduces a novel model called Input-aware Neural Knowledge Tracing Machine (INKTM). Unlike standard FM-based methods that focus on the feature interactions, our model focuses more on the information contained in each feature itself. INKTM converts the weight of each feature to a multidimensional vector, which can be used to learn a unique weight for each feature in different instances by an atten-tion network, so as to highlight the most important features and enhance its interpretability. What’s more, it can capture the non-linear and complex in-herent structure of real-world data through deep neural networks. Especial-ly, when one problem is related with multiple knowledge components, our model can capture more useful information than other FM-based methods by retaining all information of 2-order feature interactions and distinguish-ing contributions of each features. Extensive experiment results show our model consistently outperform existing models in a range of KT datasets.

Input-aware Neural Knowledge Tracing Machine

Training machine learning models in a supervised manner
requires vast amounts of labeled data, where the labels are typically
provided by humans manually annotating samples using a variety of
tools. In this work, an intelligent annotation tool is proposed to combine
the fast and efficient labeling capabilities of modern machine learning
models with the reliable and accurate but slow correction capabilities of
human annotators. For the example of instance segmentation in videos,
techniques to interactively condition a model on previously predicted
and manually annotated (corrected) frames are explored to create an
iterative workflow combining the advantages of the intelligent model and
the human annotator. Thereby, the intelligent model conducts the bulk
of the work, performing instance detection, tracking, and segmentation,
and enables the human annotator to correct individual frames selectively
and instances. The proposed approach is based on the one-shot learning
paradigm, and, thus, avoids the computational cost of online retraining.
For this purpose, we use Siamese neural networks to transfer annotations
from one video frame to another. Multiple interaction options regarding
the choice of the additional input data to the neural network, e.g., model
predictions or manual corrections, are explored to refine the labeling
performance of the given model and speed up the annotation process.

Intelligent and Interactive Video Annotation for Instance Segmentation using Siamese Neural Networks

State-of-the-art instance segmentation techniques currently provide a bounding box, class, mask, and scores for each instance. What they do not provide is an epistemic uncertainty estimate of these predictions. With our approach, we want to identify corner cases by considering the epistemic uncertainty. Corner cases are data/situations that are underrepresented or not covered in our data set. Our work is based on Mask R-CNN. We estimate the epistemic uncertainty by extending the architecture with Monte-Carlo dropout layers. By repeatedly executing the forward pass, we create a large number of predictions per instance. Afterward, we cluster the predictions of an instance based on the bounding box coordinates. It becomes possible to determine the epistemic position uncertainty for the bounding boxes and the classifier's epistemic class uncertainty. For the epistemic uncertainty regarding the bounding box position and the class assignment, we provide a criterion for detecting corner cases utilizing the model's epistemic uncertainty.

Towards Corner Case Detection by Modeling the Uncertainty of Instance Segmentation Networks

We present an approach for driver identification, which is useful in many automotive applications such as safety or comfort functions. Driver identification would also be of great interest to other business models, such as car rental and car-sharing companies. The identification method is based on the driver's physiological state or rather his/her electrocardiogram (ECG) data. For this purpose, we have recorded ECG data of 25 people driving in a simulated environment. To identify a driver, we extend our existing novelty detection by aggregating local features over time. To do so, we extracted features and trained a Gaussian Mixture Model (GMM) to exploit localities present in the recorded sensor data. With novelty detection by aggregating local features, we are smoothing the noisy signal and reducing the dimensionality for further processing in a one-class SVM classification. Based on the output, a decision function decides whether the driver is unknown or well-known, and if the driver is well-known, who of the known driver is it.

Novelty based Driver Identification on RR Intervals from ECG Data

In the era of industry 4.0, factories around the world are developing towards automation and artificial intelligence, in which industrial detection plays an important role. After the cutting process, the surface of a stainless steel plate may produce various defects, such as scratches, chisels, and stains. Due to the characteristic of bright reflections on the surface of a stainless steel plate, the traditional manual comparison detection method is time-consuming, laborious, and prone to different detection results due to the interference of high reflection, resulting in the outflow of defective products. This paper used existing mature deep learning models for object detection, YOLOv3(You Only Look Once) and SSD(Single Shot MultiBox Detector), which are the base network architectures for the defect detection of stainless steel plates, in order to eeffectively improve the accuracy of stainless steel plate detection. Through image preprocessing, the relative positions of sample defects are marked to improve data processing before training, in order that a large number of image samples can be quickly and effectively processed for training.

Defect Detection of Stainless Steel Plates Using Deep Learning Technology

Semantic segmentation and shape detection of lumbar vertebrae, sacrum, and femoral heads from clinical X-ray images are important and challenging tasks. In this paper, we propose a new multi-path convolutional neural network, BiLuNet, for semantic segmentation on X-ray images. The network is capable of medical image segmentation with very limited training data. With the shape fitting of the bones, we can identify the location of the target regions very accurately for lumbar vertebra inspection. We collected our dataset and annotated by doctors for model training and performance evaluation. Compared to the state-of-the-art methods, the proposed technique provides better mIoUs and higher success rates with the same training data. The experimental results have demonstrated the feasibility of our network to perform semantic segmentation for lumbar vertebrae, sacrum, and femoral heads.

BiLuNet: A Multi-path Network for Semantic Segmentation on X-ray Images

The presentation reviews the work and demonstrates the purposed hybrid machine learning techniques to predict the energy consumption with multi-source data.

A Hybrid Machine Learning Approach for Energy Consumption Prediction in Additive Manufacturing

We present a multiregion image segmentation approach which utilizes multiscale anisotropic diffusion based scale spaces. By combining powerful edge preserving anisotropic diffusion smoothing with isolevel set linking and merging, we obtain coherent segments which are tracked across multiple scales. A hierarchical tree representation of the given input image with progressively simplified regions is used with intra-scale splitting and inter-scale merging for obtaining multiregion segmentations. Experimental results on natural and medical images indicate that multiregion, multiscale image segmentation (MMIS) approach obtains coherent segmentation results.

Multiregion multiscale image segmentation with anisotropic diffusion

Deep-learning-based methods have been used for 3D action recognition in recent years. Methods based on recurrent neural networks (RNNs) have the advantage of modeling long-term context, but they fo- cus mainly on temporal information and ignore the spatial relationships in each skeleton frame. In addition, it is difficult to handle a very long skeleton sequence using an RNN. Compared with an RNN, a convolu- tional neural network (CNN) is better able to extract spatial information. To model the temporal information of skeleton sequences and incorpo- rate the spatial relationship in each frame efficiently using a CNN, this paper proposes a multi-feature skeleton representation for encoding fea- tures from original skeleton sequences. The relative distances between joints in each skeleton frame are computed from the original skeleton sequence, and several relative angles between the skeleton structures are computed. This useful information from the original skeleton sequence is encoded as pixels in grayscale images. To preserve more spatial re- lationships between input skeleton joints in these images, the skeleton joints are divided into five groups: one for the trunk and one for each arm and each leg. Relationships between joints in the same group are more relevant than those between joints in different groups. By rear- ranging pixels in encoded images, the joints that are mutually related in the spatial structure are adjacent in the images. The skeleton repre- sentations, composed of several grayscale images, are input to CNNs for action recognition. Experimental results demonstrate the effectiveness of the proposed method on three public 3D skeleton-based action datasets.

Premium content

Skeleton-based Methods for Speaker ActionClassification on Lecture Videos

Downloads

Next from ICPR 2020

An Empirical Analysis of Integrating Feature Extraction to Automated Machine Learning Pipeline

Similar lecture

Writer Identification Using Deep Neural Networks: Impact of Patch Size and Number of Patches