Italy


Semantic segmentation requires both a large receptive field and accurate spatial information. Although existing methods based on a fully convolutional network have greatly improved the accuracy, the prediction results still do not show
satisfactory when parsing small objects and boundary regions. We propose a refinement algorithm to improve the result generated by the front network. Our method takes a modified double-branches network to generate both segmentation masks and semantic boundaries, which serve as refinement algorithms’ input. We creatively introduce information entropy to represent the confidence of the neural network’s prediction corresponding to each pixel. The information entropy combined with the semantic boundary can capture those unpredictable pixels with
low-confidence through Monte Carlo sampling. Each selected pixel will serve as the initial seed for directed local search and refinement. According to the initial seed, our purpose is tantamount to searching the neighbor high-confidence regions,
and the re-labelling approach is based on high-confidence results. Remarkably, our method adopts a directed regional search strategy based on gradient descent to find the high-confidence region effectively. Our method can be flexibly embedded into
the existing encoder backbone and further improve the state of the art method’s accuracy both on Cityscapes and PASCAL VOC datasets. In evaluating some small objects, our method surpasses most of the state of the art methods.

ICPR 2020

Semantic Segmentation Refinement Using Entropy and Boundary-guided Monte Carlo Sampling and Directed Regional Search


semantic segmentation

image processing

deep learning

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


poster

In this work, we propose CBiGAN -- a novel method for anomaly detection in images, where a consistency constraint is introduced as a regularization term in both the encoder and decoder of a BiGAN. Our model exhibits fairly good modeling power and reconstruction consistency capability. We evaluate the proposed method on MVTec AD -- a real-world benchmark for unsupervised anomaly detection on high-resolution images -- and compare against standard baselines and state-of-the-art approaches. Experiments show that the proposed method improves the performance of BiGAN formulations by a large margin and performs comparably to expensive state-of-the-art iterative methods while reducing the computational cost. We also observe that our model is particularly effective in texture-type anomaly detection, as it sets a new state of the art in this category. Our code is available at https://github.com/fabiocarrara/cbigan-ad/.

Combining GANs and AutoEncoders for Efficient Anomaly Detection

I will present a state-of-the-art reinforcement learning method for autonomous driving. Our approach employs temporal difference learning in a Bayesian framework to learn vehicle control signals from sensor data. The agent has access to images from a forward facing camera, which are pre- processed to generate semantic segmentation maps. We trained our system using both ground truth and estimated semantic segmentation input.
Based on our observations from a large set of experiments, we conclude that training the system on ground truth input data leads to better performance than training the system on estimated input even if estimated input is used for evaluation.
The system is trained and evaluated in a realistic simulated urban environment using the CARLA simulator. The simulator also contains a benchmark that allows for comparing to other systems and methods. The required training time of the system is shown to be lower and the performance on the benchmark superior to competing approaches.

A Bayesian Approach to Reinforcement Learning of Vision-Based Vehicular Control

The quantification of visual affect data (e.g. face images) is essential to build and monitor automated affect modeling systems efficiently. Considering this, this work proposes quantified facial Temporal-expressiveness Dynamics (TED) to quantify the expressiveness of human faces. The proposed algorithm leverages multimodal facial features by incorporating static and dynamic information to enable accurate measurements of facial expressiveness. We show that TED can be used for high-level tasks such as summarization of unstructured visual data, and expectation from and interpretation of automated affect recognition models. To evaluate the positive impact of using TED, a case study was conducted on spontaneous pain using the UNBC-McMaster spontaneous shoulder pain dataset. Experimental results show the efficacy of using TED for quantified affect analysis.

Quantified Facial Temporal-Expressiveness Dynamics for Affect Analysis

Deep Learning systems have proven to be extremely
successful for image recognition tasks for which significant
amounts of training data is available, e.g., on the famous
ImageNet dataset. We demonstrate that for robotics applications
with cheap camera equipment, the low image quality, however,
influences the classification accuracy, and freely available data
bases cannot be exploited in a straight forward way to train
classifiers to be used on a robot. As a solution we propose to train
a network on degrading the quality images in order to mimic
specific low quality imaging systems. Numerical experiments
demonstrate that classification networks trained by using images
produced by our quality degrading network along with the high
quality images outperform classification networks trained only
on high quality data when used on a real robot system, while
being significantly easier to use than competing zero-shot domain
adaptation techniques.

A Simple Domain Shifting Network for Generating Low Quality Images


The text embedded in scene images can be seen everywhere in our lives. However, recognizing text from natural scene images is still a challenge because of its diverse shapes and distorted patterns. Recently, advanced recognition networks generally treat scene text recognition as a sequence prediction task. Although achieving excellent performance, these recognition networks consider the feature map cells as independent individuals and update cells state without utilizing the information of their related cells. And the local receptive field of traditional convolutional neural network (CNN) makes a single cell that cannot cover the whole text region in an image. Due to these issues, the existing recognition networks cannot extract the global context information in a visual scene. To deal with the above problems, we propose a Multi-head Self-relation Network(MSRN) for scene text recognition in this paper. The MSRN consists of several multi-head self-relation layers, which are designed for extracting the global context information of a visual scene. Then the information of the related cells can be fused by multi-head self-relation layer. Furthermore, experiments over several public datasets demonstrate that our proposed recognition network achieves superior performance on several benchmark datasets including IC03, IC13, IC15, SVT-Perspective.

A Multi-head Self-relation Network for
Scene Text Recognition

This works focuses on examining the performance of the Single Shot Detector (SSD) model in resource restricted systems where maintaining the power of the full model comprises a significant prerequisite. The proposed SSD variations examine the behavior of lighter versions of SSD while propose measures to limit the unavoidable performance shortage. The outcomes of the conducted research demonstrate a remarkable trade-off between performance losses, speed improvement and the required resource reservation. Thus, the experimental results evidence the efficiency of the presented SSD alterations towards accomplishing higher frame rates and retaining the performance of the original model.


A modified Single-Shot multibox Detector for beyond Real-Time Object Detection


Semantic scene segmentation is vital for a large variety of applications as it enables understanding of 3D data. Nowadays, various approaches based upon point clouds ignore the mathematical distribution of points and treat the points equally. The methods following this direction neglect the imbalance problem of samples that naturally exists in scenes. To avoid these issues, we propose a two-stage semantic scene segmentation framework based on self-attention mechanism and achieved state-of-the-art performance on 3D scene understanding tasks. We split the whole task into two small ones which efficiently relief the sample imbalance issue. In addition, we have designed a new self-attention block which could be inserted into submanifold convolution networks to model the long-range dependencies that exists among points. The proposed network consists of an encoder and a decoder, with the spatial-wise and channel-wise attention modules inserted. The two-stage network shares a UNet architecture and is an end-to-end trainable framework which could predict the semantic label for the scene point clouds fed into it. Experiments on standard benchmarks of 3D scenes implies that our network could perform at par or better than the existing state-of-the-art methods.

Progressive Scene Segmentation Based on Self-Attention Mechanism


We develop a new contour tracing algorithm to enhance the results of the 
latest object contour detectors. The goal is to achieve a perfectly closed,
single-pixel wide and detailed object contour, since this type of contour could
be analyzed using methods such as Fourier descriptors (FD). Therefore, we
present the Walk the Lines (WtL) algorithm: a standard regression CNN
trained to follow object contours. Convolutional Neural Networks (CNNs)
are rarely used for contour tracing, and we see great potential in using their
capabilities for this task. As initial step, we train the CNN only on ship contours,
but the principle is applicable to other objects. In contrast to the comparable
Non-Maximum Suppression (NMS) method, our approach produces connected contours with finer details. Closed object contours are binarized to produce excellent segmentations.

Walk the Lines: Object Contour Tracing CNN for Contour Completion of Ships


A rapidly growing amount of content posted online, such as food recipes, opens doors to new exciting applications at the intersection of vision and language. In this work, we aim to estimate the calorie amount of a meal directly from an image by learning from recipes people have published on the Internet, thus skipping time-consuming manual data annotation. Since there are few large-scale publicly available datasets captured in unconstrained environments, we propose the pic2kcal benchmark comprising 308,000 images from over 70,000 recipes including photographs, ingredients and instructions. To obtain nutritional information of the ingredients and automatically determine the ground-truth calorie value, we match the items in the recipes with structured information from a food item database.
We evaluate various neural networks for regression of the calorie quantity and extend them with the multi-task paradigm. Our learning procedure combines the calorie estimation with prediction of proteins, carbohydrates, and fat amounts as well as a multi-label ingredient classification. Our experiments demonstrate clear benefits of multi-task learning for calorie estimation, surpassing the single-task calorie regression by 9.9%. To encourage further research on this task, we make the code for generating the dataset and the models publicly available.

Multi-Task Calorie Prediction on a Novel Large-Scale Recipe Dataset Enriched With Nutritional Information

This presentation discusses a novel setup for automatic visual inspection of cracks in ceramic tile as well as and a study about the effect of various classifiers and height-varying illumination conditions for this task. The intuition behind this setup is that cracks can be better visualized under specific lighting conditions than others. Our setup, which is designed for field work with constraints in its maximum dimensions, can acquire images for crack detection with multiple lighting conditions using the illumination sources placed at multiple heights. Crack detection is then performed by classifying patches extracted from the acquired images in a sliding window fashion. We study the effect of lights placed at various heights by training classifiers both on customized as well as state-of-the-art architectures and evaluate their performance both at patch-level and image-level, demonstrating the effectiveness of our setup. More importantly, ours is the first study that demonstrates how height-varying illumination conditions can affect crack detection with the use of existing state-of-the-art classifiers. We provide an insight about the illumination conditions that can help in improving crack detection in a challenging real-world industrial environment.

Premium content

Semantic Segmentation Refinement Using Entropy and Boundary-guided Monte Carlo Sampling and Directed Regional Search

Downloads

Next from ICPR 2020

Combining GANs and AutoEncoders for Efficient Anomaly Detection

Similar lecture

Transfer Learning Through Weighted Loss Function and Group Normalization for Vessel Segmentation from Retinal Images