Italy

In cataract surgery, the operation is performed with the help of a microscope. Since the microscope enables watching real-time surgery by up to two people only, a major part of surgical training is conducted using the recorded videos. To optimize the training procedure with the video content, the surgeons require an automatic relevance detection approach. In addition to relevance-based retrieval, these results can be further used for skill assessment and irregularity detection in cataract surgery videos. In this paper, a three-module framework is proposed to detect and classify the relevant phase segments in cataract videos. Taking advantage of an idle frame recognition network, the video is divided into idle and action segments. To boost the performance in relevance detection, the cornea where the relevant surgical actions are conducted is detected in all frames using Mask R-CNN. The spatiotemporally localized segments containing higher-resolution information about the pupil texture and actions, and complementary temporal information from the same phase are fed into the relevance detection module. This module consists of four parallel recurrent CNNs being responsible to detect four relevant phases that have been defined with medical experts. The results will then be integrated to classify the action phases as irrelevant or one of four relevant phases. Experimental results reveal that the proposed approach outperforms static CNNs and different configurations of feature-based and end-to-end recurrent networks.

ICPR 2020

Relevance Detection in Cataract Surgery Videos by Spatio-Temporal Action Localization

phase recognition

instance segmentation

recurrent neural networks

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


technical paper


This presentation discusses the properties of the generalized conics that are used to create a unified framework for generating various types of the distance fields. The proposed representation provides a possibility to efficiently compute the proximity, arithmetic mean of the distances and a space tessellation with regard to the given set of polygonal objects, line segments and points. In addition, the weights can be introduced for objects, their parts and combinations. This fact leads to a hierarchical representation that can be efficiently obtained using the pixel-wise operations. The practical value of the proposed ideas is demonstrated on an example of applications like skeletonization, smoothing and optimal location finding.

Generalized Conics: properties and applications

Appliance identification using a histogram post-processing of 2D local binary patterns for smart grid applications

Pipeline for XAI based Automatic Audio Call Audit

Assessement of CNN-Based Methods for Poverty Estimation from Satellite images

Despite the rapid development of pedestrian detection, the problem of dense pedestrian detection is still unsolved, especially the upper limit of Recall caused by Non-Maximum-Suppression (NMS). For this reason, R2NMS is proposed to simultaneously detect full and visible body bounding boxes, by replacing the full-body BBoxes with less occluded visible body BBoxes in the NMS algorithm, achieving a higher recall. However, the P-RPN and P-RCNN modules proposed in R2NMS for simultaneous high quality full and visible body prediction require non-trivial positive/negative assigning strategies for anchor BBoxes. To simplify the prerequisites and improve the utility of R2NMS, we incorporate clustering analysis into the learning of visible body proposals from full-body proposals. Furthermore, to reduce the computation complexity caused by a large number of potentially visible body proposals, we introduce a novel occlusion pattern prediction branch on top of the R-CNN module (i.e. F-RCNN) to select the best-matched visible proposals for each full-body proposals and then feed them into another R-CNN module. Incorporated with R2NMS, our DualBox model can achieve competitive performance while only requires a few hyper-parameters. We validate the effectiveness of the proposed approach on the CrowdHuman and CityPersons datasets. Experimental results show that our approach achieves promising performance for detecting both non-occluded and occluded pedestrians, especially heavily occluded ones.

DualBox: Generating BBox Pair with Strong Correspondence via Occlusio­­n Pattern Clustering and Proposal Refinement

We introduce a real-time method that estimates the 3D human pose directly in the popular Bio Vision Hierarchy (BVH) format, given estimations of the 2D body joints originating from monocular color images. Our contributions include: (a) A novel and compact 2D pose representation. (b) A human body orientation classifier and an ensemble of orientation-tuned neural networks that regress the 3D human pose by also allowing for the decomposition of the body to an upper and lower kinematic hierarchy. This permits the recovery of the human pose even in the case of significant occlusions. (c) An efficient Inverse Kinematics solver that refines the neural-network-based solution providing 3D human pose estimations that are consistent with the limb sizes of a target person (if known). All the above yield a 33\% accuracy improvement on the Human 3.6 Million (H3.6M) dataset compared to the baseline method (MocapNET) while maintaining real-time performance (70 fps in CPU-only execution).

Occlusion-tolerant and personalized 3D human pose estimation in RGB images


Robotics applications in urban environments are subject to obstacles that exhibit specular reflections hampering autonomous navigation. On the other hand, these reflections are highly polarized and this extra information can successfully be used to segment the specular areas. In nature, polarized light is obtained by reflection or scattering. Deep Convolutional Neural Networks (DCNNs) have shown excellent segmentation results, but require a significant amount of data to achieve best performances. The lack of data is usually overcomed by using augmentation methods. However, unlike RGB images, polarization images are not only scalar (intensity) images and standard augmentation techniques cannot be applied straightforwardly. We propose to enhance deep learning models through a regularized augmentation procedure applied to polarimetric data in order to characterize scenes more effectively under challenging conditions. We subsequently observe an average of 18.1% improvement in IoU between non augmented and regularized training procedures on real world data.

Polarimetric Image Augmentation

Reinforcement Learning has been able to solve many complicated robotics tasks without any need for feature engineering in an end-to-end fashion. However, learning the optimal policy directly from the sensory inputs, i.e the observations, often requires processing and storage of a huge amount of data. In the context of robotics, the cost of data from real robotics hardware is usually very high, thus solutions that achieve high sample efficiency are needed. We propose a method that aims at learning a mapping from the observations into a lower-dimensional state
space. This mapping is learned with unsupervised learning using loss functions shaped to incorporate prior knowledge of the environment and the task. Using the samples from the state space, the optimal policy is quickly and efficiently learned. We test the method on several mobile robot navigation tasks in a simulation
environment and also on a real robot.

Low Dimensional State Representation Learning with Reward-shaped Priors

Fashion analysis is an attractive domain for vision research due to its direct applications in e-commerce contexts.
However, fashion datasets are commonly rather demanding, as both objects and attributes tend to be fine-grained and thus result in very long-tailed datasets.
Furthermore, relationships between objects and attributes are often dense, but are crucial for the performance of fashion applications.
In this paper, we propose to generate scene graphs for existing fashion datasets.
By detecting relationships between fashion objects, their parts, and their attributes we gain a better understanding of the scenes.
As no current fashion dataset provides scene graphs, we generate relationships between fashion objects from existing annotations.
The output is post-processed and filtered to generate a meaningful scene graph for each image.
In the experiments, we can show existing applications like image retrieval benefiting from the scene graph understanding.
We first evaluate the accuracy of the generated scene graphs.
Then, we employ scene graphs to fashion image retrieval in order to showcase their performance in real applications.
The results show various benefits for fashion applications by exploiting scene graph knowledge.
The resources and model for the proposed method are available on GitHub (https://github.com/shabnamsadegh/FashionGraph).

FashionGraph: understanding fashion data using scene graph generation


Lecture videos are rapidly becoming an invaluable source of information for students across the globe. Given the large number of online courses currently available, it is important to condense the information within these videos into a compact yet representative summary that can be used for search-based applications. We propose a framework to summarize whiteboard lecture videos by finding feature representations of detected handwritten content regions to determine unique content. We investigate multi-scale histogram of gradients and embeddings from deep metric learning for feature representation. We explicitly handle occluded, growing and disappearing handwritten content. Our method is capable of producing two kinds of lecture video summaries - the unique regions themselves or so-called key content and keyframes (which contain all unique content in a video segment). We use weighted spatio-temporal conflict minimization to segment the lecture and produce keyframes from detected regions and features. We evaluate both types of summaries and find that we obtain state-of-the-art peformance in terms of number of summary keyframes while our unique content recall and precision are comparable to state-of-the-art.

Premium content

Relevance Detection in Cataract Surgery Videos by Spatio-Temporal Action Localization

Downloads

Next from ICPR 2020

Generalized Conics: properties and applications

Similar lecture

Relevance Detection in Cataract Surgery Videos by Spatio-Temporal Action Localization