Italy

This work describe an algorithm for the automatic analysis of the waggle dance of honeybees. The algorithm analyses a video of a beehive with 13,624 frames, acquired at 25 frames/second. The algorithm employs the following traditional image processing steps: conversion to grayscale, low pass filtering, background subtraction, thresholding, tracking and clustering to detect run of bees that perform waggle dances. The algorithm detected 44,530 waggle events, i.e. one bee waggling in one time frame, which were then clustered into 511 waggle runs. Most of these were concentrated in one section of the hive. The accuracy of the tracking was 90% and a series of metrics like intra-dance variation in angle and duration were found to be consistent with literature. Whilst this algorithm was tested on a single video, the
ideas and steps, which are simple as compared with Machine and Deep Learning techniques, should be attractive for researchers in this field who are not specialists in more complex techniques.

ICPR 2020

Automatic Analysis of Bees’ Waggle Dance

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


technical paper

We investigate the task of entity counting in overhead imagery from the perspective of re-purposing representations learned from ground imagery, e.g., ImageNet, via feature adaptation. We explore two directions of feature adaptation and analyze their performances using two popular aerial datasets for vehicle counting: PUCPR+ and CARPK. First, we explore proxy self-supervision tasks such as RotNet, jigsaw, and image inpainting to re-fine the pretrained representation. Second, we insert additional network layers to adaptively select suitable features (e.g., squeeze and excitation blocks) or impose desired properties (e.g., using active rotating filters for rotation invariance). Our experimental results show that different adaptations produce different amounts of performance improvements depending on data characteristics.

Fine-tuning for one-look regression vehicle counting in low-shot aerial datasets

3D city-scale point cloud stitching is a critical component for large data collection, environment change detection, in which massive amounts of 3D data are captured under different times and conditions. 
This paper proposes a novel point cloud stitching approach, that automatically and accurately stitches multiple city-scale point clouds, which only share relatively small overlapping areas, into one single model for a larger geographical coverage. The proposed method firstly employs 2D image mosaicking techniques to estimate 3D overlapping areas among multiple point clouds, then applies 3D point cloud registration techniques to estimate the most accurate transformation matrix for 3D stitching.
The proposed method is quantitatively evaluated on city-scale reconstructed point cloud dataset and real-world city LiDAR dataset, in which, our method outperforms other competing methods with significant margins and achieved the highest precision score, recall score, and F-score.
Our method makes an important step towards automatic and accurate city-scale point cloud data stitching, which could be used in a variety of applications.

City-Scale Point Cloud Stitching Using 2D/3D Registration for Large Geographical Coverage

Motion imagery interpretability is commonly represented by the video National Imagery Interpretability Rating Scale (VNIIRS), which is a subjective metric based on human analysts’ visual assessment. Therefore, VNIIRS is a very time-consuming task. This paper presents the development of a fully automated motion imagery interpretability prediction, called AMIIP. AMIIP employs a three-dimensional convolutional neural network (3D-CNN) that accepts as inputs many video blocks (small image sequences) extracted from motion imagery and outputs the classified label for each video block. The result is a histogram of the labels/categories that is then used to estimate the interpretability of the motion imagery. For each training video clip, it is labeled based on its subjectively rated VNIIRS level; thus, the required human annotation for training data is minimized. By using a collection of 76 high definition aerial video clips, three preliminary experimental results indicate that the estimation error is within 0.5 VNIIRS rating scale

On the development of a classification based automated motion imagery interpretability prediction system

Current AI machine learning approaches perform well with similar sensors for data collection, training, and testing. The ability to learn and analyze data from multiple sources would enhance capabilities for AI systems. This paper presents a deep learning-based multi-source self-correcting approach to fuse data with different modalities. The data-level fusion approach maximizes the capability to detect unanticipated events/targets augmented with machine learning methods. The proposed Domain Adaptation for Efficient Learning Fusion (DAELF) deep neural network adapts to changes of the input distribution allowing for self-correcting of multiple source classification and fusion. When supported by a distributed computing hierarchy, the proposed DAELF scales up in neural network size and out in geographical span. The design of DAELF includes various types of data fusion, including decision-level and feature-level data fusion. The results of DAELF highlight that feature-level fusion outperforms other approaches in terms of classification accuracy for the digit data and the Aerial Image Data analysis

Deep Learning Based Domain Adaptation with Data Fusion for Aerial Image Data Analysis

Full Motion Video (FMV) and Wide Area Motion Imagery (WAMI) systems have been used extensively in the recent years to collect overhead imagery to produce value-added geospatial products. In order to better understand the limitations of these systems and to define the best flight conditions under which they can be operated, an integrated modeling and simulation (M&S) framework named Sensair was developed. This paper presents how this tool can be leveraged to simulate data collections with airborne camera systems using different parameters under varied conditions. Sensair can simulate realistic large-scale environments with moving vehicles and generate representative FMV and WAMI high-resolution imagery. The built-in interactive analysis tools enable the study of the different factors impacting the application of computer vision algorithms to aerial images. This paper presents Sensair’s M\&S capabilities and describes use cases where the framework is used to perform tasks that include 3D reconstruction and the generation of an image dataset to support the evaluation of vehicle detection approaches.

Modeling and Simulation Framework for Airborne Camera Systems

We propose a novel unsupervised approach based on a two-stage object-centric adversarial framework that only needs object regions for detecting frame-level local anomalies in videos. The first stage consists in learning the correspondence between the current appearance and past gradient images of objects in scenes deemed normal, allowing us to either generate the past gradient from current appearance or the reverse. The second stage extracts the partial reconstruction errors between real and generated images (appearance and past gradient) with normal object behaviour, and trains a discriminator in an adversarial fashion. In inference mode, we employ the trained image generators with the adversarially learned binary classifier for outputting region-level anomaly detection scores. We tested our method on four public benchmarks, UMN, UCSD, Avenue and ShanghaiTech and our proposed object-centric adversarial approach yields competitive or even superior results compared to state-of-the-art methods.

Local Anomaly Detection in Videos using Object-Centric Adversarial Learning

Recent years, typhoon damages has become social problem owing to climate change. In 9 September 2019, Typhoon Faxai passed on the Chiba in Japan, whose damages included with electric provision stop because of strong wind recorded on the maximum 45 meter per second. A large amount of tree fell down, and the neighbor electric poles also fell down at the same time. These disaster features have caused that it took 18 days for recovery longer than past ones. Immediate responses are important for faster recovery. As long as we can, aerial survey for global screening of devastated region would be required for decision support to respond where to recover ahead. This pa-per proposes a practical method to visualize the damaged areas focused on the typhoon disaster features using aerial photography. This method can classify eight classes which contains land covers without damages and areas with disaster. Using target feature class probabilities, we can visualize disas-ter feature map to scale a color range. Furthermore, we can realize explain-able map on each unit grid images to compute the convolutional activation map using Grad-CAM. We demonstrate case studies applied to aerial photo-graphs recorded at the Chiba region after typhoon.

Natural Disaster Classification using Aerial Photography Explainable for Typhoon Damaged Feature

The improvement of the spatial resolution of natural images is an important task for many practical applications of current image processing. The endeavor of applying single image super-resolution, which only uses one low-resolution input image, is frequently hampered by the presence of noise or artifacts in the input image, so denoising and super-resolution algorithms are usually applied to obtain a noiseless high-resolution image. In this work, several traditional and deep learning methods for denoising and super-resolution are hybridized to ascertain which combinations of them yield the best possible quality. The experimental design includes the introduction of Gaussian noise, Poisson noise, salt-and-pepper noise, and uniform noise into the low-resolution inputs. It is found that denoising must be carried out before super-resolution for the best results. Moreover, the obtained results indicate that deep learning techniques clearly outperform the traditional ones.

Performance of Deep Learning and Traditional Techniques in Single Image Super-Resolution of Noisy Images

The classification performance of Convolutional Neural Networks (CNNs) can be hampered by several factors. Sensor noise is one of these nuisances. In this work, a study of the effect of noise on these networks is presented. The methodological framework includes two realistic models of noise for present day CMOS vision sensors. The models allow to include separately Poisson, Gaussian, salt \& pepper, speckle and uniform noise as sources of defects in image acquisition sensors. Then, synthetic noise may be added to images by using that methodology in order to simulate common sources of image distortion. Additionally, the impact of the brightness in conjunction with each selected kind of noise is also addressed. This way, the proposed methodology incorporates a brightness scale factor to emulate images with low illumination conditions. Based on these models, experiments are carried out for a selection of state of the art CNNs. The results of the study demonstrate that Poisson noise has a small impact on the performance of CNNs, while speckle and salt \& pepper noise together the global illumination level can substantially degrade the classification accuracy. Also, Gaussian and uniform noise have a moderate effect on the CNNs.

The effect of noise and brightness on convolutional deep neural networks

Noise which can appear in images affects the classification performance of Convolutional Neural Networks (CNNs). In particular, the impact of linear motion blur, which is one of the possible noises, in the classification performance of CNNs is assessed in this work. A realistic vision sensor model has been proposed to produce a linear motion blur effect in input images. This methodology allows analyzing how the performance of several considered state of the art CNNs is affected. Experiments that have been carried out indicate that the accuracy is heavily degraded by a high length of the displacement, while the angle of displacement deteriorates the performance to a lesser extent.

Premium content

Automatic Analysis of Bees’ Waggle Dance

Downloads

Next from ICPR 2020

Fine-tuning for one-look regression vehicle counting in low-shot aerial datasets

Similar lecture

Analyzing Zero-shot Cross-lingual Transfer in Supervised NLP Tasks