Italy

Eye tracking systems that estimate the point-of-gaze are essential in extended reality (XR) systems as they enable new interaction paradigms and technological improvements. It is important for these systems to maintain accuracy when the headset moves relative to the head (known as device slippage) due to head movements or user adjustment. One of the most accurate eye tracking techniques, which is also insensitive to shifts of the system relative to the head, uses two or more infrared (IR) light emitting diodes to illuminate the eye and an IR camera to capture images of the eye. 
An essential step in estimating the point-of-gaze in these systems is the precise determination of the location of two or more corneal reflections (virtual images of the IR-LEDs that illuminate the eye) in images of the eye. Eye trackers tend to have multiple light sources to ensure at least one pair of reflections for each gaze position. The use of multiple light sources introduces a difficult problem: the need to match the corneal reflections with the corresponding light source over the range of expected eye movements. 
Corneal reflection detection and matching often fail in XR systems due to the proximity of camera and steep illumination angles of light sources with respect to the eye. The failures are caused by corneal reflections having varying shape and intensity levels or disappearance due to rotation of the eye, or the presence of spurious reflections.
We have developed a fully convolutional neural network, based on the UNET architecture, that solves the detection and matching problem in the presence of spurious and missing reflections. Eye images of 25 people were collected in a virtual reality headset using a binocular eye tracking module consisting of five infrared light sources per eye. A set of 4,000 eye images were manually labelled for each of the corneal reflections, and data augmentation was used to generate a dataset of 40,000 images. The network is able to correctly identify and match 91% of corneal reflections present in the test set. This is comparable to a state-of-the-art deep learning system, but our approach requires 33 times less memory and executes 10 times faster. The proposed algorithm, when used in an eye tracker in a VR system, achieved an average mean absolute gaze error of 1°. This is a significant improvement over the state-of-the-art learning-based XR eye tracking systems that have reported gaze errors of 2-3°.


ICPR 2020

Detection and Correspondence Matching of Corneal Reflections for Eye Tracking Using Deep Learning

virtual reality

eye tracking

deep learning

Brought to you by **National Defense Industrial Association ([NDIA](https://www.google.com/url?q=https%3A%2F%2Fwww.ndia-sd.org%2F&sa=D&sntz=1&usg=AFQjCNHj_BQx1K4KVKKp1NTIsAltZOG9Vg))** and hosted by the **Battlespace Exploitation of Mixed Reality ([BEMR](https://cdn.dvidshub.net/media/video/2101/DOD_108139354/DOD_108139354-1024x576-1769k.mp4)) Lab**, at **Naval Information Warfare Center Pacific ([NIWC PAC](https://www.google.com/url?q=https%3A%2F%2Fwww.public.navy.mil%2Fnavwar%2FNIWC-Pacific%2FPages%2Fdefault.aspx&sa=D&sntz=1&usg=AFQjCNFNS160IEsocfaBrnCNB3u7NqLHPw))**, and welcoming academia, government/DOD, and industry leaders who support Industrial Augmented Reality/Virtual Reality by using the following technologies:   


* Augmented Reality (AR) / Virtual Reality (VR)

* 2D/3D displays

* mobile displays

* wearables

* any other "immersive" technology

### In all fields that touch:

* 3D modeling

* Manufacturing 

* Installations 

* Maintenance and Repairs

* Construction

* Logistics

* Additive manufacturing

* Training

The technical exchange is a two day event held online this year due to COVID-19. It will feature two days of oral presentations, demonstrations, and posters from leaders in AR/VR and other display technologies, including government briefings on active projects within the DOD. The technical exchange is unclassified and we welcome participation from the Navy, DoD as well as from industry and academia. 

The goals of TEAR 3M are:

*  To spread awareness of current immersive technology research relevant to Navy/DOD applications

*  To connect AR researchers with experts in DOD to learn about needs and requirements

*  To build and strengthen collaborations within the DOD research community

![](https://assets.underline.io/uploads/markdown_image/1/image/0c13ac0ffa096b2353573964d8b89789.png)

TEAR 3M General Chairs:

Heidi Buck,  Director @ BEMR Lab, Naval Information Warfare Center, Pacific

Lauren Burnette, BEMR Lab, Naval Information Warfare Center, Pacific


For more information, contact Lauren and Heidi at:  tear3m@spawar.navy.mil

Questions regarding sponsorship/registration contact:  ndiasd@sandiegositepros.com

Conference host POC is Tammy Sanchez (NDIA):  tamsanch@cisco.com

TEAR 3M 2021

poster

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


Real-time Cross-dataset Quality Production Assessment in Industrial Laser Cutting Machines

SURVANT: an innovative semantics-based Surveillance Video Archives Investigation Assistant

Assessement of CNN-Based Methods for Poverty Estimation from Satellite images


This paper presents chewing data analysis based on the new approach of chewing detection for diet monitoring applications. The proposed approach is based on chewing detection using a proximity sensor in capturing the temporalis muscle movement during chewing. The aim is to support the development of non-contact-based chewing detection. A wearable device of eye-glass is used with the sensor mounted at the right side of the eyeglass temple using a 3D printed housing. The main activity involved in this study is resting and eating, three test food that represents different food hardness (carrot, banana, and apple) with a portion of one spoonful. Several upper cut-off frequencies (fc2) of bandpass filters were used during the analyses. The signals were evaluated using accuracy and F1-score for classification and the absolute mean of error for chewing count estimation. In the classification stage, using the setting of a 10-fold cross-validation method and a 3s segmented window, fc2 of 6Hz gives the highest accuracy with 97.6%, while, 2.5Hz gives the lowest accuracy of 92.6%. However, in the chewing count estimation stage, which is based on a 240s segmented window, 2.4Hz able to give a smaller percentage absolute error of 2.69%, compare to 6Hz with 12.11%. It can be concluded that the chewing frequency was under 2.5Hz, but, the self-reporting labeling approach used in this study reduced the accuracy of the system as fc2 equal to 2.5Hz is used. Further analysis of chewing count shows that might in relating the total chewing count with different food hardness.

Analysis of Chewing Signals Based on Chewing Detection Using Proximity Sensor for Diet Monitoring

King-Sun Fu Award: Ching Yee Suen

Tour #1 with live discussion

Tour #2 with live discussion

Tour #3 with live discussion

Tour #4 with live discussion

Tour #5 with live discussion

technical paper

live discussion

https://d3smihljt9218e.cloudfront.net/transcript/9159/subtitle/1efc7759a2c5677d5452af82643f21e4.vtt

English (automatic)

Premium content

Detection and Correspondence Matching of Corneal Reflections for Eye Tracking Using Deep Learning

Downloads

Next from ICPR 2020

Real-time Cross-dataset Quality Production Assessment in Industrial Laser Cutting Machines

Similar lecture

Enabling Human-Centered AR Through VR Human Performance Experimentation