United States

Today we are already seeing vision based Artificial Intelligence (AI) systems being developed throughout defense. In autonomous vehicles we see that there are systems that can take video and other sensor information and detect objects, signs and lane markings. In robotics systems have to use perception to understand their environment so they can safely interact with humans and recognize components and products that they are going to manipulate. In aircraft maintenance training systems there is the need to replace manual aircraft inspections to identify components that need repair. Finally, we also see vision based AI systems in force protection where we can identify anomalous behavior and potential threats on the stream of massive data that we have collected. This presentation will start by going over the different challenges associated with computer vision applications for Augmented Reality (AR) today. It will then introduce you to Unity Perception. Highlighting what is it, what it does, and why AR based computer vision applications can benefit from it. Then it will provide a short overview of an AR iPad application customer use case Unity worked on. Lastly, it will wrap up with where Unity can take Unity Perception in the future. 



TEAR 3M 2021

Unity Computer Vision

augmented reality

synthetic data

computer vision

Brought to you by **National Defense Industrial Association ([NDIA](https://www.google.com/url?q=https%3A%2F%2Fwww.ndia-sd.org%2F&sa=D&sntz=1&usg=AFQjCNHj_BQx1K4KVKKp1NTIsAltZOG9Vg))** and hosted by the **Battlespace Exploitation of Mixed Reality ([BEMR](https://cdn.dvidshub.net/media/video/2101/DOD_108139354/DOD_108139354-1024x576-1769k.mp4)) Lab**, at **Naval Information Warfare Center Pacific ([NIWC PAC](https://www.google.com/url?q=https%3A%2F%2Fwww.public.navy.mil%2Fnavwar%2FNIWC-Pacific%2FPages%2Fdefault.aspx&sa=D&sntz=1&usg=AFQjCNFNS160IEsocfaBrnCNB3u7NqLHPw))**, and welcoming academia, government/DOD, and industry leaders who support Industrial Augmented Reality/Virtual Reality by using the following technologies:   


* Augmented Reality (AR) / Virtual Reality (VR)

* 2D/3D displays

* mobile displays

* wearables

* any other "immersive" technology

### In all fields that touch:

* 3D modeling

* Manufacturing 

* Installations 

* Maintenance and Repairs

* Construction

* Logistics

* Additive manufacturing

* Training

The technical exchange is a two day event held online this year due to COVID-19. It will feature two days of oral presentations, demonstrations, and posters from leaders in AR/VR and other display technologies, including government briefings on active projects within the DOD. The technical exchange is unclassified and we welcome participation from the Navy, DoD as well as from industry and academia. 

The goals of TEAR 3M are:

*  To spread awareness of current immersive technology research relevant to Navy/DOD applications

*  To connect AR researchers with experts in DOD to learn about needs and requirements

*  To build and strengthen collaborations within the DOD research community

![](https://assets.underline.io/uploads/markdown_image/1/image/0c13ac0ffa096b2353573964d8b89789.png)

TEAR 3M General Chairs:

Heidi Buck,  Director @ BEMR Lab, Naval Information Warfare Center, Pacific

Lauren Burnette, BEMR Lab, Naval Information Warfare Center, Pacific


For more information, contact Lauren and Heidi at:  tear3m@spawar.navy.mil

Questions regarding sponsorship/registration contact:  ndiasd@sandiegositepros.com

Conference host POC is Tammy Sanchez (NDIA):  tamsanch@cisco.com

technical paper

To address the Navy's need for cross-domain transition goggles with an integrated, illuminated display, Intellisense Systems, Inc. is developing a new Transition Wide-Angle Goggles with Integrated Illuminated Display (TWIGIID) based on an innovative design that uniquely integrates an ellipsoidal visor with reflective coating, free-form plastic-molded optics, and high resolution miniature microdisplays. The use of ellipsoidal reflectors provide wide Field-of-View; (FOV) distortion-free collimation; and free-form surface design to correct the geometrical distortion over swim goggles. Water-proof sealing of the microdisplay and electronics allows seamless transition between underwater and above-water operation. Use of a passive coating and purging with dry gas enables the small goggles to be cleared if flooded with water, to equalize pressure as needed, and to be fog-free. The proposed TWIGIID device will have revolutionary new capabilities in Augmented Reality (AR) visualization, with an extremely light and low-profile goggles integrated display, that will be producible at low cost. By providing these new capabilities, the TWIGIID device can help provide the Navy, Air Force, Marine Corps, and other military forces with enhanced situational awareness and capabilities.

Transition Wide-Angle Goggles with Integrated Illuminated Display

The digital revolution can no longer be ignored, the warfighter requires real-time access to fully interactive digital twins in support of their specific mission. Using automation digital twins can be created rapidly and affordably for every DoD weapon system in support of design, training, and logistics.

Digital Twins for the Warfighter

Mixed Reality is evolving rapidly with ever increasing examples of successful operational deployment and growing confidence in the maturity of Mixed Reality solutions to support a range of analytics, training, and operational functions and deliver the operational effect and efficiencies that were predicted. Sensor data and AI integration in mixed reality will provide new way to interact with and explore sensed environmental and geo-spatial data, as well as high-resolution imagery enabling increased autonomy for remotely deployed operators, while retaining the capacity to reach back and collaborate with experts to support analysis, decision making and mission planning and rehearsal.

Operational Remote Support

There is immense pressure to improve the effectiveness and quality of maintainer training with a direct correlation between maintenance efficiency and operational readiness. The spread and acceptance of virtual and augmented reality technologies have made these capabilities a must-have for training personnel.

These technologies immerse and engage students, reinforce proper techniques, save on equipment wear and tear, enabling students to train at their own pace, and improve the safety and efficiency of training in a safe environment.

VE mentor allows developers to easily transition training content from the classroom to the hands-on experience in augmented reality on HoloLens 2 and mobile devices. At the same time, VE Mentor allows content authored on the “flight-line” to flow back into the classroom, closing the loop on providing efficient and effective maintenance training.

This presentation will discuss utilizing Mixed-Reality technology to keep classroom training current, efficient, and comprehensive.


VE Mentor: A Mixed-Reality Training System with a Secure Cloud Solution

Anyone who has spent time recently working in the immersive training and simulation space fully understands the voracious appetite for new training content. Instructors and curriculum designers are often unable to keep up with the ongoing content demands of their XR training programs. The ultimate solution to address the current content drought is to deliver tools to end users that democratize and streamline the process of creating and maintaining content to enable training professionals to be more independent and efficient in managing their training program.

During this session, we will present our perspective on the key characteristics that we believe must be addressed in order to provide a meaningful capability for instructors and curriculum designers to be self-sufficient in the creation and maintenance of training content as part of their immersive learning programs.

A Primer: No-Code Content Authoring Tools to develop Fully-Immersive, 6DoF Experiences

In this talk, we will highlight our work in collaborative augmented reality (AR) CAD visualization and procedure assistance with ProtoSpace. Being used at JPL for several years both on the web and with HoloLens, ProtoSpace has proven to be useful in various scenarios ranging from early design, procedure planning and execution, as well as post-launch troubleshooting. This talk will showcase our work with AR procedure execution with the Cold Atom Lab, the undersea habitat NEEMO, as well as most recently the VITAL ventilator project.

Collaborative AR CAD Visualization and Procedure Assistance with ProtoSpace

Imagine all the problems augmented reality (AR) could solve if a simple tool existed that allowed anybody to publish AR experiences within minutes. BUNDLAR is a web based platform that provides a no-code, drag and drop interface for authoring, editing, and publishing AR. Novice users can learn to publish AR in as little as 30 minutes. While simple to use, the BUNDLAR platform provides a powerful set of features that make it easy to publish and access on-demand training, performance support, and instructional materials. This talk will demonstrate - in real time - how to structure, develop, and publish a set of AR based instructions using the BUNDLAR platform.

No-Code Authoring, Editing and Publishing of Augmented Reality to Mobile Devices using BUNDLAR

This talk presents a framework for quantitatively evaluating Augmented Reality and other training solutions for a given training environment based on four factors: Task Factors, Training Media Factors, Return on Investment (ROI) Factors, and Training Efficacy Factors. The framework, based upon the “House of Quality” matrix, helps compare, visualize, and assess user and technical requirements as well as identify research gaps.

Framework for Evaluating Training Efficacy for Augmented Reality

Digital humans, created from mixed reality, volumetric video capture, are fast becoming the next great teaching tool that any global-scale operation can benefit from. Imagine capturing your best technicians, engineers, soldiers and first responders in three-dimensional video to train future generations on new tools, software, maintenance, procedures and more, on-demand and anywhere in the world.

Avatar Dimension aims to drive innovative volumetric captures to the forefront with its new, state-of-the-art studio in Washington D.C. In this talk, Zenk will discuss how the company is focused on bringing this technology to all sectors of government, military, healthcare, private industry and more, including how it works, potential applications, ability to scale and ROI impact. He’ll also discuss how the U.S. government and military are uniquely positioned to benefit from volumetric capture-based training due to its previous investments in Hololens2 and Azure.


Technical Training for Industry and Military Leaps Forward with XR Volumetric Capture

Augmented Reality helps us tell a better story. It erases the line between imagination and the real world and opens the doors for new possibilities. With immersive devices becoming more accessible, the focus is not how we interact with these objects, but rather what it takes to create these objects. Investing too much time in modeling something that already exists in the real world can be a nightmare. With 3D scanning, this process is quick and precise. Discover the benefits of Creaform's metrology grade solutions and how they can be used to speed up asset creation for your AR applications.

Premium content

Unity Computer Vision

Downloads

Next from TEAR 3M 2021

Transition Wide-Angle Goggles with Integrated Illuminated Display

Similar lecture

NIWC Pacific, RESTORE Lab: “3D Localizations and Recognition of Text in Shipboard Point Cloud Data Using Corresponding 2D Images via Virtual Camera”