United States

Our RetroActivity system automatically builds computational models of a complex physical task, such as a maintenance activity, given only a handful of recorded demonstrations of the task. Once such a model is built, RetroActivity can finely track the job status from live video, to guide a worker through the task, provide independent training, and perform analytics. We enable ordinary AR developers to build AI-mediated feedback within hours, instead of requiring teams of specialized computer vision engineers who build temporal causation rules on top of 3rd-party visual recognition modules over months.

TEAR 3M 2021

AI-Mediated Job Status Tracking in AR

workflow monitoring

computer vision

Brought to you by **National Defense Industrial Association ([NDIA](https://www.google.com/url?q=https%3A%2F%2Fwww.ndia-sd.org%2F&sa=D&sntz=1&usg=AFQjCNHj_BQx1K4KVKKp1NTIsAltZOG9Vg))** and hosted by the **Battlespace Exploitation of Mixed Reality ([BEMR](https://cdn.dvidshub.net/media/video/2101/DOD_108139354/DOD_108139354-1024x576-1769k.mp4)) Lab**, at **Naval Information Warfare Center Pacific ([NIWC PAC](https://www.google.com/url?q=https%3A%2F%2Fwww.public.navy.mil%2Fnavwar%2FNIWC-Pacific%2FPages%2Fdefault.aspx&sa=D&sntz=1&usg=AFQjCNFNS160IEsocfaBrnCNB3u7NqLHPw))**, and welcoming academia, government/DOD, and industry leaders who support Industrial Augmented Reality/Virtual Reality by using the following technologies:   


* Augmented Reality (AR) / Virtual Reality (VR)

* 2D/3D displays

* mobile displays

* wearables

* any other "immersive" technology

### In all fields that touch:

* 3D modeling

* Manufacturing 

* Installations 

* Maintenance and Repairs

* Construction

* Logistics

* Additive manufacturing

* Training

The technical exchange is a two day event held online this year due to COVID-19. It will feature two days of oral presentations, demonstrations, and posters from leaders in AR/VR and other display technologies, including government briefings on active projects within the DOD. The technical exchange is unclassified and we welcome participation from the Navy, DoD as well as from industry and academia. 

The goals of TEAR 3M are:

*  To spread awareness of current immersive technology research relevant to Navy/DOD applications

*  To connect AR researchers with experts in DOD to learn about needs and requirements

*  To build and strengthen collaborations within the DOD research community

![](https://assets.underline.io/uploads/markdown_image/1/image/0c13ac0ffa096b2353573964d8b89789.png)

TEAR 3M General Chairs:

Heidi Buck,  Director @ BEMR Lab, Naval Information Warfare Center, Pacific

Lauren Burnette, BEMR Lab, Naval Information Warfare Center, Pacific


For more information, contact Lauren and Heidi at:  tear3m@spawar.navy.mil

Questions regarding sponsorship/registration contact:  ndiasd@sandiegositepros.com

Conference host POC is Tammy Sanchez (NDIA):  tamsanch@cisco.com

technical paper

Text localization and recognition in 3D
point cloud environment are challenging tasks. In this paper, we introduce
a method to adapt 2D text detection and recognition
techniques in panorama images with appropriate map-
ping; combined with some heuristic methodologies such
as artificial camera to create a fast and efficient 3D
text localization and recognition system. The simplistic nature of this system is attractive and computationally inexpensive because it focuses on 2D processing instead of 3D. The system shows promising results
with over 90% accuracy for detection, localization tasks
and 95% on recognition task.

NIWC Pacific, RESTORE Lab: “3D Localizations and Recognition of Text in Shipboard Point Cloud Data Using Corresponding 2D Images via Virtual Camera”

During this presentation we will expose our return on experience and identify key factors allowing AR application developers to focus on their application use cases and not on implementing the technology to make them work. With Unreal Engine's extensive support for XR platforms and devices, you can use Unreal Engine to deploy your project to any XR platform as well as utilize the immersive devices in your content creation pipeline. Epic Games is one of the founding members of the OpenXR Working Group along with the Khronos Group and industry partners. This multi-company group is working to solve XR fragmentation with a cross-platform standard.

Back to the mission : changing focus from the tech to the operational use cases.

Limited performance of mobile AR and VR devices. Time-consuming cross-platform development. Concerns about data privacy. While immersive technologies are a big driver for global digitization, the XR community faces major challenges that apply to every player in the field. This presentation introduces a new cross-platform approach to XR streaming. A technology that enables an unprecedented level of detail and performance for a faster and safer way of developing, experiencing and interacting with 3D content. Gain useful insights into how remote rendering works and its benefits. Learn about first-hand examples of streaming entire XR applications with ISAR SDK.

Remote Rendering & Real-time Streaming of XR Apps with ISAR

Emerging Augmented Reality (AR) headsets hold great promise for enhancing Soldier effectiveness by improving situational awareness and decision-making in operations and increasing training efficiency through high fidelity training scenarios. Future multidomain operations (MDOs) will present Soldiers with significant quantities of complex and rapidly changing information. How do we develop human-centered AR systems that provide overmatch in MDO when AR technology itself is constantly evolving? Rapid prototyping and simulation of AR capabilities in Virtual Reality (VR) can address this need. Simulating AR capabilities in VR enables full control over hardware and software features coupled with the ability to simulate realistic and Soldier-relevant tasks to assess the human performance impacts of emerging capabilities and design decisions. In this talk I present two such simulation experiments to investigate emerging AR capabilities.

Enabling Human-Centered AR Through VR Human Performance Experimentation

ARRMSS provides the ability for a fleet maintainer to communicate with a Subject Matter Expert (SME) remotely using video/audio/text with the added capabilities of the SME being able to place augmentations/drawings/files onto systems and objects in the maintainer’s field of view to help guide them during operation/maintenance/troubleshooting. Allowing the SME to see in real-time what the maintainer sees can exponentially help reduce system downtime and improve fleet performance.

Augmented Reality Remote Maintenance Support Service (ARRMSS)

Are you struggling with making design | engineering | manufacturing 3D content ready for AR/VR at scale?
This talk presents a solution based on Pixyz and AWS to automatically process CAD data for real-time 3D experiences at scale.

Unlock CAD Data for any Real-Time 3D Experiences at Scale

An introduction to the Air Force Rapid Sustainment Office mission and operating concept. This includes an overview of our infrastructure architecture to support rapid development and fielding of AR/VR capabilities and the guiding values that defines winning technology. Our projects to date have identified targets for AR/VR hardware growth which we would like to collaborate on to overcome.

AR/VR for Air Force Sustainment

An overview of VRgluv ENTERPRISE the world's best force feedback gloves for VR Training. We will provide an overview of why force feedback gloves are critical for VR Training as well as a look at some of the applications for this technology already being worked on with in the Department of Defense.

Haptic Force Feedback Gloves for VR Training and Simulation

Artec poses a great generalist solution for AR and VR applications as well as engineering and bespoke design. Their versatility and ability to scan any size of object makes for a compelling, easy to use scanning ecosystem for digitizing objects of all types.

Artec 3D - A Complete 3D Scanning Ecosystem for all Sizes

State of the art technology, practical effects and magic combine to make the impossible seem real through "hyper reality". What is hyper reality? For decades Curtis has used illusion design to amaze audiences, but when you combine magic with VR... that’s when things get really interesting.

Premium content

AI-Mediated Job Status Tracking in AR

Downloads

Next from TEAR 3M 2021

NIWC Pacific, RESTORE Lab: “3D Localizations and Recognition of Text in Shipboard Point Cloud Data Using Corresponding 2D Images via Virtual Camera”

Similar lecture

NIWC Pacific, RESTORE Lab: “3D Localizations and Recognition of Text in Shipboard Point Cloud Data Using Corresponding 2D Images via Virtual Camera”