United States

The Georgia Tech Research Institute has been supporting the JAIC Joint Logistics virtual sensing National Mission Intititive to develop the VSTAR digital twin capability. VSTAR is a digital twin MBSE for the H-60 4G accelerometer targeted at data reconstruction for vertical acceleration to model, simulation, reconstruct, and data stitch an AI/ML synthetic signal to replace corrupted flight data. This data stream is critical due to post-flight analysis and maintenance planning (RUL, CRTs, etc) being directly based off damage calculations using the flight regimes experienced by the H-60 determined from the 4G accelerometer measurements. The poster discusses the hybrid multi-physics and AI/ML MBSE approach leveraged to develop the VSTAR capability.

NAML 2021

Digital Twin Virtual Sensor Development for H-60 Accelerometer Data Reconstruction

virtual sensing

mbse

predictive maitenance

ai/ml

digital twin

logistics

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

poster

The maritime transportation industry faces a number of logistical challenges, including navigating safely under bridges, reducing costs due to intermodal transfers, and handling various water and weather conditions. In November 2020
alone, maritime safety incidents occurred at the Centreville Turnpike Bridge near Chesapeake, Virginia and the Barataria Bridge near Lafitte, Louisiana. Such incidents incur major economic losses due to the negative impact of the closure of roads and maritime passages in their respective areas. Trabus Technologies (TRABUS) has created the River Information Systems Enterprise (RISE) framework to address many of the logistical challenges faced by mariners along the U.S. inland waterways. At the heart of RISE is an artificial intelligence (AI) based river level prediction system known as the River Prediction Analytics Toolkit (RPAT), that forecasts hourly river heights out to seven days in the future. These predictions are made using deep neural network models by processing historical hydrological readings at over 300 different river gage sites within the U.S. inland waterways and weather observations from nearly 100 nearby Automated Surface Observing System (ASOS) sensor network stations at U.S. airports. River level forecasts provide mariners with predictive analytics that enable them to plan their trips, make optimal decisions on how much cargo to load on barges, account for varying climatic conditions such as drought and early spring snow melt, and safely navigate under bridges. Predictions generated by RPAT are useful in additional maritime decision support tools such as the Bridge Air Gap (BAG) tool. US Army Corps of Engineers (USACE) provides metadata for many bridges along the United States Inland Waterway system. BAG tool calculations are made on the fly by subtracting water level elevation from the bridge low steel elevation. The water level elevation incorporates both a gage datum elevation and the RPAT projected river height. The predictions improve on existing river level forecast methods such as the National Oceanic and Atmospheric Administration (NOAA) river predictions and the National Water Model (NWM) by increasing the reporting frequency and prediction accuracy. Due to the lack of a publicly available repository of bridge air gap recorded data, prediction errors on bridge clearance must be estimated. Bridge low steel elevation and gauge datum elevation are both theoretically constant, so errors in clearance calculations depend solely on RPAT errors. For many sites in the U.S. inland waterways, 7-day RPAT projections have a root mean squared error (RMSE) of 1.5 feet or less and an accuracy range between 75-98%. This means that a typical BAG tool prediction also should have a RMSE of 1.5 feet or less and an accuracy between 75-98%. This presentation discusses the generation of the RPAT models, highlights issues during the development of the BAG tool, and offers suggestions for future work involving bridge navigation, river currents and cargo load optimization.

River Level and Bridge Air Gap Predictions for United States Inland Waterways

A common focus across the DoD is: How can our tools and information help improve fleet readiness? It is evident that Data Science plays an immense role when it comes to decision making and the analysis of issues. The increase of data digitization only furthers these efforts. One avenue of interest is Natural Language Processing (NLP), which is a subset of Artificial Intelligence (AI). NLP focuses on developing models to understand human language. This work focuses on a specific area of NLP called Word Embeddings, which maps words to vectors of real numbers. These numerical representations of words helps enhance modeling tasks such as multiclass classification. Our use-case demonstrates how Word Embeddings were able to enhance classification results of maintenance narratives.

The maintenance narratives are free-form text that the sailors input when completing maintenance actions. The dataset of about 60,000 records is labeled and has six different categories of maintenance. The process is then broken down into two approaches, the classical approach and then the word embedding approach. The classical approach involves the textual data being broken down into tokens and forming a Document Frequency Matrix (DFM). This DFM is then used to build supervised classifiers such as Naive Bayes and Gradient Boosting Machines. This is then compared to the Word Embedding approach, which also tokenizes the textual data, but then uses it to form word vector representations of each token. These vectors are then used to build a supervised classifier that yields better results than the classical approach.

Word Embedding models allows for a deeper understanding of human language as opposed to classical approaches. It is able to absorb more context and syntactic similarity from the text. The corpora of data that is used to train it determines how the language model will behave. In our case, the model is centric to naval maintenance data. However, such work could be applied in various fields where multiclass classification would be beneficial.


Maintenance Classification using Word Vectors

Recent budget constraints require the Naval Aviation Enterprise to maximize readiness while minimizing cost. As such, it is important to know which aircraft are the most expensive to keep in a mission capable (MC) state and what factors affect this cost. An AI/ML model is developed as an extension to the Digital Aviation Readiness Technology Engine (DARTE) family of models to determine the factors that lead to a high average cost per day of MC for an aircraft in the Naval FA-18 fleet. This model then predicts if the average cost per day of MC will be above a fixed cost threshold in the future. Various model architectures are explored, including tree-based ensemble methods and long short-term memory (LSTM) networks. These models are trained on a combination of traditional readiness datasets (manning, training, equipping) and financial datasets. The resulting models are then examined with explainable artificial intelligence (XAI) techniques to obtain a rigorous and comprehensive feature importance which may be used to determine past and predict future aircraft status.

Understanding and Predicting the Future Cost of Readiness for an Aging Naval FA-18 Fleet

The Strike Force Interoperability Office (SFIO) assembles and filters large quantities of data from a number of sources to produce Task Group readiness briefs, providing guidance in assisting mission planners with their recommendations. This process is historically manually intensive and significantly impacts personnel workload and time better spent conducting analysis.

PRIOR, a cloud native application, provides:

Secure and reliable semi-automated data driven analysis which leads to time and cost savings
Flexible microservice architecture and use of Natural Language Processing (NLP) techniques which support the transformation of large quantities of disparate data to assist with mission readiness assessment.


Planning Resources for Interoperability Options and Readiness (PRIOR)

The current state of machine learning and artificial intelligence mirrors where digital technology was 20 years ago; systems are being developed without rigorous testing and with limited (if any) security mechanisms. This lack has created challenges across both the DoD and industry with respect to the amount of trust in the safety, robustness, and performance of intelligent systems, resulting in degraded capabilities and technological utility for the warfighter. We argue for early, often, and specific consideration of security and robustness as part of a comprehensive DevSecOps process – one that incorporates tooling, methodology, and changes to acquisition strategy.

Critical Gaps in Trusted AI

We discuss the Chinese strategy in AI research in development. It is trying many things simultaneously with the weakness of not doing any of the adequately. Much of their development is top-down and they lack a culture of innovation, both of which may impede progress.

China's Artificial-Intelligence Strategy: Human Neural Networks

NAUTICA, a NISE funded workforce development project, intends to become a catalyst in development of AI talent across the Navy. NAUTICA, a loose acronym for "Naval AI Unity Training and Challenge", will offer a NIWC Pacific designed/customized AI/ML curriculum tailored towards our budding DoD AI community throughout FY21 Q2 and Q3. NAUTICA team will provide a series of online/live and recorded training lectures to provide its students in practical AI know-how in Unity ML Agents (includes Reinforcement and Imitation Learning), OpenAI Gym, genetic algorithms, Statistical Forward Planning algorithms, Monte Carlo Tree Search/AlphaZero, NEAT/HyperNEAT, NGTS integration of trained agents, supervised/unsupervised learning, neural networks, explainable AI and more. Weekly exercises will be provided following each of the lectures, as well as technical AI support. Graduates of training will be awarded certificates of completions. 
After conclusion of the series of Unity courses, we will then hold the NAUTICA Challenge for participants to train ML agents to compete within a simple wargame environment during the summer of 2021 (FY21Q4). Certificates are awarded to all participants with tournament performance statistics. Trophies will be awarded to the top three winners. The details about the NAUTICA Challenge are still in planning and early design, but we are considering using the Unity RIDE environment developed by USC ICT, and is available to all DoD government employees and FVEY partners. 


NAUTICA Naval AI Unity Training and Challenge Poster

Text localization and recognition in 3D point cloud environment are challenging tasks. In this paper, we introduce a method to adapt 2D text detection and recognition techniques in panorama images with appropriate map-ping; combined with some heuristic methodologies such as artificial camera to create a fast and efficient 3D text localization and recognition system. The simplistic nature of this system is attractive and computationally inexpensive because it focuses on 2D processing instead of 3D. The system shows promising results
with over 90% accuracy for detection, localization tasks and 95% on recognition task.

Unsupervised Text Recognition And Localization From 3D Point Cloud Via
Feature Clustering In Correspondence 2D Projection

As sensor and algorithmic capabilities improve, understanding the world in 3D is becoming an ever greater focus of computer vision. For the Naval community, there is a growing treasure trove of shipboard data available for developing new capabilities. Several commands are actively scanning ships for purposes of installations and maintenance using Light Distance and Ranging (LiDAR) and photogrammetry. They are collecting geometrically accurate 3D point cloud data sets and associated 2D images. We present some applications of this data and provide an overview of our own research efforts -- which are separately submitted to NAML as individual deeper dives. We remind the community about the existence of this data and our ongoing collection in hopes of inviting further collaboration.

Naval Applications of Shipboard Point Cloud Data

Adversarial examples are inputs to a machine learning system intentionally crafted by an attacker to fool the model into producing an incorrect output. These examples have achieved a great deal of success in several domains such as image recognition, speech recognition and spam detection. In this paper, we study the nature of the adversarial problem in Network Intrusion Detection Systems (NIDS). We focus on the attack perspective, which includes techniques to generate adversarial examples capable of evading a variety of machine learning models. More specifically, we explore the use of evolutionary computation (particle swarm optimization and genetic algorithm) and deep learning (generative adversarial networks) as tools for adversarial example generation. To assess the performance of these algorithms in evading a NIDS, we apply them to two publicly available data sets, namely the NSL-KDD and UNSW-NB15, and we contrast them to a baseline perturbation method: Monte Carlo simulation. The results show that our adversarial example generation techniques cause high misclassification rates in eleven different machine learning models, along with a voting classifier. Our work highlights the vulnerability of machine learning based NIDS in the face of adversarial perturbation.

Premium content

Digital Twin Virtual Sensor Development for H-60 Accelerometer Data Reconstruction

Next from NAML 2021

River Level and Bridge Air Gap Predictions for United States Inland Waterways

Similar lecture

Requirements Considerations for AI/ML Digital Twin Model Transition