United States

In recent years, the concept of Digital Twin (DT) has come up to play a massive role in the efficient functioning of various Industry and Manufacturing sectors. Originally designed for the aerospace sector, the benefits of the framework were soon recognised for other arenas. One particularly exciting avenue for this concept is for healthcare where DTs of Human Beings can have applications for the prognosis of muscle injuries and diseases and rehabilitation of patients, while also providing the ability of virtually testing solutions before they are actually implemented. A type of a DT of Human Beings are those of Human Musculo-Skeletal (MSK) systems which require multiple muscle parameters and muscle excitation signals, traditionally found by solving a system identification problem through optimisation. In this work, a framework for the system identification of a DT of a Human MSK system is proposed where the problem is solved using Physics Informed Recurrent Neural Networks (PI-RNNs) to identify the equation parameters and predict the muscle excitation signals, given the recorded motion data and the system of equations governing the rigid body movement of the Human MSK system. PI-RNNs have been formulated by enhancing traditional Recurrent Neural Networks (RNNs) with physics-based constraints such as residuals of the differential equation and/or time integration schemes for differential equations. RNNs are extensively used to model sequential time series data and adding the physics-based residuals as a regularisation term to the loss function allows for a robust system identification procedure while modern machine learning architectures lend a computationally efficient framework to solve for system parameters and signals. The future work in uniting various data streams of the subject including motion capture data (position, velocity, and acceleration) and wearable sensor data (kinetic and kinematic data of the muscle groups) for a more accurate solution and connecting the system identification of the multi-body dynamics of the MSK DT to the patient specific muscle mechanical properties is also highlighted.

NAML 2021

System identification in Digital Twins of Human Musculo-Skeletal systems.

physics informed neural networks

digital twin

machine learning

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

technical paper

Wideband signal detection is often achieved by careful design of signal-specific matched filters, requiring expert knowledge of signal protocols to develop optimal, matched filter design, which is both expensive and time-consuming. Learning a detector for new or unknown signals in a timely manner has important operational implications. Furthermore, the ability of autonomously monitoring a wideband in order to find signals-of-interest (SOI) or anomalous signals can increase operator efficiency and their ability to quickly react to new threats. In this work, we present CLAWS — Control Learning from Anomalies in a Wideband Spectrum — a machine learning system for wideband spectrum awareness and autonomous radio control able to achieve a demonstrated real-time throughput of 1 Gigasample per second. Our system can accurately detect SOIs and anomalies and make intelligent control decisions to maximize probability of intercept given available system resources. The core components of CLAWS are a Deep-Learning (DL) based SOI detector, anomalous signal detector, and a radio controller based on joint utility optimization. The DL detectors are realized with a multi-scale U-Net inspired architecture which consumes wideband complex-valued short-time Fourier transform slices, processes these slices with a combination of complex-valued and real-valued layers, and outputs time/frequency bounding boxes and signal type classification labels for signals of interest. The network is fully-convolutional, thus allowing training and testing to happen at different sample rates. This multi-rate property can significantly speed up the training process by reducing the footprint of the training set both in memory and on disk. The DL detector can be trained without expert signal knowledge and only requires time-frequency boxed examples of signals in I/Q form. In order to enable autonomous radio control, we build a Gaussian Mixture probabilistic model of the environment that serves as a prior distribution of SOI. The joint-utility based radio controller is trained to maximize the probability of intercepting the SOI in the wideband by tuning the radio center frequency and beam angle, trading-off the exploitation of the environment model with random exploration. We present results of both our SOI and anomaly detectors as well as the full CLAWS system using both government provided data as well as over-the-air collects, demonstrating its operational relevance.

Control Learning from Anomalies in a Wideband Spectrum (CLAWS)​

In November of 2017, the Consortium for Robotics and Unmanned Systems Education and Research (CRUSER) at the Naval Postgraduate School (NPS) conducted the Multi-Thread Exercise on San Clemente Island. The experiment provided a platform to test the collaboration of an autonomous network of UxVs with manned assets. The scenario was built around a NSW unit tasked with landing on San Clemente Island and traversing the island to act on a known target. The NSW unit was supported by both manned and unmanned assets operating together to provide ISR support, transportation, and battlespace awareness. Previous work took real world data, the NSW team’s path from their insertion point to their objective, and introduced a Disruption-Tolerant Network into the NCS to loosen the communication constraints and increase mission effectiveness. A simple, novel algorithm was introduced that leveraged distributed submodularity to arrive at near-optimal network topologies in a decentralized manner by utilizing local searches by and limited information sharing between the agents in the network. This algorithm presents a novel, viable method for using distributed submodularity to near-optimally position network agents in real-time. This research analyzes how well an adversary is able to predict an individual UxV future path and the location of a point of interest based on the paths of one or more UxV. It also explores how well an adversary would be able to predict the location of the NSW team at various time steps if the adversary assumed that a ground unit was present and controlling the UxV through some local communication platform. If an adversary would be able to predict the future paths of one or more UxV and the NSW, we investigate how well the adversary also predict the location of the ultimate objective, thereby providing the adversary with advanced warning of a raid or other operation. Similarly, we investigate how an adversary could also use such experimental data for the purposes of identifying tactics, techniques, and procedures of the Naval Surface Warfare units and UxV operations. And finally, we develop machine learning algorithms which can disguise the network’s intent. We use a two-stage approach to predicting UxV and NSW positions. The first stage consists of a time series analysis to develop ARIMA models for the X and Y coordinates of each UxV. Once the ARIMA model was created, we forecast the future positions of each UxV. The second stage of the model uses the predicted UxV locations to model possible NSW locations. We use other analytical methods to determine if the probability of accurately predicting the path or target is within acceptable limits or if adjustments should be made to the network configuration.

Adversarial Machine Learning: Disguising the Intent of Networked Control Systems

Machine learning (ML) applied to radio frequency (RF) data has some unique challenges beyond applying ML to other fields. This is largely due to the nature of the RF domain, where there are very high data rates, dynamics, and a large degree of freedom space for parameters and context that can impact received signals. In naval, and defense applications in general, these are only more exacerbated by adversarial use, competition, disruption, and control of the electromagnetic spectrum and span across RF applications, including for electronic warfare, communications, radar, signal intelligence, etc. These challenges pose obstacles to transition RF ML research into real-world applications, especially within deep learning approaches which are data hungry.
In this talk we will discuss some recent successes in overcoming these challenges to enable emerging RF deep learning research to get closer to operational applications. In particular, we will cover the application of Transfer Learning and Data Augmentation to the RF domain. Transfer learning can use available and synthetic datasets to better learn deep learning features when operational datasets are not available. Similarly, data augmentation can create more robust deep learning networks and enable generalization into new operating conditions not yet experienced or only containing a few exemplars, such as with wartime reserve modes.
We will start with our recent work in RF transfer learning. First we will present results from large scale experiments with various types of pre-trained models as well as re-tuning, showing how to the lack of operational data during training can be overcome in many cases and can be mitigated to have graceful degradation in others. Next we will cover how to build an unsupervised model from synthetic data to learn lower level features that can transfer to a real-world classification task, demonstrating near identical results to the supervised training case of learning directly on the real-world operational dataset. Finally, we will discuss how to better learn features during training to handle the presence of new, unknown emitters that can be present during operation and guard against misclassifications and spoofing.
We will conclude with how data augmentation can enable successful transitions to real-world applications by surveying the successes of other domains, discuss initial work in RF data augmentation, and lay out a path forward. We will review work in image data augmentation to understand how to add variance to the environment and in the data capture process. We also review work from text augmentation to understand how to add variance to the source generation process. We then look at some initial RF data augmentation and layout what we believe the path forward is, highlighting areas where there are gaps that, when filled, can help get us closer to operational use of RF ML.


How to Make Radio Frequency Machine Learning Operational

NSWC Corona and NSWC Dahlgren present a new approach to streamline missile safety documentation for PEO IWS 3H into an informative search application for the Program Management and the Safety Community to use. To the best of our knowledge, this is the first safety traceability pipeline which is a corpus-wide intelligent search engine beyond exact matching to query strings.

The safety traceability pipeline is composed of the following modules: corpus ingestion, document metadata extraction, requirements checklist generation, document similarity computation, and a Neo4j Graph Database construction. Corpora consisting of .txt, PDF, or XML documents are ingested into the traceability pipeline from which TextRank, regular expressions, and Latent Dirichlet Allocation (LDA) generate metadata such as keywords, dates, references, and topics. Each document’s requirements checklist is generated by applying parse rules on Parts-of-Speech tags and Dependency Parse tags. The document-level metadata in conjunction with the requirements checklist is used to construct a Neo4j Graph Database, where each node corresponds to a document with the respective metadata, and each directed edge corresponds to a document reference. The weight of each directed edge is the document similarity score computed from cosine distance of Term Frequency – Inverse Document Frequency (TF-IDF) vectors of the connected node-pair.

Given a query, the pipeline traverses through the Neo4j Graph Database to extract the associated requirements, organizations and people’s roles and responsibilities, and the relevant documents. These capabilities enable users to access the relevant information with minimal noisy and redundant information, and significantly reduce human oversights, human interpretation errors, and effort. Furthermore, the continual development of the pipeline allows users to verify that paperwork is compliant with current safety requirements relevant to their programs.
The objective of this application is to aid the safety community and program management by providing an intuitive search engine, curated lists of salient information including to-do and requirements lists, to check program compliance with standards, and identify roles and responsibilities of entities mentioned in the documents. Currently, the NSWC Corona application peNLP hosts a search engine and interactive node diagram that is constantly evolving. In the future, this will allow the safety community and program management to check if submitted paperwork follows compliance of all safety requirements relevant to their programs.

NLP Application of Safety Documentation

Artificial intelligence (AI) and machine learning (ML) systems are data intensive—requiring large amounts of data in order to train ML algorithms. Godbole (2020) states that AI/ML-based models are data hungry and the accuracy of the model largely depends on the quality and quantity of the data. As a reference point, it took approximately 40 days including 29 million games to train DeepMind’s AlphaGo Zero system to learn to play the game of Go (Feldman, Dant, Massey, 2019). Now imagine an AI/ML weapons system algorithm that needs to factor in concepts such as war context (war games, cold war, peace time), friend or foe, ethics, and legality (Feldman, Dant, Massey, 2019). The enormous amount of time needed to gather data and model such complex AI/ML systems will only be feasible if the Department of the Navy (DoN) has a central library for data.
There is currently no standardization of data collection across the DoN and no centralized library for data collection to be employed by future DoN AI/ML programs. This Naval Postgraduate School student-led project is taking a systems engineering approach to analyze the data needs of DoN AI/ML programs in order to develop requirements and a conceptual design of a Central AI Library (CAIL) system to support future DoN digital readiness for AI/ML systems.
The creation of a centralized library to access data points across the DoN could initiate the cultivation of data in order to provide future AI/ML programs with the quantity of data needed for consumption. The organic, real world data sets that would reside in the central library would provide a solid foundation to support future AI/ML algorithms. The library would have to be extensible to handle legacy needs and adaptable to support new requirements. The library would have to be able to load and query data simultaneously without degrading performance with the capacity to securely share and discover shared data.
Data gathered at the lowest levels could be used to build higher level concepts and complex structures to support AI/ML systems. The application of these data for DoN AI/ML programs would include domains such as weapon systems, maintenance, training, and security. These domains have the same barrier to entry that exist in the DoN for AI/ML programs; accessing data points that have been aggregated over time. If these AI/ML programs could select the best-case scenarios generated from the centralized library, these programs would be able to proceed to real world testing validated from test scenarios that were correctly predicted based on real event testing. Establishing data requirements early allows for information to be collected for future use. Additionally, if the Navy isn’t careful with the data that feeds AI/ML, the output could be biased, skewed, or wrong. The data collection methods, standards, and integrity of the data are being explored as part of this NPS project. 


Data Management for Artificial Intelligence and Machine Learning Implementation Across the Department of the Navy

Given the growth of adversary capabilities, learned algorithms need to be trained to offer tactically complex recommendations that can be statistically analyzed to ensure mission success, warfighter acceptance and naval superiority. This research investigated Event-Verb-Event (EVE) ontological constructs to address this training need. Two projects were involved: (1) Battle Readiness Engagement Management (BREM) decision aid (currently used at the Naval Postgraduate School for training algorithms) based on NWC documentation, and (2) an autonomous package delivery service modeled at the training data level. The EVE construct mapped complex mixes of tactical actions to statistical results for use as training data for supervised and reinforced algorithm applications. BREM research involved 21 EVE classes/categories with over a hundred attributes. Each class/category was modeled using defined attributes which were simulated for statistical analysis. The autonomous delivery service involved five classes and 17 attributes. From both projects, Naïve Bayes, Logistic Regression, Random Forest, XG Boost, k-Nearest Neighbor, Q-Learning (using reinforced learning) and a mini-max algorithm feeding a non-linear optimization routine were investigated. When describing the first or second event (E) within the EVE construct, an event is modeled as status from a sensor, communication link or human input. An EVE attributes/features and consumes no simulation time. Status can convey a measurement of mission success, e.g., package delivered within an upper and lower bound confidence interval of time. The verb (V) within the EVE construct represents the core of the action, consuming simulation time and always initiated by a predecessor (or preceding) event defined by its input attributes. Successor events represent a measurable mission outcome. If one or more predecessor events are reported, then an action will occur. If an action occurs, then one or more successor events are reported. A successor event becomes a predecessor event for the next EVE in the sequence of tactics. By using conditional probability dependencies to cause statistical independence between predecessor and successor events, meta-model outputs from simulations can train algorithms. Conditions for statistical independence are based on modeling status defined by attributes for an EVE/ML class. Attributes within the model represent the state or predecessor event. The meta-model output of the EVE constructed simulation, combined with the input attributes, completes the synthetic training set for supervised and reinforced learning algorithms. Because the meta-model output is a measurement or statistical analysis of measurements, the algorithm can learn statistical output, supporting user understanding of the odds of mission success from the ML result. The results of the research demonstrate how EVEs can model complex mixes of tactics represented as ML class, their related attributes and mission success statistics. It also showed that by defining tactical sequences/threads using EVE constructs, the developer can use simulation output to train algorithms that identify complex courses of action based on the highest likelihood of success.

Using Event-Verb-Event (EVE) Constructs to Train Algorithms to Recommend a Complex Mix of Tactical Actions that can be Statistically Analyzed

Naval Ordnance Safety and Security Activity (NOSSA) determined a need to ensure quality and quantity of training data is sufficient for systems deploying Artificial Intelligence (AI)/Machine Learning (ML) Algorithms. Acquiring quality and quantity of training sets becomes increasing difficulty when limited to synthetic data. Research in Generative Adversarial Networks (GANs) applications focuses on synthetic generation of images using neural networks. Inspired by this approach, this research describes the use of generative, discriminative constructs within a modeling and simulation environment. The generator would model and simulate sparse and missing data supporting “realistic” operational conditions – creating training artifacts. When the discriminator approves the artifacts, the algorithm is ready to be trained on synthetic situations involved with sensor malfunction, message corruption and human input errors. Results demonstrate effective training of a variety of ML algorithms to handle external and internal inputs, e.g., sensor tracks, communication links or human input, all non-image related artifacts. We developed an autonomous package delivery simulation to measure reliable delivery time. Using generative, discriminative constructs, Naïve Bayes, Logistic Regression, Random Forest, XG Boost, k-Nearest Neighbor were trained to support selection of autonomous systems and the most reliable route. The generative construct consisted of randomly generated input variables for the simulation, e.g., speed, distance, weather, surface quality. Availability of status sources were randomly selected to represent missing and sparse data issues that the ML algorithm might encounter during an operational deployment. For the discriminator, our research developed two measurement techniques. The Training Set Alignment Test determined the quality of the training set, i.e., representing operational realism, and the Simulation to Attribute Ratios for nth (StAR-n) Order verified if there was sufficient training data quantity, i.e., attribute variation modeled. Data sets were generated, i.e., modeled and simulated, to target a specific ML algorithm for training. StAR-n showed that the total number of artifacts did not determine sufficient quantity for algorithm training. Instead, quantity determination was based on the number of attribute types. TSAT ensured accurate mapping of requirements to attribute types, therefore supporting the quality of the model and related results for training. Both measurements showed that quality and quantity were tightly coupled to meet operational deployment needs. If the quality and quantity of the entire training data did not meet acceptance criteria defined in both measurements, then the entire training set was determined as fake. The training data was then discarded and a new training set was generated for discriminator review using the two measurements. We were able to minimize generative, discriminative cycle times by shaping the random generator selecting attributes for model simulation to more accurately replicate deployment issues involving status availability to the algorithm, e.g., sensor malfunctions, communication link failures and human input error. The generative, discriminative constructs using simulates and measurements ensure that the quality and quantity of data sets can train ML algorithms to adequately perform in operational conditions when status availability is uncertain.

Applying Generative Adversarial Network constructs to Mission-based Simulations to produce “Realistic” Synthetic Training Data for Machine Learning Algorithms

Casey Sapp with VRTUL (www.VRTUL.co) will discuss his ROV client’s recent successes using VRTUL’s AR/VR teleoperations platform to reduce the burden on operators. By utilizing a combination of Unreal Engine and AR/VR headsets VRTUL is rapidly enabling ROV pilots to perform complex tasks at full ocean depth, and concurrently building photorealistic synthetic simulations to better train ROVs to perform autonomous tasks on their own. Leverett Bezanson with Seebyte will highlight some of the software architecture involved for high speed RL, simulation, and an AI/ML roadmap to fully unsupervised ROV autonomy.

VRTUL and SeeByte have partnered on two SBIRs for the USN dealing with AI and ML as it applies to maritime robotics command and control. Specifically, those SBIRs are “N192-099 3D Visualization Capability for Fleet Remotely Operated Vehicles (ROVs)” and “N202-133 Multimodal Interaction Technologies to Support Small Unit Leaders”. This talk will discuss the transition of these two efforts to the fleet and how they complement each other through the use of AI and ML. 

Casey Sapp is CEO of VRTUL (Virtual Reality Technology Underwater Limited) focused on underwater AR/VR technology solutions. VRTUL's clients include the Navy, BBC, OceanX, National Geographic, Monterey Bay Aquarium Research Institute and many others in need of high performing underwater multi-camera arrays.

Leverett Bezanson is the Engineering manager for the SeeByte US office. Leverett Leads all of the U.S. based engineering efforts. SeeByte is involved in many ongoing Office of Naval Research, JIEDDO, and Naval Sea and Air Systems Command projects, as well as the US Navy EOD MK 18 Family of Systems Program of Record. SeeByte also has commercial presence in the offshore oil and gas industry and has achieved some significant milestones and firsts in autonomous platform operations, pipeline, and cable tracking. The US office is responsible for much of the development, testing and integration of cutting-edge Autonomy, AI, and ML software for the maritime industry.

ROV VR and High Speed Reinforcement Learning

The Georgia Tech Research Institute has been supporting the JAIC Joint Logistics virtual sensing National Mission Inititive called VSTAR. Through the development of the VSTAR digital twin MBSE and engagement with DoD transition partners, the development team has assembled a discussion on AI/ML transition best practices. The presentation will give a brief VSTAR background for context, why a digital twin virtual sensor was developed for the HH-60G, and the critical lessons learned from development through transition to successfully deliver a new capability to the warfighter.

Requirements Considerations for AI/ML Digital Twin Model Transition

The outbreak of COVID-19 on the USS Theodore Roosevelt in March of 2020 presented the Navy with a challenge: manage a novel disease outbreak with limited resources. In response, Defense Digital Service deployed “My Symptoms”, a system in which sailors reported a variety of symptoms to assess their need for immediate medical intervention. The Cloud and Analysis Support during COVID-19 Response (CASCR) group at the Naval Information Warfare Center (NIWC) Pacific supported the deployment of “My Symptoms” for multiple sites while collecting, visualizing, and analyzing the data; this lead to actionable insights for the care of quarantined sailors. The CASCR group later applied machine learning techniques to the symptom data to and showed that data collected for triage purposes could be used to determine if a sailor was a symptomatic carrier of SARS-CoV-2.

Premium content

System identification in Digital Twins of Human Musculo-Skeletal systems.

Downloads

Next from NAML 2021

Control Learning from Anomalies in a Wideband Spectrum (CLAWS)

Similar lecture

Applying a Decision Framework to Increase Machine Learning Impact on the V-22 Osprey Program

Premium content

System identification in Digital Twins of Human Musculo-Skeletal systems.

Next from NAML 2021

Control Learning from Anomalies in a Wideband Spectrum (CLAWS)​

Similar lecture

Applying a Decision Framework to Increase Machine Learning Impact on the V-22 Osprey Program

Control Learning from Anomalies in a Wideband Spectrum (CLAWS)