United States

Naval War College (NWC) and other leadership facilities provide extensive training on “best” ways to use technology during a single battle and over a prolonged campaign. AI has now added another level of complexity, both in testing and developing decision aids. Test and Evaluation, Verification, Validation (TEVV) and Live Virtual Construct (LVC) operational facilities are charged with testing new technologies to a given set of requirements to ensure performance. The challenge and difference is noted when teaching and then operationally testing autonomous AI-enabled systems. Algorithms are very sensitive to the training data used and lack of alignment to what is taught during a war game, as represented in the training data, vs what is tested at the TEVV/LVC facility can cause AI performance failures during deployment. Additionally, are AI-based decision aids in alignment? Warfighters are being trained to have expectations of performance, but will the AI system be limited in its response? Will the decision aid provide adequate recommendations and has the AI-based system been adequately testing in terms of expectations? The solution offered in this paper is a schema structure that is automated and visually friendly in the approach used to represents an activity ontology, knowledge representation describing cause and effect tactical sequencing. The paper describes the schema used for knowledge representation as well as how it will capture past war game knowledge played by war gaming institutes. Examples are offered to illustrate the approach and convey how this type of ontology developed from war gaming can ensure TEVV/LVC test scripts are designed to operationally “stress test” autonomous AI-enabled systems following these equivalent tactical sequences.

NAML 2022

Semantic methods for activity ontology for testing AI and developing decision aids

wargaming

test

ontology

artificial intelligence

semantic

machine learning

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

To spread awareness of current machine learning research relevant to Naval applications

To connect machine learning researchers with experts in military requirements

To build and strengthen collaborations within the defense research community


<figure class="video_container">
  <iframe src="https://s3.amazonaws.com/pf-user-files-01/u-59356/uploads/2022-03-22/hk13mxo/NAML_ReceptionWithtitle.mp4" width=826  height=526  frameborder="0" allowfullscreen="true"> </iframe>
</figure>

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

To spread awareness of current machine learning research relevant to Naval applications

To connect machine learning researchers with experts in military requirements

To build and strengthen collaborations within the defense research community



poster

Congress recently created middle tier acquisition (MTA) programs, which provide the military services rapid prototyping and fielding pathways with new program flexibilities and an explicit schedule constraint. The services are executing multiple MTAs, resulting in a set of MTA experiments related to development, execution, and governance. There is little published information on MTA performance; we used public data to quantify planned schedules. We introduce a quantified schedule risk measure based on Monte Carlo simulations. The simulations provide insights into MTA program schedule risk and program performance relative to a statistically-based reference.

Estimating Middle Tier Acquisition schedule risk

Training AI agents within a mission relevant simulation will be crucial in providing insights and suggestions to operators and strategic planners. In order to train such agents, the AI engineers should examine how to setup an AI agent to interact with its en-vironment. Representation of the simulation environment is very important to the creation of an AI agent. The state of the world and the reward received will impact the actions the agent will take. The modeling simulation environments must have the flexibility to represent a scenario in a digestible form for the AI agent. We constructed a messaging plugin to the Joint Cognitive Operations Research Environment (JCORE), to connect a Deep Q-Network based agent to a preliminary scenario. The architec-ture used to facilitate the communication between the agent and the simulation will be presented here. The project results demonstrate the ease of integrating AI agents into military modeling environments, provided some requirements are met, illustrating a pipeline process that could be replicated for other DoD projects.

AI Training framework for a medium fidelity environment.

Machine Learning (ML), in particular Deep Learning (DL), has become an essential enabling technology for the coming fourth industrial revolution, which integrates physical, digital, and biological technologies for various innovative cyber-physical-human smart systems. Time-Series Classification (TSC) is a type of problem broadly existing in science, engineering systems, and human activities. In recent years, DL has emerged as a powerful technology for solving TSC problems with many applications, in particular for Human Activity Recognition (HAR). This research presents a DL approach to Human Activity Recognition and Identification (HARI) by detecting an entity’s “behavioral signature” utilizing the signals produced by the accelerometer and gyroscope of an iPhone 6s. The research expands on the previous study by modifying the sample length used in classification along with updating the architecture and hyperparameters for the models. The methods introduced in this research have increased the identification model accuracy from 82% to 90% using a one second sample length. While classifying the entire trial length for each participant the model reaches an accuracy of 97%.

Human Activity Recognition and Identification

This presentation will detail the collaborative effort on building an ML/AI based approach for GPS Spoof Detection between BlueRISC Inc. and Raytheon Missiles and Defense’s Maritime Position, Navigation, a and Timing (PNT) Products.  The bolt-on, ML/AI-based solution operates independently of other Assured PNT (APNT) solutions and integrates with Raytheon systems e.g., GPNTS, GHOST, etc., via the inclusion of a software module.  It is complementary and scalable to other APNT solutions, and is agnostic to sensor manufacturers, supporting various GPS/INS and PNT sensor configurations.  It has been validated utilizing Raytheon provided GPNTS tactical equivalent data sets (Live Sky and Spirent GPS simulation and WSN7 INS simulated data), against sophisticated time walk-off attacks aimed at slowly overcoming an anti-jam antenna by spoofing a single and/or multiple satellites.  The solution has been validated against GPS spoof attacks occurring in the presence of active Jamming attacks as well.  Our ML/AL solution achieves high detection rates for small drifts on the order of 15-80ns, and continues to be beneficial even down to 5ns for added Warfighter situational awareness in contested environments.  Detection processing and response achieved was on the order of several milliseconds, running on a 1.3GHz processor enabling fast and efficient response.  The nature of the solution provides detection capabilities for other, previously unseen and emerging GPS spoof attacks, as well. 
 
About the speaker: Michael Rodriguez, Embedded Security Research Scientist at BlueRISC. Michael was valedictorian in his high school class and went on to earn a BS and MS in electrical engineering and computer science from MIT. In addition to BlueRISC, he worked as a firmware engineer and researcher for various government contractors. During his career he worked in securing sensor system architecture with ML/AI and was the lead embedded engineer for the design and deployment of a sensing system for a DOE nuclear site where he was the key ML/AI architect. He co-led the development effort of the ML/AI-enabled GPS Spoof Identification solution that will be discussed. 
 
About BlueRISC: BlueRISC, Inc., started in 2002, is a cleared small business that works actively with both commercial and DoD customers.  Its focus is on researching and developing solutions in the embedded system assurance space providing detection and resilience capabilities, tools and systems, to cutting-edge attack vectors.  These include software exploitation and reverse-engineering, sensor spoofing, malicious hardware modification (e.g., insertion of hardware Trojans), and others.  Many of BlueRISC’s solutions leverage ML/AI.  The company has 30+ patents and customers in 21 countries.
About Raytheon Missiles and Defense (RMD) Maritime PNT Programs: For the past decade, Raytheon Missiles and Defense has been supporting the Navy (NAVWAR PMW/A 170) in architecting and developing an Open System Architecture (OSA), Modular System Approach (MOSA) compliant, Centralized Assured PNT Data Fusion and Distribution System to support all Maritime Surface Platforms.   RMD is the OEM of the Global Positioning System (GPS)-based Positioning, Navigation, and Timing Service (GPNTS) for Systems, Software, and Hardware; and is currently in the second year of a 10-year GPNTS Software Support Contract.

Bolt-on ML/AI GPS Spoof Detection Capability

Today our data is protected by encryption software tools and libraries such as the secure sockets layer (SSL), which of course, runs on general-purpose processors. While the computing efficiency of encryption may be less concerned than other factors in multi-tenancy cloud environments with high-end server processors, it can be a critical factor in embedded computers where power budgets tightly bound the devices or high-performance computing where both latency and multiplexing computing power is a big concern. Since the inefficiency associated with general-purpose processors is increasingly becoming an issue in the post-Moore era, we study both hardware algorithm and implementation strategy on lightweight cryptographic (LWC) accelerators that can leverage a hardware specialization approach. This paper studied the efficiency of using mixed hardware languages for implementing other hash functions as GAGE with full implementation on FPGA and ASIC technologies.


RECO-LSHFL: Reconfigurable LightweightSponge-Based Hash Functions Hardware Library

The Digital Aviation Readiness Technology Engine (DARTE) Board provides a comprehensive, up-to-date picture of future F/A-18 aircraft readiness at the squadron and buno levels across the Naval Aviation Enterprise. It acts as a visual user interface in the form of Qlik dashboards for the Mission Capable (MC) status forecast powered by machine learning in ADVANA Databricks. This tool uses models created with the DARTE framework to forecast the daily MC status of squadrons and units of a TMS across the entire fleet for the next seven days. Predictions are also interrogated with explainable AI (XAI) so that users can see the leading factors that affect the predictions. The DARTE Board can display this information in multiple functional views, providing the warfighters with a tool to derive data-driven actionable insights into future readiness.

Visualizing the Digital Aviation Readiness Technology Engine (DARTE) with the Advana Platform

Ingest, Inference, Training, Learning, Iterations, and various types of machine learning and AI categories - each require a wide variety of GPU resources, FPGAs, NICs and other devices attached to servers. this variability in devices causes a wide variety of infrastructure challenges that Liqid solves with an elegant solution. LIQID CONVERTS THE IN-SERVER PCIE BUS INTO AN EXTERNALIZED PCIE FABRIC - ENABLING IMPOSSIBLE CONFIGURATIONS LIKE TEN OR TWENTY GPUS BEING CONNECTED TO A SINGLE 1RU SERVER IN ABOUT TEN SECONDS WITHOUT REBOOT OR APPLICATION RESTART. OTHER USE CASES INCLUDE DEVICE VALIDATION, PERFORMANCE, FUNCTIONAL AND REGRESSION TESTING. BEST IN UNKNOWN OR UNPREDICTABLE MULTIVARIANT SCIENTIFIC COMPUTING, AI, ML, DL, HPC AND ANALYTICS ENVIRONMENTS. 




AI/Ml Operational Efficiency, Agility and Density

Digital Twin is a virtual replica of an object, process, or system “twinned” to its counterpart in the real world through data and information feeds. The digital representation can be informed by AI processed data pulled from schematics, computer-aided design (CAD) 3D models, building information models (BIM), and much more. A digital twin can be as simple as a replica of the engine in your car or as complex as multitude of interconnected digital twins representing thousands of linked, enterprise systems.

AI application in Digital Twin

Communicable diseases can compromise the abilities of individual warfighters and the command-and-control and operational capabilities of military environments. Thus, there is a critical need within the military to be able to evaluate an individual’s risk of becoming ill before being sent on a physically difficult operation and to identify outbreaks of communicable diseases early in order to be able to contain, mitigate, and track its spread. This need is especially important when soldiers and personnel have to be kept in close quarters on a ship, in barracks or in other close quarter environments. Wearable health monitoring devices provide the capability to record a variety of real time physiological time-series data including, but not limited to: respiratory rate, respiratory rate variability, heart rate, steps, calories burned and state of mind. Theoretically these physiological measures may be used to predict the health status of individuals over the course of an acute respiratory infection (ARI) from healthy, infected, pre-symptomatic, symptomatic, and recovered. To test this hypothesis, we are using data collected from Spire HealthTag devices that were deployed during a community cohort contact tracing study conducted by the University of Maryland during the 2018-2019 influenza season. The objective of this study is to use the wearable data to predict the onset of an ARI due to viruses such as parainfluenza, coronavirus, adenovirus, influenza, and rhinovirus. With the future goal of pre-symptomatic illness detection in mind, for now the goal is to be able to classify between sick and healthy days. A random forest classification model and its results will be presented. 

Predicting of Health Status from Wearable Physiological Monitoring Data project

Engagement complexity, along with technological advances, is growing faster than shipboard technical upgrades. The conundrum is that because of the challenges of peer-to-peer engagements, upgrades to more powerful computers is needed, but will likely be unavailable in the next several years. Lighter processor power solutions need to be researched to address this issue. Because of the increase in Engagement complexity, users’ needs solutions that fit their current IT while in the operational theater to determine practical recommendations in achieving mission success. This means that a management aids and decision aids need to effectively run at lower computer processing power than needed for complex analytics in order to provide decision support. This research investigated the use of meta-models that can be ported to current IT level processing machines associated with deployed environments. The research focused on approaches that allowed for simulations of advanced, complex engagement issues using meta-models that can be processed to provide the highest likelihood of a successful recommendation that includes platforms following mixed sequences of actions to achieve end goals. Meta-models are the synthetic results of a multiple simulation. A meta-model consists of: (1) the input variables associated with the capabilities of a platform’s weapons, electronic warfare, communication links, and sensors (2) the variables numerical values and (3) statistically analyzed output. When the sequence of events is decomposed into statistically independent activities following a conditional probability construct, the meta-models can be rearranged based on the desired mission outcome. The arrangement of meta-models, related inputs and outputs, constitute a statistically-based sequence of events to obtain desired goals for analysis. In this manner, meta-models, i.e., input variables to output success statistics, are light weight data structures that can be queried and processed within a laptop or equivalent IT system. The research implemented meta-models to support a decision aid software application based on game theory for blue vs red wargaming analysis. It ran Monte Carlo simulations to support recommendations with statistical results. This decision aid is used on laptops at the Naval Postgraduate School for AI/ML experimentation. The tool using meta-models to help resolve engagement complexity using government restricted laptops, demonstrating potential value in the approach. This research shows evidenced that meta-models can be used for “real time” decision aid recommendations when limited to deployed IT environments, but need to be customized to one set of goals. Additionally, recommendations can be accompanied by Monte Carlo-based, statistical outcomes that represent results of complex engagement models running on processing-hindered government laptops.

Premium content

Next from NAML 2022

Estimating Middle Tier Acquisition schedule risk

Similar lecture

White cell automated adjudication for TEVV/LVC facilities in assessing Autonomous technology strengths and weaknesses