United States

The US Navy systems may have unexpected significant cost growth for many reasons. Navy systems may have unexpected significant cost growth for many reasons. The Office of the Chief of Naval Operations (OPNAV) is charged, among other responsibilities, with executing the planning, programming, budgeting, and execution (PPBE) process through a series of concurrent annual planning cycles guided by a Program Objectives Memorandum (POM). The objective of the research is to leverage advanced analytics to help the OPNAV understand the common elements and causes of existing Navy systems that have significant cost growth from historical data, requirements documents, and open-source media. 

This research explores two questions:
1. What are the common elements of requirements that create excessive cost growth in Navy systems?
2. Assuming the elements are identified, what is the risk (likelihood and magnitude) of cost growth from common elements for both procurement and sustainment costs? 

We applied two categories of methods: 1. classic data sciences and business intelligence tools and 2. an artificial general intelligence framework to address the needs and research questions to analyze structured and unstructured data together and correlate them with excessive cost or cost growth of Navy systems. Specifically, we applied LLA, a semantic network analyzer, anomaly detection, and causal learning to discover patterns and deep causes that can lead to high cost or cost growth. 

The OPNAV’s Program Budget Information System (PBIS) has been modernized as an authoritative knowledge system including historical data of planned and executed POM information and spending each year. Data relevant to PBIS include structured data and unstructured data. For example, structured data include number of platforms procured and procurement and sustainment costs for Navy systems. Program elements (PEs) or Budget Exhibits (BEs) contain PPBE information as well as unstructured data of unclassified high-level program descriptions and their elements. 
We analyzed two unclassified data sets provided by the topic sponsors. We found patterns and deep causes for high cost or cost growth programs using lexical link analysis, natural language processing (NLP) tools, a semantic network analyzer, anomaly detection, and causal learning concepts. Programs with anomalous characteristics can lead to high costs or high growth. These tools provide counterfactual and drill-down discovery of the key words that explain the deep causes of cost growth. 

We recommend to apply these tools for the total benefits of analyzing Navy programs and requirements of post mortem data, towards modernizing the PBIS to become a knowledge system that can effectively learn from historical data to make better risk predictions and decisions for the future POMs.

NAML 2023

Structured and Unstructured Data Sciences and Business Intelligence for Analyzing Navy Programs and Requirements of Post Mortem Data

pom program budget information system pbis

program objectives memorandum

lexical link analysis lla named entity extraction nee parts of speech tagging pos spacy semantic network analysis sna centrality measures

unsupervised machine learning

transformers

### Welcome to the 9th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
<br>Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations
<br><br>

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

NAML 2025

The 9th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.

poster

### Welcome to the 7th annual workshop on Naval Applications of Machine Learning!

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific<br><br>

<button name="button" onclick="window.open('https://sites.google.com/go.spawar.navy.mil/naml/','blank')">**NAML 2023 Website**</button>

<br><br><br>

The 7th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


Military operations, particularly in the littorals, have been contested and challenging. It is imperative to develop tools and methods to help tactical units execute distributed operations. Distributed, collaborative and networked agents have been associated with peer-to-peer models. 
The military operation applications require each tactical unit to prioritize requests, recommend content and services to balance the load of a whole peer-to-peer network without the total knowledge and communication with the whole network, therefore, effectively reduce the operation signatures of individual agents and avoid the detection by the adversaries. This objective can not be achieved using the traditional collaborative filters or distributed bandit algorithms. 

We show innovative collaborative learning agents (CLAs) applied to distributed operations. In a distributed operation, each unit or node is represented as a single CLA. A unit can be a knowledge supplier (e.g., capability or service), or knowledge consumer (e.g., a demand or request from its peer units or environment). When a unit receives a new request, it searches its peer network for the best match to fulfil the request, meanwhile the whole network constantly self-organizes and balances the work load of the nodes, lowers the signatures, and avoids detection. By employing CLAs, we first map the need of lowering operation signatures to a load-balancing problem, then apply lexical link analysis (LLA) and principle of quantum entanglement and superposition into a framework of LLA quantum intelligence game (LLAQIG). LLAQIG optimizes the value of an agent itself and in the same time helps a peer network achieve the Nash Equilibrium and attain the optimal total social welfare. We show a use case and data set for distributed transportation units to handle transportation movement requests. We demonstrate that the resulted newly formed peer groups using LLAQIG have the characteristics of smaller load mean range between peer groups, lower load standard deviation within groups, higher number of unique equipment utilized than other methods and the random configuration. These discovered metrics indicate new peer groups are less likely to be detected with less movement of units within peer groups, therefore maintaining lower operation signatures.

The military tactical units can potentially leverage the results for flexible command and control (C2), organizational structures, and modernization. The peer-to-peer and self-organizing DMO and EABO equipped with CLAs allow units, with traditional warfare capabilities of sensors, platforms, networks, weapons, and emerging technologies, optimize their value or load in an autonomous and self-organizing fashion with smaller footprint.


Self-Organizing and Load-balancing via Quantum Intelligence Game for Peer-to-Peer Collaborative Learning Agents and Flexible Organizational Structures

Advanced numerical methods and neural networks are explored as part of a procedure to improve the performance of trained decision trees without requiring access to training, validation, or test data. By using only the information contained within a trained decision tree, a model's performance can be improved by more than 5% over the original trained tree. There are many applications of this technique such as when data may not be retained, where federated learning is employed, or where models are housed on edge devices.

Modeling the Model: Improving Trained Decision Trees Without Access to Data

Icarus is an applied research project aimed at developing a model for deep object detectors from unmanned aerial system (UAS) imagery. We applied edge-based algorithms to a real world task-specific application of passive tracking for littoral unmanned underwater vehicles (UUVs) from the UAS. The detection algorithm runs on the onboard processor on the UAS which also includes flight controls to follow the detected UUV. A two-stage object detection algorithm, Thundernet, was used to maximize throughput and accuracy on small objects.

Icarus

Jupiter, the Naval Enterprise Data and Analytics Environment, enables Sailors, Marines, and Civilians to integrate, share, and exploit data at speed and scale across the Warfighting, Business, and Readiness pillars for operational advantage and increased efficiency. It enables Artifical Intellegence across the Department of Navy.


Jupiter - Use Cases & Capabilities

Accurate and efficient corrosion detection is a difficult but important issue with immediate relevance to maintenance of Naval ships. Current practice requires an inspector to search for defects in a space, however the process requires a time-consuming manual inspection and physical access to the space. Considering both the inspector's and the ship’s schedule, this is not always a straightforward process. 
There is a significant amount of research into automatic detection of corrosion via computer vision algorithms but performing pixel level segmentation introduces added difficulty. There are two key reasons for this: the lack of annotated data and the inherent difficulty in the type of problem. In this work, we utilized a combination of annotated data from a different domain and a small hand labeled dataset of panoramic images from our target domain: the inside of empty ship tanks. 
We trained two High-Resolution Network (HRNet) models for our corrosion detector; the first with a dataset outside our target domain, the second with our hand annotated panoramic tank images. By ensembling our two models, we were able to increase the F1-Score by ~120% and mIOU score by ~176% with respect to the single baseline corrosion detector. 
The data collection process via LiDAR scanning allows the inspection process to be performed remotely. Additionally, the setup of the detector leads to a natural expansion of the corrosion dataset as panoramas are continually fed through the detector and detections are validated. This allows for the corrosion models to be later retrained for potential improvement in accuracy and robustness.

Shipboard Tank Corrosion Segmentation via Deep Learning 

An overview of the NAVWAR Data Science Learning Program discussing course available to all Navy personnel

NAVWAR Digital Transformation Learning Program

Neural networks and ensemble techniques achieve state-of-the-art results but often incur a considerable resource cost. Decision trees, in contrast, require comparatively fewer resources, to the detriment of model performance. To bridge this gap, soft trees are introduced. Soft trees represent a novel variant of decision trees which incorporate additional contextual information into the predictions. Under this methodology, predictions comprise a summation of leaf values weighted by threshold distance and probability. Such predictions provide a significant improvement over the performance of standard trees while retaining trees’ reduced-resource properties. Soft tree models may then allow applications to realize the resource benefits of using a decision tree while also producing improved results within a constrained environment.

Soft Trees: Smoothing Decision Thresholds for Improved Performance

Unsupervised Degrader Ranking

An algorithm that can accurately rank Naval Aviation Support Equipment (SE) top degraders is a desired product to allocate resources effectively. This project utilized Factor Analysis of Mixed Data (FAMD), Isometric Mapping (ISOMAP), and Magnetic Properties and Correlation (URMC) to calculate the ranking of top SE degraders for two different Program/Project Manager (PMA) programs. The current methodology for ranking top degraders relies on a z-scores and Subject Matter Expert (SME) provided weights used by the existing degrader ranking products. The goal of the project would be to optimize the degrader ranking solution to use a method that is less subjective in nature and more flexible, which can be automated and applied across a variety of equipment.

Research to date in this project tackled the main issue by looking into two ways to calculate degrader ranking in an unsupervised fashion: FAMD and ISOMAP. Factor Analysis of Mixed Data reduces the dimensionality of both categorical and numerical data to that of N-dimensions specified by the user before finding a best way to fit all data points on a line in each dimension. ISOMAP, like FAMD, reduces the dimensionality of the data, but in a non-linear methodology, and is only used for numerical variables that are statistically significant. For both algorithms, data obtained from the current degrader products is reduced to a varying range of two to four dimensions while being assigned a degrader percentile label (Rank/Total) to aid in the algorithms’ potencies.

At this point in time, only a FAMD implementation has been fully tested, and with reduction to three components, can explain 84% of the incoming data from the current degrader product’s rankings. Additional exploration and research is needed to evaluate the URMC algorithm’s performance and comparison against both FAMD and ISOMAP models.

Unsupervised Degrader Ranking

Longitudinal data, which is widely used in many disciplines to study cause and effect, poses significant computational challenges to both modeling and analysis. Longitudinal data is composed of readings on the same variable collected over time and is often high-dimensional with correlated features. The combinatorial search approach for identifying the optimal features is unrealistic for most applications. The alternative approaches, such as heuristics, greedy searches, and regularization techniques, including LASSO, can result in models that suffer from both low accuracy and unclear feature attribution. In this presentation, we propose a binary transformation on the data before applying LASSO for feature learning. As demonstrated in the paper, the binary transformation enhances signal in the data, resulting in highly accurate feature attribution, including associated time lags. It avoids the typical shortcomings of the LASSO algorithm, including saturation of the feature space and arbitrary or inconsistent sparse feature selection. Both synthetic data and real-world data sets were used to demonstrate the value of the proposed transformation and in every case substantial improvements in feature learning were seen. In addition, the scalability of the solution is superior to that of the standard LASSO since an integrated high performance computing (HPC) solution would utilize integer math for most of the computational effort.


LASSO Logic Engine: Harnessing the Logic Parsing Capabilities of the LASSO Algorithm for Longitudinal Feature Learning 

GPS has proven to be a valuable, highly accurate Positioning, Navigation, and Timing (PNT) solution in many military applications. Its susceptibility to jamming and interference, however, has sparked a great deal of research in PNT alternatives to address vulnerabilities in GPS-denied environments. In this study, we utilize micro-electro-mechanical system (MEMS) IMU sensors mounted on different parts of the body to estimate movement speed and position. MEMS IMUs are small and light enough to be wearable on a person and can accurately estimate orientation, but fails to accurately estimate position. To address this, we create a system that uses GNSS, motion capture, and inertial data to train a machine learning speed estimator. The estimated speed is paired with a main IMU and an extended Kalman filter to estimate position. We will then compare our results with two other IMU-based navigation techniques: step counting and zero-velocity update algorithm (ZUPT). Step counting works well for normal walking or running, but fails for more complex movements like crawling, strafing, or walking backwards. ZUPT requires high end IMUs mounted on the foot, which can be very cumbersome. We believe our proposed method addresses these problems and performs more accurately.

Premium content

Next from NAML 2023

Self-Organizing and Load-balancing via Quantum Intelligence Game for Peer-to-Peer Collaborative Learning Agents and Flexible Organizational Structures

Similar lecture

Harnessing Spatial AI for Advanced Geospatial Intelligence