United States

Throughout the Fleet, operators are faced with complex tasks that require collaborative interactions between heterogeneous teams and software systems. This is especially embodied by Naval planners working through the Naval Planning Process (NPP). Although recent work has focused on modernizing the way this process unfolds through the introduction of digital systems like the Advanced Multi-echelon Planning Tool (AMPT), we believe that empowering planners with more natural ways to interact with tools like this will result in better user experiences and more streamlined workflows to bring them from initial concept to the final planning artifact. Achieving this requires the concurrent development of machine learning (ML) enabled processes such as automated speech recognition (ASR), user intent recognition, information extraction and synthesis, and in-context learning. Ultimately our goal brings all these pieces together such that Naval planners can employ the unstructured domain specific language they use with one another to directly interface with the planning systems needed to complete their jobs.

NAML 2025

Natural Language Interfaces for Naval Planners

llms

planning

The purpose of the AAAI conference is to promote research in artificial intelligence (AI) and scientific exchange among AI researchers, practitioners, scientists, and engineers in affiliated disciplines. 

AAAI 2022

technical paper

### Welcome to the 9th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
<br>Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations
<br><br>

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

The 9th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.

Mnemosyne is an advanced Retrieval Augmented Generation (RAG) system inspired by the Greek goddess of memory, who learned something new every day and never forgot, ultimately becoming the most knowledgeable of all the Titans. This system embodies continual learning and knowledge accumulation, forming the foundation of its architecture.
Mnemosyne employs a Mixture of Pipelines (MOP) approach, derived from mixture-of-models architectures, to address diverse question types. Specialized pipelines handle general knowledge queries, quantitative (counting) questions, temporal analyses (summarizing time-based data and trends), and comprehensive summaries (synthesizing entire corpora).
To enhance performance, Mnemosyne integrates several intelligent agents. The RAG Fusion Agent generates alternative perspective questions that enrich responses. The Pipeline Routing Agent directs queries to the appropriate pipeline, while the Rank Shuffling Agent increases result diversity by introducing random rank changes. A Validation Agent ensures response accuracy by cross-checking summaries with source documents using alternative models.
Advanced prompt engineering techniques from the latest literature further refine the system’s capabilities. Chain of Thought reasoning improves response accuracy by allowing intermediate steps, Agent-Selected kNN Few-Shot Learning selects relevant examples from curated libraries, and answer-choice shuffled ensembling improves robustness through aggregated outputs. Mnemosyne also employs Reciprocal Rank Fusion to combine and rank the search matches of multiple pipelines, questions, and rank shuffles, promoting high-quality responses.
A key feature is Mnemosyne’s modularity and flexibility, allowing interaction with multiple models and databases to accommodate varying compute and storage capacities. This flexibility allows the use of state-of-the-art models or deployment of lightweight models for edge applications.
An optional knowledge graph integration enhances functionality by using a graph as the vector store. This enables JSON-based proximity representations to visualize relationships around matched entities and graph-based ranking refinements to improve entity matches. Mnemosyne can query graph databases like Neo4j, generating Cypher queries to extract and analyze relevant relationships.
Mnemosyne represents a significant advancement in enterprise-grade RAG systems by integrating multiple agents, innovative prompt engineering, and optional knowledge graph capabilities. Its adaptable architecture transforms data into actionable insights while continuously refining knowledge, embodying the memory and wisdom of its mythological namesake.


Mnemosyne: Architecting an Enterprise-Grade Retrieval Augmented Generation Pipeline with Mixture of Pipelines

Air combat mission planning is a complex, time-consuming process, taking anywhere from six to twenty-four hours, depending on the complexity of the mission. Generative artificial intelligence, and large language models in particular, have the potential to radically accelerate the mission planning process (Goecks & Waytowich, 2024). Starting with the commander’s operation order, the mission planner and their team reviews all the stated objectives, mission goals, and operational constraints. Battlefield assets are accounted for and plotted on map for a common operating picture. The planners then work together to develop two courses of action, the most likely and most dangerous. These COAs are then wargamed to highlight any shortcomings that need addressing. Once fully developed and analyzed, the COAs can then be executed. Here we present our initial work on Generative AI Mission Engineering (or GAME), for streamlining combat mission planning using LLMs such as ChatGPT(OpenAI, 2023) in places where its capacity to manipulate text can best be exploited. Our human-in-the-loop approach contrasts with others that try to get the LLM to develop the courses of action directly (Goecks & Waytowich, 2024).

Generative AI Mission Engineering: A Conventional Approach

The Navy faces increasing challenges in detecting suspicious maritime behavior due to complexities in integrating diverse data sources and distinguishing genuine threats from routine activities. NextGen Federal Systems proposes a robust solution that uniquely integrates advanced weather forecasting and simulation capabilities into Knowledge Graph technology—a key differentiator enhancing maritime behavior analysis. Leveraging our experience developing high-resolution weather modeling solutions for programs like the Joint Simulation Environment (JSE) Weather Services and the Army Weather Operational Effects (WxOE) initiative, we address weather-related anomalies that often produce false positives.

By integrating data from sources such as AIS, SAR, EO imagery, RF signals, ship specifications, port logs, insurance databases, and sanctions lists, our approach provides a comprehensive understanding of ship behaviors. This multimodal data fusion enables analysts to identify patterns and anomalies with greater accuracy. Examples include disruptions in vessel observability due to adverse weather conditions or ship course deviations to avoid hazardous environments. By leveraging authoritative data like NOAA’s GOES satellite imagery and synthetic weather services, we ensure analysts can differentiate genuine threats from routine or weather-driven behaviors.

To enhance usability, we propose integrating Generative AI with Retrieval-Augmented Generation (RAG) to streamline querying and analysis. RAG enables analysts to pull relevant insights directly from the Knowledge Graph, providing contextual and actionable information in real time. Building on our expertise from the Synthetic Natural Battle Environment Engine (SyNBEE) SBIR project, this capability allows analysts to interact with the Knowledge Graph using natural language prompts. Our experience with simulation tools like Ansys STK enables us to model complex behaviors, such as analyzing RF signal-to-noise ratios (SNR) through atmospheric conditions to detect likely spoofing. Combined with our multimodal data integration, these tools help analysts accurately identify and model suspicious behaviors.

Further reinforcing our solution, our development of BIFROST—a modern, high-performance, one-stop web portal for the Air Force Weather Enterprise (AFWE)—showcases our ability to deliver user-centered design. By employing a Human-Centered Design (HCD) process and leveraging DevSecOps pipelines for rapid deployment, we ensure our solutions meet operational requirements at the speed of need.

Our solution aligns with the Navy's goals of applying machine learning for tactical platform data and advancing AI human factors. By fusing multimodal data, environmental modeling, simulation expertise, and user-centric design, we offer a scalable, impactful tool for maritime domain awareness, ensuring operational efficiency and decision superiority.


NextGen's Multimodal Fusion: GenAI with Environmental and Contextual Modeling for Maritime Domain Awareness

Underwater environments present a significant challenge for imaging systems due to the dynamic interplay of light attenuation, scattering, and absorption. These factors severely degrade visibility and signal quality, complicating critical tasks such as object detection, recognition, and tracking. The turbid nature of underwater scenes further exacerbates these challenges, as light propagation along the line-of-sight (LOS) between the transmitter and detector is highly susceptible to environmental perturbations. These limitations are particularly critical in naval applications, where robust imaging systems are essential for operations such as surveillance, navigation, and search-and-rescue missions.
To address these challenges, we have developed a novel imaging system that combines multipolarized single-frequency illumination and single-photon detection. Our experimental setup includes lasers transmitting 532nm light in four distinct polarization states. To process the photon returns, we designed and engineered a meta-surface nanostructure attached to our imaging setup. This nanostructure spatially splits incoming photons into four distinct polarization channels captured by our single photon detectors, producing a total of 16 unique polarization-resolved images per capture. This modality allows us to systematically analyze how each polarization state interacts with underwater environments characterized by varying levels of turbidity.
We show that different polarization states can accentuate specific target features while mitigating the effects of scattering and absorption. By isolating and leveraging the most effective polarization states, we enhance the quality of acquired images, enabling downstream tasks such as object detection and recognition to operate more effectively. This approach not only improves image clarity but also provides a foundation for optimizing imaging systems under diverse environmental conditions.
To further refine our understanding, we employ information-theoretic approaches to quantify the information content provided by each polarization state. This analysis helps discern the unique contributions of each channel, offering deeper insights into the interaction between light and underwater environments.
The integration of multipolarized imaging and single-photon detection represents a scalable framework for adaptive underwater imaging systems. These systems have significant implications for naval applications, offering enhanced object detection and recognition capabilities that can improve situational awareness in operations ranging from mine detection to underwater robotics. By addressing the limitations of traditional imaging methods, this work paves the way for advanced optical systems capable of operating robustly in the most challenging underwater conditions.


Multi-polarization Underwater Object Recognition with Metasurfaces-based Lidar Imagery

One of the problems with NN-based object detectors is a problem endemic to deep learning-based object detection models in general. Deep learning puts an enormous amount of trust in the training set to represent all configurations detectable objects within the imagery to be captured will take. Our contention here is that while NN-based object detectors’ ability to find objects is extremely valuable, NN-based object detectors tends to be “brittle” in the sense that when an object gets obscured by some means or changes its configuration beyond the scope of the confines of the training set, the NN-based object detectors-generated object detection bounding box disappears; thus, generating a false negative in current and (possibly) subsequent video frames. We assert that this phenomenon can be overcome by means of inserting an “adjudicator” based on elastic shape analysis (ESA). Our adjudicator algorithm, which we name LISA or Light Informed Shape Analysis operates by capturing and scoring the shape of the object contained in a prospective region of interest using only the present illumination on the object. This reliance on nothing more than current illumination in our scene has the effect of dismissing the need for a large and comprehensive training set. LISA creates an object detection score of the object’s illumination-defined shape in a prospective ROI by means of aggregating the detected object’s illumination-defined shapes that neural network-based object detectors previously captured. This aggregation occurs by using a weighted Karcher mean, and the scoring occurs by using the Path Straightening Algorithm (PSA). Combining LISA with YOLO leads to SOLO or Shapes Only Look Once and ultimately serves to enhance YOLO’s inference step. We provide evidence of SOLO’s false-negative mitigating ability for YOLO by considering a series of examples depicting objects that are partially concealed and operating in challenging video acquisition circumstances.

Mitigating False Negatives in Neural Network Object Detectors through Light Informed Shape Analysis (LISA)

The presentation will discuss a notional framework for the application of generative AI in addressing human machine teaming being used in guiding ONR investments in the Science of AI Applied Research Portfolio. The extension of the framework to address a challenge problem posed by RADM Rothenhaus (Chief of Naval Research - CNR) to with decisin making of Commanders of Carrier Strike Grouos (CCSGs) is introduced, as well as a proposed FY27 Innovative Naval Prototype (INP) program that responds to the CNR challenge. Decision Driven Command and Control (D2C2) is introduced with an invitation to discuss Science and Technology problems implied by the INP.

Generative AI for Human Machine Teaming (HMT) Applications

In this talk we will introduce the notion of topological nodes in neural networks, focusing specifically on the initial applications of a category called circular nodes. Circular nodes can be implemented as node pairs with shared constraints allowing one to encode angular information—like signal phase—within the network. We will present a brief discussion of the theory behind these nodes and present a novel implementation of these compatible with existing deep learning frameworks. Using this new implementation, we recreate examples from the original paper and apply circular nodes to toy classification tasks. Finally, we will demonstrate initial success capturing phase changes in radio frequency signals and conclude with a discussion of future use cases.

Topological Nodes and their Applications

With the recent increase of autonomous and unmanned systems, decision making is increasingly getting pushed to the edge to improve the accuracy and reduce the latency of the observe, orient, decide and act (OODA) loop. Effective and safe autonomy can only be achieved when edge sensors have high quality representations of the environment, and deep learning is an important ML approach for distilling sensor information into useful insights. Getting the best possible performance on these devices is essential, especially in Denied, Degraded, Intermittent, and Limited (DDIL) environments where a central node cannot be reached to process complex information. However, deploying to legacy, compute limited or size, weight and power (SWaP) constrained devices in edge environments makes it a challenge to leverage cutting edge developments in deep learning. Frequently these solutions rely on scaling both in terms of data and number of parameters to make performance improvements. Numerous compression techniques exist to reduce the runtime, power usage, and memory requirements of neural networks, but there are not standard methods and tools to easily compare these metrics on common edge devices. To understand the impact of these techniques on a diverse set of SWaP and performance metrics, we developed a suite of bare-metal, device agnostic measurement tools for quantized and pruned machine learning models. Using this toolkit, we tested the impact of compression techniques against baseline models in a variety of settings: on AWS cloud VM instances, local workstations, and the Nvidia Jetson Orin. Results from these experiments showed the impact of model compression techniques depended heavily on hardware used both for the compression process and deployment, even when applying techniques to pretrained models. A few techniques were broadly effective, including the use of mixed precision and model pruning when supported by hardware, showing significant decreases in latency and memory with minimal impact on accuracy. However, most importantly, we found an overall understanding of model architecture and application goals is key to effective compression without the loss of overall task performance, even when tools are available for effective comparison between models, techniques, and hardware impact.

Lean and Mean: Evaluating Compression Techniques for SWaP-Conscious Deep Learning

Panel Discussion: MAPT drivers: symbiotic relationships between the defense and commercial sectors

Microelectronics and Advanced Packaging Technologies (MAPT) is a critical multidisciplinary field with the potential to transform the design and manufacture of future microchips. These advances build upon breakthroughs in advanced packaging, 3D monolithic and 2.5D/3D heterogeneous integration, electronic design automation, nanoscale manufacturing, and energy-efficient computing. The workshop will present the first 3D semiconductor roadmap to guide the forthcoming microelectronic revolution, like the International Technology Roadmap for Semiconductors (ITRS) has served in the past. The main goal of this workshop is to discuss with the DoD community the topics of the MAPT Roadmap, identify the core shared issues and collect input to the MAPT Roadmap 2.0 edition.

Premium content

Downloads

Next from NAML 2025

Mnemosyne: Architecting an Enterprise-Grade Retrieval Augmented Generation Pipeline with Mixture of Pipelines

Similar lecture

Bounding Quality in Diverse Planning