United States

Our task is to create an equivalent to “follow me” mode for ships at sea. Allowing a Sailor to launch a small quadcopter and forget about it while it follows and circles the ship of interest could be incredibly useful. Our particular application is autonomous inspection and battle damage assessment, say in the aftermath of a Naval casualty. 

We develop a computer vision pipeline that uses a drone’s camera feed to track the behavior of a ship. It uses camera frames to estimate the ship’s position and orientation to potentially correct the drone’s trajectory, ensuring that the ship is always in view, even in adverse conditions. Our proposed method can also be effective in broader applications in fields like Autonomous Driving and Robotics, where the main agent takes an action based on its surrounding environment. 

The first stage of our computer vision pipeline consists of calculating the orientation of the ship. Our method passes the images from the camera through a UNet image segmentation model to get the ship’s mask and boundaries. We then use an OpenCV algorithm to generate a rotated rectangle that minimizes its area while enclosing the entire mask of the ship. This rotated bounding box effectively represents the direction that the ship is facing based on its physical features, so we use its angle to estimate the ship’s orientation with respect to the drone. This technique is beneficial because it is less complex than 3D Pose Estimation models, allowing for faster real-time processing of image data. 

For our image segmentation model to perform effectively in various environmental conditions, it is crucial to have a variety of flight paths, elevation angles, and visibility conditions to prevent overfitting. Collecting real data with these criteria is an expensive and laborious process, so we have started with Unity-generated synthetic data with relevant variations as ground truth labels for training. Using this data to train our computer vision model resulted in highly accurate predicted masks from unseen flight test images.

NAML 2024

Estimating Ship Position and Orientation using Deep Neural Networks and Synthetic Data

orientation estimation

image segmentation.

object tracking

pose estimation

synthetic data

computer vision

deep learning

### Welcome to the 9th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
 Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

NAML 2025

The 9th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.

poster

### Welcome to the 8th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific 

**[More about NAML 2024](https://sites.google.com/go.spawar.navy.mil/naml/)**

Please log in to explore this workshop.

The 8th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


While Federated Learning (FL) on a distributed network is used extensively in commercial applications, the safety-critical nature of Naval applications presents a unique set of challenges. Specifically, the method must be robust against dynamic network conditions, where devices and communication channels within the network may experience unpredictable changes during both training and inference. Vertical Federated Learning (VFL) is a class of FL in which each client within a distributed network shares a common sample space but possesses only a subset of the overall features. This setup is particularly pertinent in practical scenarios such as sensor networks deployed for surveillance, where each sensor (client) gathers specific data (features) about the same environment (sample space). While VFL addresses several privacy concerns inherent in distributed learning, it typically operates under the assumption of ideal hardware and seamless communication capability. This assumption is a major limiting factor in the broader application of VFL, especially in the context of edge devices. Edge devices are inherently heterogeneous in their operational capabilities, and may frequently connect or disconnect from the network, leading to a dynamic and unpredictable network environment.

To address this gap, we study the test time performance of VFL under dynamic network conditions, which we call DN-VFL. We first formalize DN-VFL, including a message passing distributed inference algorithm, the corresponding risk, and a serverless setup. We develop a novel DN-VFL approach that synthesizes replication, gossiping, and selective feature omission that improves performance significantly over baselines. Furthermore, we implement new methods, propose metrics, and extensively analyze results based on a simulated sensor network. The results show that naively using VFL for DN-VFL is not the best approach. Rather, the proposed methods present a better alternative to handle changes in the network during
inference.


Fault-Tolerant Vertical Federated Learning on Dynamic Networks

SysML (Systems Modeling Language) is a widely-used formalism and international standard for designing and communicating complex system designs, and is especially relied upon in the development of safety-critical autonomous systems. SysML v2 is a major new version of SysML currently undergoing finalization, scheduled to be released mid-2024. Among many key enhancements, SysML v2 introduces a formal semantics for specifying the precise meaning of SysML models, which makes it possible to apply formal verification and automated reasoning to analyze safety and correctness properties of SysML designs automatically.

At Imandra, we are applying our Imandra Automated Reasoning System to the design and analysis of SysML v2 models. Key to this work is a formal semantics for SysML v2 models expressed in Imandra's logic, and corresponding verification automation for answering deep questions about the possible behaviors of SysML models. In addition to verification, we can then leverage Imandra's "proof-carrying prose" integration with LLMs to obtain conversational assistants (chatbots) which can rigorously answer complex questions about SysML v2 designs, backing every answer up with an independently verifiable logical audit trail produced by Imandra reasoning about the SysML v2 model.

During this talk, we will demonstrate how to use Automated Reasoning with SysML v2 models to:
- Mathematically prove properties of system correctness.
- Add a hallucination-free natural-language interface (chatbot) to empower more stakeholders to gain insights and leverage the SysML v2 model.

Imandra + SysML: Automated Reasoning for SysML (Systems Modeling Language) v2

The ever-growing size of deep learning models necessitates more extensive resources, making them unsuitable for tasks requiring lower latency and less computational power. This is particularly true for edge devices. Multiple compression techniques have been developed to bring the capabilities of these larger models to a smaller form factor, one of the most successful of which has been knowledge distillation. Knowledge distillation is employed to transfer knowledge from a large "teacher" model to a smaller "student" model. This technique facilitates learning a compressed representation of data, maintaining the performance benefits of large models while reducing computational and memory demands. It involves training the student model on both hard labels and soft labels provided by the teacher model, leading to better generalization and informed predictions.

Our work increases the effectiveness of knowledge distillation by augmenting the labels of both the teacher and student models with a vector quantized vector. Vector quantization is a mapping of high-dimensional data to a lower-dimensional codebook, effectively reducing data dimensionality. This technique has traditionally been applied to speech and image compression particularly in Variational Autoencoders (VQ-VAEs), and has proven effective at markedly increasing the effectiveness of student models to learn from teacher model representations.

The work presented here seeks to enhance the classification capabilities of the student model, carefully maintaining its compact size to balance efficiency and compactness. This approach improves retained knowledge distillation performance, while conferring model interpretability benefits. We show that while also increasing student model classification performance, the inclusion of the VQ layer also provides a measure for detecting classification quality and detect data which is out of distribution. This approach promises to increase the capabilities of edge deployed models both with respect to their mission performance as well as out of distribution detection.


Improving Knowledge Distillation with Discrete Feature Vector Representation

My poster presentation explains documented co-symbolic AI decision support and automated security. It includes contracted use and exercise support with US Special Operations Command; US Army Intelligence Command; US Army Program Executive Officer for Simulation, Training, and Instrumentation; and the US Marine Corps Tactical Operation Group. By Light combines a mature multi-domain warfighting knowledge graph symbolic AI (expert system), a budding self-supervising artificial general intelligence (AGI), and a proven faster than real-time wargaming tool. Each of the parts come together to support warfighting decisions while overcoming critical warfighting AI vulnerabilities, not achievable by most AI. These include 1) allowing risk manageability/trust across assurance, attribution, understanding, traceability, and bias; 2) preventing model collapse; 3) eliminating human data annotation bias and expense; 4) affording real-time data tag, label, aggregation, and neural networking with minimal if any latency regardless of schema, structure, or unstructured; 5) surmounting data latency - neural networking 10 petabytes of historical data in less than year; 6) acting as master middleware, applying all data and integrating all apps with low cost of entry - only web capable; 7) learning warfighter user behavior, optimizing and customizing to predominantly push answers instead of users always querying an answer; 8) learning operating environment behavior in real-time and recognizing asymmetry; 9) applying multi-domain warfighter expertise across all battle and warfighting functions; and 10) cyber-securing artificial intelligence automation across all information technology, operational technology, and internet of things infrastructure, because it takes AI to secure AI.
The presentation will include government results from theater and tactical level multi-domain peer-to-peer exercises and video demonstration of the parts. It will challenge the audience to consider the warfighting vulnerabilities present in other AI, such as ChatGPT and introduce a suitable, feasible, secure, trustworthy, and acceptable path for warfighter AI decision support.


Knowledge Enhanced Multi-domain Artificial Intelligence

Naval Asset Corrosion and Defect Segmentation Using UNet Architecture 

Abstract: 

Corrosion poses a significant challenge in the maintenance and upkeep of naval assets, leading to degradation and high cost. Timely and accurate detection of corrosion and other defects are crucial to ensure operational efficiency and system availability. Traditional methods for detection often involve manual inspections, which are time-consuming, prone to human error, and require inspectors with specialized training and expertise. 

There is a significant amount of research into automatic detection of corrosion via computer vision algorithms and semantic segmentation that we utilized to combat this problem. Our method makes use of hand-annotated high-resolution images of existing weathered structures (conex boxes) as inputs for a machine learning model to detect basic corrosion, as well as classify additional defects such as paint blistering and cracking. Due to the limited amount of data representing the extended classes, image augmentations were performed to expand our dataset. 

When classifying “stuff”, where the defects can be any shape or size, having an extremely high level of precision is necessary. To account for this, a UNet machine learning architecture, which consists of a contracting and symmetric expanding path, was chosen due to its capability of performing precise pixel-level image segmentation with fewer training data requirements. We have experimented with multiple variations of UNet and implemented additional modules like attention and deep supervision in order to increase the overall performance and generalization of the model.

Naval Asset Corrosion and Defect Segmentation Using UNet Architecture 

Overhead imaging sensors can positively contribute to Maritime Domain Awareness (MDA) missions. The Naval Center for Space Technology at the Naval Research Laboratory (NRL) continuously investigates the latest technologies for space-based sensing. This poster presentation provides an overview of ongoing Artificial Intelligence (AI) and Machine Learning (ML) projects being conducted by the center. Project topics include Automatic Target Recognition (ATR), automated detection of sensor anomalies, scene simulation, and the associated components (e.g. optics, sensors, processing electronics) that enable these projects. NRL has examined the performance of different AI-based vessel detection models on space-ready electronics. NRL has evaluated AI-based methods to detect sensor anomalies. NRL has stood up a laboratory to develop, test, and validate algorithms using simulated overhead scenes. This poster presentation will provide information on each of these efforts.

Overview of AI/ML projects at Naval Center for Space Technology

The landscape of maritime security is rapidly evolving,
necessitating advanced surveillance and threat detection systems.
We introduce a multimodal framework employing a zero-shot
approach designed for detecting anomalies, novelty, and out-
of-distribution (OOD) events, particularly focusing on warships
and fishing vessels. This system, utilizing a pretrained model for
feature extraction, innovatively integrates high-resolution
satellite imagery, drone videos, and radio frequency data. The
fusion of these diverse data modalities into a unified latent feature
space significantly enhances precision, recall, and accuracy in
anomaly detection, while substantially reducing the noise
associated with individual modalities. This improvement stems
from the richer contextual information provided by the multimodal
fusion.
The framework features a user-centric design, allowing
operators to specify regions of interest (ROI) for monitoring
through an intuitive interface. This approach ensures that
vigilance is focused on critical areas, thus improving the relevance
and precision of threat detection. Within these ROIs, our zero-
shot transformer-based multimodal clustering algorithm
analyzes naval movement patterns, comparing them against
established normative behaviors derived from historical data.
This method efficiently filters anomalies, uncovers novel
behaviors, and enhances OOD detection by flagging significant
deviations from the training labels distribution.
Upon detecting a behavioral change within an ROI, the system
promptly issues alerts to operators. These alerts are enriched
with contextual information from the multimodal data
sources, including visual confirmations from drone footage and
satellite imagery, ensuring operators receive a comprehensive
understanding of the situation for swift and informed decision-
making.
Our proposed framework represents a significant advancement in
maritime security, setting a precedent for adaptable and scalable
surveillance systems across various domains. Its ability to
integrate new data types and sources, combined with its focus on
user-defined areas of interest, renders it a robust and versatile
tool for current and future security challenges. Empowering
operators to monitor specific maritime regions effectively and
leveraging advanced multimodal machine learning techniques for
threat detection, the proposed approach is set to revolutionize
naval surveillance and security.

Precision Maritime Monitoring: Integrated Anomaly, Novelty, and OOD Detection in Designated Naval Areas

Active learning seeks to minimize annotation costs by selecting the most useful training data for deep machine learning models. However, until recently, the most successful active learning algorithms have required a large initial labeled set to work properly. This makes them impractical for labor-intensive labeling tasks such as object detection and modalities such as synthetic aperture radar (SAR). Improvements in extracting useful intrinsic features from unlabeled data through self-supervised learning have reduced the need for large amounts of labeled data. As a result, AL can focus on selecting a smaller subset of the best data to label for supervised learning on downstream tasks. Since annotation costs tend to be higher for these downstream tasks, it is crucial that the AL algorithm perform well under low annotation budgets. In this work, we study feature representations for AL in the low-budget regime specifically. Our AL approach builds upon existing self-supervised representation learning methods by leveraging features learned from pretrained models and applying them to the SAR modality. Using these self-supervised learned features, we employ a diversity-based sampling strategy to select the examples for labeling under a low annotation budget. Qualitative results indicate that self-supervision with datasets well-matched to the target domain, when combined with cluster sampling-based AL algorithms, can lead to the selection of diverse samples.

Representations for Low-Budget Synthetic Aperture Radar (SAR) Active Learning

Many of the current state of the art Intelligent Autonomous Systems (IAS) and robotics systems use Inertial Measurement Units (IMU) for navigation and control. This includes, Boston Dynamics SPOT, Ghost Robotics quadrupeds, Clearpath Warthog, and numerous UAS systems all of which are being tested for military applications. IMU sensor allows for a high-resolution measurements of the orientation, angular velocity, linear velocity of the IAS. These measurements assume a stationary frame reference; this assumption does not hold in a moving reference frame such as an aircraft carrier which is has six degrees of freedom. The moving reference frame of the aircraft carrier will cause measurement errors for IAS systems which can cause their controller and navigation systems to fail. The magnitude of the effect is unknown leaving a large gap in knowledge.

SEA LEGS: Ship Experiments for Autonomous Locomotion and Guidance Euler Systems

The goal of this work is to create a robotic subsystem that is capable of effectively fighting fires in a shipboard environment both autonomously and via teleoperation. The system will be designed as a firefighting attachment for mobile robots incorporated into the fleet in the future. SAFERS will use a COTS motorized firefighting monitor, a variable power aimed water cannon, to target and extinguish fires. The targeting control software will allow for both automatic targeting of the fire and manual targeting control by an operator. Both automatic and manual targeting decisions will be informed by the various cameras that make up the perception system. The perception system will also use the video feeds to identify fires and hotspots within the fires to provide ideal targeting suggestions to most effectively fight fires. A variety of different perception sensors will be explored in order to test their effectiveness for firefighting tasks in a complex environment. Additionally, various learning and classical vision algorithms will be trained and tested to evaluate performance in the tasks of fire and hot spot detection across sensor spectrums. Some of the machine learning methods currently being explored are deep convolutional neural networks and visual transformers for object detection, semantic segmentation, and monocular depth estimation. The detected objects and estimated depths of the environments will be used by the control algorithms to make intelligent targeting decisions during firefighting scenarios. Ideally, the developed system will be useful tool for fleet firefighters as both an autonomous and remote system that will lower risk to sailors while simultaneously improving overall firefighting capability.



Premium content

Next from NAML 2024

Fault-Tolerant Vertical Federated Learning on Dynamic Networks

Similar lecture

Adversary Attacks on Ship Tracking