United States

We consider the problem faced by an intelligent attacker that wants to discover new attack paths to compromise a network in a cyber-security network intrusion scenario. This problem is challenging because the number of possible actions for the attacker could increase exponentially with the number of nodes in the network and the number of services running on those nodes. We formulate the attacker&#39;s problem as a reinforcement learning task and use a decision transformer network to tackle the complexity of the attacker’s action space. We have evaluated our techniques within a gamified network intrusion detection scenario called Cyber AI Gym Environment (CAGE). Our results show that the proposed approach can enable the attacker to learn successful attack strategies against two difficulty levels of defender strategies.

NAML 2024

Decision Transformer for Learning Intelligent Attacker Strategies in a Network Intrusion Detection Game

game-playing ai

cyber-security

transformer

reinforcement learning

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

We consider the problem faced by an intelligent attacker that wants to discover new attack paths to compromise a network in a cyber-security network intrusion scenario. This problem is challenging because the number of possible actions for the attacker could increase exponentially with the number of nodes in the network and the number of services running on those nodes. We formulate the attacker's problem as a reinforcement learning task and use a decision transformer network to tackle the complexity of the attacker’s action space. We have evaluated our techniques within a gamified network intrusion detection scenario called Cyber AI Gym Environment (CAGE). Our results show that the proposed approach can enable the attacker to learn successful attack strategies against two difficulty levels of defender strategies.

poster

### Welcome to the 8th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific 

**[More about NAML 2024](https://sites.google.com/go.spawar.navy.mil/naml/)**

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this workshop.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

The 8th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


The diverse range of naval vessels, ranging from aircraft carriers to unmanned surface vessels, and infrastructure, such as smart naval mines or buoys, presents a challenge for traditional machine learning (ML) techniques in naval defense. Traditional ML, which centralizes network data, struggles with the vast distances and unstable or insecure communication links typical in naval defense scenarios. 

Federated learning (FL) offers a solution by keeping the data and ML model training local to the naval vessels and only performing occasional ML model combinations, or "global aggregations" at a central server, or "mothership" for naval applications. FL does address some of the heterogeneity inherent to the naval setting, for example, it minimizes the communication burden during deployments through sporadic global ML model aggregations. "Statistical heterogeneity" here refers to data-related differences among network elements, such as various dataset sizes. "Systems heterogeneity" relates to physical differences between naval objects, such as varying reliability of communication links. 

However, existing literature in FL typically assumes the presence of fully labeled datasets across all naval elements, an unrealistic scenario in practical naval deployments where a diverse mixture of both \textbf{labeled} and \textbf{unlabeled} data is common. Current FL-based methodologies tend to produce a uniform global ML model for all naval elements, neglecting the information contained with the unlabeled data, potentially leading to suboptimal global ML models for diverse naval resources. 

To address this, we introduce a decentralized federated domain adaptation methodology which leverages both labeled and unlabeled data to identify the underlying statistical divergences among naval vessels. In doing so, we are able to optimize the transfer of ML models from naval elements with high quality labeled data (called sources) to naval resources with low quality or unlabeled data (called targets), thereby personalizing ML models to better fit the real-time needs of different naval vessels in a deployment. Our methodology, called Source-Target Determination and Link Formation (ST-LF), optimizes both (i) classification of naval resources into sources and targets and (ii) source-target link formation, in a manner that considers the trade-off between ML model accuracy and communication energy expenditures. To obtain a concrete objective function, we derive a measurable generalization error bound that accounts for estimates of source-target hypothesis deviations and divergences between data distributions. The resulting optimization problem is a mixed-integer signomial program, a class of NP-hard problems, for which we develop an algorithm based on successive convex approximations to solve it tractably. Subsequent numerical evaluations of ST-LF demonstrate that it improves classification accuracy and energy efficiency over state-of-the-art baselines.

Envisioning Multi-Source to Multi-Target Federated Domain Adaptation for Naval Defense Scenarios

Our task is to create an equivalent to “follow me” mode for ships at sea. Allowing a Sailor to launch a small quadcopter and forget about it while it follows and circles the ship of interest could be incredibly useful. Our particular application is autonomous inspection and battle damage assessment, say in the aftermath of a Naval casualty. 

We develop a computer vision pipeline that uses a drone’s camera feed to track the behavior of a ship. It uses camera frames to estimate the ship’s position and orientation to potentially correct the drone’s trajectory, ensuring that the ship is always in view, even in adverse conditions. Our proposed method can also be effective in broader applications in fields like Autonomous Driving and Robotics, where the main agent takes an action based on its surrounding environment. 

The first stage of our computer vision pipeline consists of calculating the orientation of the ship. Our method passes the images from the camera through a UNet image segmentation model to get the ship’s mask and boundaries. We then use an OpenCV algorithm to generate a rotated rectangle that minimizes its area while enclosing the entire mask of the ship. This rotated bounding box effectively represents the direction that the ship is facing based on its physical features, so we use its angle to estimate the ship’s orientation with respect to the drone. This technique is beneficial because it is less complex than 3D Pose Estimation models, allowing for faster real-time processing of image data. 

For our image segmentation model to perform effectively in various environmental conditions, it is crucial to have a variety of flight paths, elevation angles, and visibility conditions to prevent overfitting. Collecting real data with these criteria is an expensive and laborious process, so we have started with Unity-generated synthetic data with relevant variations as ground truth labels for training. Using this data to train our computer vision model resulted in highly accurate predicted masks from unseen flight test images.

Estimating Ship Position and Orientation using Deep Neural Networks and Synthetic Data

While Federated Learning (FL) on a distributed network is used extensively in commercial applications, the safety-critical nature of Naval applications presents a unique set of challenges. Specifically, the method must be robust against dynamic network conditions, where devices and communication channels within the network may experience unpredictable changes during both training and inference. Vertical Federated Learning (VFL) is a class of FL in which each client within a distributed network shares a common sample space but possesses only a subset of the overall features. This setup is particularly pertinent in practical scenarios such as sensor networks deployed for surveillance, where each sensor (client) gathers specific data (features) about the same environment (sample space). While VFL addresses several privacy concerns inherent in distributed learning, it typically operates under the assumption of ideal hardware and seamless communication capability. This assumption is a major limiting factor in the broader application of VFL, especially in the context of edge devices. Edge devices are inherently heterogeneous in their operational capabilities, and may frequently connect or disconnect from the network, leading to a dynamic and unpredictable network environment.

To address this gap, we study the test time performance of VFL under dynamic network conditions, which we call DN-VFL. We first formalize DN-VFL, including a message passing distributed inference algorithm, the corresponding risk, and a serverless setup. We develop a novel DN-VFL approach that synthesizes replication, gossiping, and selective feature omission that improves performance significantly over baselines. Furthermore, we implement new methods, propose metrics, and extensively analyze results based on a simulated sensor network. The results show that naively using VFL for DN-VFL is not the best approach. Rather, the proposed methods present a better alternative to handle changes in the network during
inference.


Fault-Tolerant Vertical Federated Learning on Dynamic Networks

SysML (Systems Modeling Language) is a widely-used formalism and international standard for designing and communicating complex system designs, and is especially relied upon in the development of safety-critical autonomous systems. SysML v2 is a major new version of SysML currently undergoing finalization, scheduled to be released mid-2024. Among many key enhancements, SysML v2 introduces a formal semantics for specifying the precise meaning of SysML models, which makes it possible to apply formal verification and automated reasoning to analyze safety and correctness properties of SysML designs automatically.

At Imandra, we are applying our Imandra Automated Reasoning System to the design and analysis of SysML v2 models. Key to this work is a formal semantics for SysML v2 models expressed in Imandra's logic, and corresponding verification automation for answering deep questions about the possible behaviors of SysML models. In addition to verification, we can then leverage Imandra's "proof-carrying prose" integration with LLMs to obtain conversational assistants (chatbots) which can rigorously answer complex questions about SysML v2 designs, backing every answer up with an independently verifiable logical audit trail produced by Imandra reasoning about the SysML v2 model.

During this talk, we will demonstrate how to use Automated Reasoning with SysML v2 models to:
- Mathematically prove properties of system correctness.
- Add a hallucination-free natural-language interface (chatbot) to empower more stakeholders to gain insights and leverage the SysML v2 model.

Imandra + SysML: Automated Reasoning for SysML (Systems Modeling Language) v2

The ever-growing size of deep learning models necessitates more extensive resources, making them unsuitable for tasks requiring lower latency and less computational power. This is particularly true for edge devices. Multiple compression techniques have been developed to bring the capabilities of these larger models to a smaller form factor, one of the most successful of which has been knowledge distillation. Knowledge distillation is employed to transfer knowledge from a large "teacher" model to a smaller "student" model. This technique facilitates learning a compressed representation of data, maintaining the performance benefits of large models while reducing computational and memory demands. It involves training the student model on both hard labels and soft labels provided by the teacher model, leading to better generalization and informed predictions.

Our work increases the effectiveness of knowledge distillation by augmenting the labels of both the teacher and student models with a vector quantized vector. Vector quantization is a mapping of high-dimensional data to a lower-dimensional codebook, effectively reducing data dimensionality. This technique has traditionally been applied to speech and image compression particularly in Variational Autoencoders (VQ-VAEs), and has proven effective at markedly increasing the effectiveness of student models to learn from teacher model representations.

The work presented here seeks to enhance the classification capabilities of the student model, carefully maintaining its compact size to balance efficiency and compactness. This approach improves retained knowledge distillation performance, while conferring model interpretability benefits. We show that while also increasing student model classification performance, the inclusion of the VQ layer also provides a measure for detecting classification quality and detect data which is out of distribution. This approach promises to increase the capabilities of edge deployed models both with respect to their mission performance as well as out of distribution detection.


Improving Knowledge Distillation with Discrete Feature Vector Representation

My poster presentation explains documented co-symbolic AI decision support and automated security. It includes contracted use and exercise support with US Special Operations Command; US Army Intelligence Command; US Army Program Executive Officer for Simulation, Training, and Instrumentation; and the US Marine Corps Tactical Operation Group. By Light combines a mature multi-domain warfighting knowledge graph symbolic AI (expert system), a budding self-supervising artificial general intelligence (AGI), and a proven faster than real-time wargaming tool. Each of the parts come together to support warfighting decisions while overcoming critical warfighting AI vulnerabilities, not achievable by most AI. These include 1) allowing risk manageability/trust across assurance, attribution, understanding, traceability, and bias; 2) preventing model collapse; 3) eliminating human data annotation bias and expense; 4) affording real-time data tag, label, aggregation, and neural networking with minimal if any latency regardless of schema, structure, or unstructured; 5) surmounting data latency - neural networking 10 petabytes of historical data in less than year; 6) acting as master middleware, applying all data and integrating all apps with low cost of entry - only web capable; 7) learning warfighter user behavior, optimizing and customizing to predominantly push answers instead of users always querying an answer; 8) learning operating environment behavior in real-time and recognizing asymmetry; 9) applying multi-domain warfighter expertise across all battle and warfighting functions; and 10) cyber-securing artificial intelligence automation across all information technology, operational technology, and internet of things infrastructure, because it takes AI to secure AI.
The presentation will include government results from theater and tactical level multi-domain peer-to-peer exercises and video demonstration of the parts. It will challenge the audience to consider the warfighting vulnerabilities present in other AI, such as ChatGPT and introduce a suitable, feasible, secure, trustworthy, and acceptable path for warfighter AI decision support.


Knowledge Enhanced Multi-domain Artificial Intelligence

Naval Asset Corrosion and Defect Segmentation Using UNet Architecture 

Abstract: 

Corrosion poses a significant challenge in the maintenance and upkeep of naval assets, leading to degradation and high cost. Timely and accurate detection of corrosion and other defects are crucial to ensure operational efficiency and system availability. Traditional methods for detection often involve manual inspections, which are time-consuming, prone to human error, and require inspectors with specialized training and expertise. 

There is a significant amount of research into automatic detection of corrosion via computer vision algorithms and semantic segmentation that we utilized to combat this problem. Our method makes use of hand-annotated high-resolution images of existing weathered structures (conex boxes) as inputs for a machine learning model to detect basic corrosion, as well as classify additional defects such as paint blistering and cracking. Due to the limited amount of data representing the extended classes, image augmentations were performed to expand our dataset. 

When classifying “stuff”, where the defects can be any shape or size, having an extremely high level of precision is necessary. To account for this, a UNet machine learning architecture, which consists of a contracting and symmetric expanding path, was chosen due to its capability of performing precise pixel-level image segmentation with fewer training data requirements. We have experimented with multiple variations of UNet and implemented additional modules like attention and deep supervision in order to increase the overall performance and generalization of the model.

Naval Asset Corrosion and Defect Segmentation Using UNet Architecture 

Overhead imaging sensors can positively contribute to Maritime Domain Awareness (MDA) missions. The Naval Center for Space Technology at the Naval Research Laboratory (NRL) continuously investigates the latest technologies for space-based sensing. This poster presentation provides an overview of ongoing Artificial Intelligence (AI) and Machine Learning (ML) projects being conducted by the center. Project topics include Automatic Target Recognition (ATR), automated detection of sensor anomalies, scene simulation, and the associated components (e.g. optics, sensors, processing electronics) that enable these projects. NRL has examined the performance of different AI-based vessel detection models on space-ready electronics. NRL has evaluated AI-based methods to detect sensor anomalies. NRL has stood up a laboratory to develop, test, and validate algorithms using simulated overhead scenes. This poster presentation will provide information on each of these efforts.

Overview of AI/ML projects at Naval Center for Space Technology

The landscape of maritime security is rapidly evolving,
necessitating advanced surveillance and threat detection systems.
We introduce a multimodal framework employing a zero-shot
approach designed for detecting anomalies, novelty, and out-
of-distribution (OOD) events, particularly focusing on warships
and fishing vessels. This system, utilizing a pretrained model for
feature extraction, innovatively integrates high-resolution
satellite imagery, drone videos, and radio frequency data. The
fusion of these diverse data modalities into a unified latent feature
space significantly enhances precision, recall, and accuracy in
anomaly detection, while substantially reducing the noise
associated with individual modalities. This improvement stems
from the richer contextual information provided by the multimodal
fusion.
The framework features a user-centric design, allowing
operators to specify regions of interest (ROI) for monitoring
through an intuitive interface. This approach ensures that
vigilance is focused on critical areas, thus improving the relevance
and precision of threat detection. Within these ROIs, our zero-
shot transformer-based multimodal clustering algorithm
analyzes naval movement patterns, comparing them against
established normative behaviors derived from historical data.
This method efficiently filters anomalies, uncovers novel
behaviors, and enhances OOD detection by flagging significant
deviations from the training labels distribution.
Upon detecting a behavioral change within an ROI, the system
promptly issues alerts to operators. These alerts are enriched
with contextual information from the multimodal data
sources, including visual confirmations from drone footage and
satellite imagery, ensuring operators receive a comprehensive
understanding of the situation for swift and informed decision-
making.
Our proposed framework represents a significant advancement in
maritime security, setting a precedent for adaptable and scalable
surveillance systems across various domains. Its ability to
integrate new data types and sources, combined with its focus on
user-defined areas of interest, renders it a robust and versatile
tool for current and future security challenges. Empowering
operators to monitor specific maritime regions effectively and
leveraging advanced multimodal machine learning techniques for
threat detection, the proposed approach is set to revolutionize
naval surveillance and security.

Precision Maritime Monitoring: Integrated Anomaly, Novelty, and OOD Detection in Designated Naval Areas

Active learning seeks to minimize annotation costs by selecting the most useful training data for deep machine learning models. However, until recently, the most successful active learning algorithms have required a large initial labeled set to work properly. This makes them impractical for labor-intensive labeling tasks such as object detection and modalities such as synthetic aperture radar (SAR). Improvements in extracting useful intrinsic features from unlabeled data through self-supervised learning have reduced the need for large amounts of labeled data. As a result, AL can focus on selecting a smaller subset of the best data to label for supervised learning on downstream tasks. Since annotation costs tend to be higher for these downstream tasks, it is crucial that the AL algorithm perform well under low annotation budgets. In this work, we study feature representations for AL in the low-budget regime specifically. Our AL approach builds upon existing self-supervised representation learning methods by leveraging features learned from pretrained models and applying them to the SAR modality. Using these self-supervised learned features, we employ a diversity-based sampling strategy to select the examples for labeling under a low annotation budget. Qualitative results indicate that self-supervision with datasets well-matched to the target domain, when combined with cluster sampling-based AL algorithms, can lead to the selection of diverse samples.

Premium content

Next from NAML 2024

Envisioning Multi-Source to Multi-Target Federated Domain Adaptation for Naval Defense Scenarios

Similar lecture

Behavioral Learning in Security Games: Threat of Multi-Step Manipulative Attacks