United States

Thousands of vulnerabilities are found on all NAVWAR systems every year, by providing a way to prioritize which vulnerabilities to patch first based upon severity provides agents with a streamlined path to protect every system. RAVEN has built out a Machine Learning classification algorithm that utilizes Common Vulnerabilities and Exposures (CVEs) descriptions and key features linked to each CVE to predict likelihood of weaponization for each vulnerability. By identifying higher risk CVEs, cybersecurity help desk agents can prioritize patches needed for vulnerabilities on every system, investigate avenues that hackers can utilize to break into across the most vulnerable CVEs, and analyze the vulnerability landscape across the fleet. 

Before ships are ready for deployment, they must go through different event windows to ensure systems are up to date and working together with no issues. Reoccurring issues between different event windows on the same ship or system is common. The Continuity of Issues project was created to link reoccurring issues as they move across event windows. By creating a record linkage framework and Natural Language Processing techniques to look at every ticket’s discrepancy description we can match tickets across 5 disparate data sources. By enabling this viewpoint of common issues we can demonstrate the timeline of ACF installs compared to number of discrepancies within event windows, track reoccurring topics and issues over multiple time periods, and track average trend of Casualty Reports (CASREPS). 

PMW/ISEA teams are dealing with thousands of trouble tickets on a daily basis. The CASREP Prediction Model was developed to predict which Trouble Tickets are most likely to escalate into a CASREP or will take longer to resolve. This allows immediate identification of high-risk issues as soon as the trouble ticket is submitted. Enables team to prioritize response actions and resources, and potentially reduce system downtime. CASREP Prediction Model uses NAVWAR Help Desk ITSM data and five different AI/ML algorithms to work together to boost accuracy. 


NAML 2024

RAVEN Machine Learning Solutions for Trouble Ticket Support and Cybersecurity

common vulnerabilities and exposures

trouble ticket

record linkage

cybersecurity

machine learning

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=580 height=700 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

NAML 2021

The annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


poster

### Welcome to the 8th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific<br><br>

**[More about NAML 2024](https://sites.google.com/go.spawar.navy.mil/naml/)**
<br><br>

Please log in to explore this workshop.

The 8th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


Typically, effective object detection in images requires thousands or tens of thousands of labeled instances. Crowd-sourced data has been used to train object detectors for commercial purposes, but the DoD cannot use this same approach to train neural networks with supervised learning. Either there is an insufficient number of instances of DOD-relevant data, or curating and labeling that many images is too cost and time prohibitive to be of use. ​

For this reason, few-shot object detection can be used to limit training data to tens or hundreds of instances rather than thousands upon thousands. Specifically, our experiments adapt few-shot object detection to the Naval domain. We set up two experiments where in the first experiment, we use a large, common dataset to train a base network followed by training with a small sub-group of novel classes of data; and, in the second experiment, we test another few-shot network, DE-Vit, which can use few-shot methods to find new objects in an image. ​

There are many methods that can be used to train neural networks with fewer data. These experiments are conducted as a part of the SAVANT project—which uses an ensemble of semi-supervised methods to build upon datasets. It will serve as a test bed for semi-supervised methods while creating some much-needed labeled Naval data.

Adapting Few-Shot Learning To The Naval Domain

Naval aviation assets on aircraft carriers such as aircraft, catapults and arresting gear are tracked through information systems that provide situational awareness to sailors. The current concept of operations requires highly skilled workstation operators to manage everything from aircraft status to flight schedules to keep information up to date. Operators receive information over radio communication channels from flight deck personnel. This process requires the operator to be dedicated to listening to updates and manually entering what they hear into workstations. This project developed an automated speech recognition prototype that can automatically listen into the audio communication pipeline and add information into computer systems.  
A speech recognition pipeline was created to recognize the standard phrases spoken by aircraft carrier flight deck personnel. Outputs can be used to correctly identify the actions that need to be performed to update computer systems automatically. The pipeline includes speech to text Automatic Speech Recognition (ASR) models and speech enhancement followed by post processing and similarity matching techniques to increase accuracy. Recursion was implemented to re-run the pipeline with a larger speech model if the result does not match a standard flight deck phrase. The team made use of Whisper, a transformer-based model with various model sizes ranging from 39 million to 1.56 billion trained parameters. The pipeline was tested in a reverberation chamber using flight deck phrases under various noise levels and model sizes. 
The team is currently working on fine-tuning the model and creating synthetic data. Data generation used by the team include the ability to fabricate semi-realistic speech in 108 varying voices and dialects. The team has tested fine-tuning efforts on a hold-out percentage of the dataset but has plans to continue fine-tuning the whisper model on both real-world and synthetic data to note results on increased model performance. While high-decibel noisy recordings were obtained from a laboratory, the researchers hope to obtain more real-world data and ultimately test the system either in a simulated environment or the flight deck environment. 


Aircraft Carrier Flight Deck Information Updates using Speech Recognition 

Continuing the effort towards data-driven RF-channel estimation from last year, this project moves towards using vector-quantized variational auto-encoders (VQ-VAEs). VQ-VAEs have shown to be able to create high-fidelity images with more ease and control than generative adversarial networks (GANs). Using simulated GNU Radio datasets from RadioML, short samples of IQ data with various digital modulations and SNRs were used to train a conditional VQ-VAE-2 with a PixelSnail network to generate the codes. The VAE-generated samples show promise of reproducing the GNU Radio simulated channel effects when compared to the training data, SNR calculations, and modulation recognition classifier results. We look to use OpenAI’s Jukebox architecture to expand these short, discrete samples in order to model longer term channel effects. With this, we hope to be able to expand this to the complex environment of high frequency (HF) data.

RF Channel Estimation with VQ-VAEs

We introduce the Spatial Auditory Network Dataset for spatialized sound event detection (SED) research. This dataset was created using the Spatial Auditory Network Dataset Synthesis (SANDS) toolset, which provides a technique for rapidly synthesizing auditory network data using modern-day 3D game development libraries. A combination of the Unity game engine and the Steam audio plugin are used to virtualize sound propagation using ray-traced multi-path sound propagation that simulates occlusion, reflection, transmission, scattering, absorption, reverberation, and Doppler effects. The dataset consists of 10,000 realizations of up to five out of eight possible sound classes moving through a residential neighborhood environment.

Spatial Auditory Network Dataset Synthesis (SANDS)

Modulation recognition of radio frequency signals has become a canonical example of radio frequency machine learning (RFML). State-of-the-art approaches employ convolutional neural networks applied to input examples batched into a fixed set of timesteps and run on GPUs. For power limited or edge applications, emerging neuromorphic hardware promises enhanced flexibility and energy savings for RFML. However, straightforward transforms of CNNs onto neuromorphic processors result in networks with too many neurons relative to current hardware. To address this, we developed a transform which takes a trained artificial neural network operating on batched inputs to a neuromorphic-ready network operating on streaming data. This streaming transform trades neuron count for synapses, making it possible to fit larger neural networks onto current neuromorphic processors. We discuss the current status and future plans, as the project is ongoing.

Streaming Modulation Recognition for Neuromorphic RF Applications

The objectives of this project are to apply an automated semi-supervised natural language processing model to parse the unstructured text columns contained in the VIDS/MAF maintenance forms. Specifically, an NER model was trained to identify components and failure modes identified in these columns. The outputs from this/these model(s) would be directly used in SE Reliability Control Board processes to identify root cause issues with the most problematic equipment. This was previously a manual process, but this effort aims to automate some of the leg work required by the PMA and IPT teams.
The project used entirely unlabeled data and an automated process was created to label multiple entities of interest contained within the text data. This sped up the process of creating fairly accurate data to train the NER model on compared to a manual labeling effort. The stop words, failure modes, and components were also stored in separate files to be used in updating future models.
Models learned entirely from the raw work order data including all typos, naval lexicon, abbreviations, etc. The only modification of the unstructured text columns was that they were concatenated into a single column and stored as lowercase characters. Some records were excluded from being used as training data in order to remove noise from the final output and abide by stakeholder objectives.
The models learned from a single general dataset and performed well on larger datasets. Smaller datasets required some fine-tuning on the components and failure modes in order to better separate unique components which are not generally seen. This process is manual but can be quickly done using a bag of words and lookup table approach on the most common words. Given most of the words already have an associated label, this eliminates most words that we need to update.
Post-processing was also required to clean the output of the model into usable results, specifically the identification of failure modes and components associated to a maintenance work order. Priority on the unstructured text columns was used to decide which components and failure modes were most relevant to the work order. Similarity scores were also used to group together misspelled tokens and similar tokens that could be rolled into a single group. The final product includes source data with the component and failure mode labeled and visualizations of the most problematic components and the most frequently occurring failure modes. The option to drilldown into more detailed visualizations is also available as needed.


Support Equipment Failure Mode Identification – Name Entity Recognition (NER)

The Marine Corps Systems Command (MCSC) Project Officers (POs) deliver and sustain Command, Control, Communication, Computer, Intelligence, Surveillance, and Reconnaissance (C4ISR) systems for the Fleet Marine Forces (FMF). As part of that process, the MCSC PO must ensure the systems meet the technical and operational requirements of the FMF. This effort involves the creation of Test & Evaluation (T&E) artifacts. This T&E process is done independently for each system and results in the verification and validation of thousands of requirements using clustering and machine learning techniques.

System Requirements Clustering with Machine Learning 

he SeaVision maritime awareness tool provides an unclassified common operating picture for the U.S. Navy and many partner nations. It was developed at Naval Information Warfare Center (NIWC) Pacific and the U.S. Department of Transportation, Volpe Center. SeaVision has a complex interface, and training materials span many hundreds of pages. This makes learning to use SeaVision a difficult task, especially for our foreign partners. 

We used open-source LLMs to develop a chatbot that can answer questions about how to use the SeaVision tool. With this chatbot, new users can ask a question and quickly learn how to use any of SeaVision's many features. We provide the LLM context by inserting text excerpts from training documents and transcripts of training videos. Our system's architecture combines both a retrieval augmented generation (RAG) system and a custom memory module. Our RAG system uses a FAISS vector database to store the training files as embedded passages; cosine similarity was used as our metric for retrieval. The custom memory module keeps track of past questions, answers and contexts. When memory is limited, the module favors preserving the newest question and answer pairs while discarding the oldest context available. The memory module also acts as a context gatekeeper by stopping duplicate contexts from being fed into the prompt. 

Although question answering is important to being able to operate SeaVision, it is only a small part of the larger picture. We plan to expand the functionality of this chatbot to perform atomic tasks carried out by maritime domain analysts such as getting the current locations of ships, the paths ships have followed through time, port call histories, inspection histories, warnings about potentially unsafe or insecure shipping, and much more with the goal of combining multiple atomic tasks into single common tasks. We plan to demonstrate how this type of tool can automate tasks and greatly improve Warfighter performance. Many other U.S. Navy systems have similarly complicated interfaces, and chatbots could improve Warfighter performance for these systems as well.

Training SeaVision Operators using Large Languages Models

In many Navy-relevant applications, dedicated datasets for machine learning are noisy in nature, small in quantity, and challenging to obtain. This work uses applications of transfer learning to exploit small noisy datasets for machine learning tasks. Our focus is on electroencephalogram (EEG) neural datasets for human in the loop optimization of warfighter performance. Specifically, we apply transfer learning to classification tasks on small noisy datasets which leverage larger, publicly available laboratory setting EEG datasets on similar but not identical task domains. We find that transfer learning from a large publicly available motor imagery dataset improves performance and reduces the amount of data required for cognitive state classification of participants learning to walk with an exoskeleton dataset (data source: Army Research Lab). Further, we are exploring the impact of transfer learning on real-time EEG state classification for user interface (UI) development. Following a similar approach as above, we will first build a transfer-learning model by harnessing laboratory-collected EEG data during cognitive workload tasks, and then apply it beta-testers as they interact with UI prototypes. Together, these examples highlight the potential impact that transfer learning can have on Navy-relevant applications by taking advantage of data sources collected in privileged non-Naval environments.

Transfer learning for human in the loop optimization of warfighter performance

You don't need to understand how to fly a plane or design a plane to understand what aviation can do for war. The same thing with artificial intelligence.
This talk walks you through strategic alignment, how the Marine Corps is thinking about implementing AI across the Corps and in support of both Marine Corps operations in the broader naval mission. 

Marine Corps is very good at fighting. But if you look at how we fought for the last 50 years compared to where we could be in the future, we certainly can fight smart. The ability of MC to pull in data, to pull from different sources, from different venues and things, apply analytics, and help us to decision advantage across a range of what we do. Whether that is at the pointy end of the stick, whether that's recruiting, whether that's our internal talent management, whether that's logistics, whether that's supply system,the ability for us to leverage some of these tools is not near infinite, but we have an enormous amount to do and we want to overall be able to fight smart. But right now we're not and this is our path to how we're gonna get to that to take into account this the emerging technology where we think this not only where AI is today, but where we think AI is going over the next few years and then in the beyond into the 2030.

Premium content

Next from NAML 2024

Adapting Few-Shot Learning To The Naval Domain

Similar lecture

Using Machine Learning to Predict CASREP Occurrence and Trouble Ticket Duration