United States

Dive into the realm of advanced AI with our poster presentation on integrating large language models and RAG architectures using Databricks. Discover how this cutting-edge technology transforms operational intelligence, offering unprecedented capabilities in data analysis, decision-making, and strategic operations. Learn how Databricks, with its robust data processing and AI tools, can harness the full potential of large language models, providing the Navy with a formidable tool in information management and operational efficiency. Join us to explore this synergy of technology and defense, paving the way for smarter, faster, and more secure Naval operations.

NAML 2024

Empowering Military Intelligence with Next-Gen AI: Unleash the Power of Databricks and Large Language Models

generativeai

predictive maintenance

machine learning

large language models

### Welcome to the 9th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
 Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

NAML 2025

The 9th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.

poster

### Welcome to the 8th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific 

**[More about NAML 2024](https://sites.google.com/go.spawar.navy.mil/naml/)**

Please log in to explore this workshop.

The 8th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


Background: The Millennium Cohort Study (MCS) is a prospective cohort study designed to examine the long-term effects of military service with surveys administered every 3-5 years. Participant non-response to follow-up surveys is a potential threat to the validity and generalizability of study findings. In recent years, predictive analytics has emerged as a promising tool to identify predictors of non-response.

Methods: Among participants enrolled at baseline and met eligibility criteria (n=70,117), we utilized a temporal cross-validation method to predict response to the 2019-2021 MCS follow-up survey and compared prediction performance of six machine learning algorithms. Using Latent Class Analysis (LCA), we classified participants based on historical response to the baseline survey administered in 2001-2003 and follow-up surveys administered in 2004-2006, 2007-2008, 2011-2013, 2014-2016. We compared prediction performance with and without the LCA variable. Feature analysis was subsequently run on the best performing model to identify the most important predictors with and without LCA. 

Results: The LCA variable consisted of six distinct classes, comprised of consistent responders (web, paper, and mode switcher), consistent non-responders (early and late drop-offs), and inconsistent responders. Notably, the LCA classes were found to be statistically significant across various military and demographic characteristics. 

Random forest outperformed all other machine learning algorithms in both analyses, with and without LCA. Without the LCA variable, random forest outperformed the benchmark regression model by 9.3% in area under the curve (AUC) of the receiver operating characteristic (ROC) curve and 7.7% in AUC of precision-recall. Including the LCA variable, all six algorithms performed comparably with one another, with random forest outperforming the benchmark regression model by 2.3% ROC AUC and 1.6% for precision-recall AUC. Moreover, inclusion of the LCA variable increased the ROC AUC for all algorithms by 10% or more, except for random forest and interaction forest which improved by approximately 5% in ROC AUC.

Feature analysis indicated LCA was the most important predictor in the model. In general, participants identified as consistent responders were predicted to respond to follow-up surveys (predicted probability, ŷ > 0.95), while consistent non-responders were predicted to not respond to follow-up surveys (ŷ < -0.84). Predicting response behavior among inconsistent responders appeared to be the most challenging (ŷ = 0.41).

Conclusion: Our findings suggest that historical response patterns can significantly improve the ability to predict participant response to follow-up surveys. Machine learning algorithms may be valuable when historical data are not available. Implementing these methods in longitudinal studies can enhance outreach efforts by strategically targeting participants and ultimately boost survey response and mitigate non-response.

Enhancing Targeted Outreach for Longitudinal Surveys: Predictive Analytics for Participant Response in the Millennium Cohort Study

Recognizing the imperative to address the reliability and transparency issues of
Large Language Models (LLM), this work proposes an LLM maturity model
tailored for text-to-query applications. This maturity model seeks to fill the existing void in evaluating LLMs in such applications by incorporating dimensions
beyond mere correctness or accuracy. Moreover, this work introduces a realworld use case from the law enforcement domain and showcases QueryIQ, an
LLM-powered, domain-specific text-to-query assistant to expedite user workflows and reveal hidden relationship in data.

Harvesting Trustworthy AI for DOD Superiority

Decision tree models form the basis for many machine learning approaches and have several useful properties; however, single-tree applications are limited by the balance between predictive accuracy and generalization. We introduce a new method for increasing the generalized predictive performance of pre-trained decision tree models. Using this method, the structure of the tree is leveraged to derive a set of decision threshold-based weights controlling the relative contribution of prediction values. The final prediction is then the result of an aggregation of these weighted values. This “soft trees” approach is applied to a set of benchmark datasets and is shown to increase r2 performance over standard decision trees. We further explore the introduction of missing or categorical input features, utility in probabilistic classification, and applications to tasks in resource-limited hardware environments.

Improving decision tree generalization using threshold smoothing

An average enterprise may use over 100 applications (even if no one can actually count them all), some redundant, some outside the visibility of a documented enterprise architecture and certainly not all integrated. As a result, users spend hours moving information between applications and ensuring the consistency of that data to the extent of their desire and ability. 
This results in mindless repetition that takes its toll creating stress-related physical conditions costing trillions of dollars and introducing expensive errors into every industry. Data quality is a root cause of many of those errors. At a time when the volume and importance of data in every enterprise is growing at exponential rates, ways to ensure the quality and accuracy of enterprise data are needed; automation is one way which brings almost unbelievable speed to the processing of it.
Automation technologies build the foundation for expanding into cognitive automation with the integration of machine learning, deep learning, sensor technology, and other emerging technologies. Strengthening operational insights with the ability to combine analytics of larger sets of data with automation is an additional benefit of adoption. 
Within the context of digital transformation, automation refers to the methods and technologies used to generate software-based execution of operational processes or tasks within processes to minimize the involvement of humans. “Intelligent automation, also referred to as hyperautomation] is achieved by mimicking the capabilities that knowledge workers use in performing their work activities (e.g., language, vision, execution and thinking & learning).” (“Intelligent Automation,” Bornet, et al.) Multiple technologies support each of the four work activities; there is no single, pre-integrated, all-encompassing technology solution. 
There are many technologies that have been in use for years, such as optical character recognition and programmatic interfaces, that feed into newer automation technologies such as smart workflows, low-code/no-code software development, and robotic process automation. An enterprise automation platform will include many tools, connections, and reusable components that can link applications and systems of record, presenting a more powerful engagement layer to interface with customers and employees. As an enterprise gains in automation maturity, more elements of artificial intelligence, including machine learning and deep learning, can be integrated along with the data collected by biometrics. API management and application management are perhaps the oldest automation techniques and most found within an enterprise architecture. 
This poster breaks down the need for intelligent automation and shares the progression of technologies that are in. Use cases within the Navy mission will be shared along with NIWC successes.


Intelligent Automation; Why and How

HorizonHunt is a cutting-edge GenAI system designed to aggregate and assimilate multifaceted data and render it fully searchable through an intuitive conversational interface. The HorizonHunt data lake and AI multi-modal platform uniquely integrates image, audio, text, PDF, csv, and other data modalities at enterprise scale and speed, enabling end-users to intuitively query within and across datasets using plain English. 

Benefits include: faster time to insight, multi-modal data analysis, domain-specific insights, adaptive learning, and reliable intelligence for Naval operations. 

Prototype datasets include: 
Static Data – 9/11 Commission Report including: PDFs, Text, Audio, Images
Dynamic Data – GDELT including: Text, CSV, Long/Lat 


Looking beyond the obvious – utilizing generative AI and LLMs to investigate relationships within multimodal datasets within a data lake.

The objective of this project is to use machine learning convolutional neural networks (CNN) to classify marine mammal spectrograms. CNNs excel at image recognition and can identify features and patterns in images. Spectrograms of marine mammal acoustics can be treated as images and put into a standardized format to be used as an input into a CNN model for classification, allowing for a more structured representation of the data than raw time series. The Marine Mammal Monitoring (M3R) on Navy ranges has an extensive data set of both labeled and unlabeled recordings, online sources such as the Watkins marine mammal sound data base have a smaller extent of data but more accessible labeled data. By taking advantage of these online data sets a robust CNN model can be developed and applied to a subset of M3R data to make classifications of marine mammals on navy ranges. 


Marine Mammal Classification using CNNs 

Generative AI (GenAI) and Large Language Models (LLMs) have become increasingly prominent in National Security, especially in the Department of the Navy (DON). Navy CIO Jane Rathbun's September 6th memo, "DON Guidance on Generative AI & LLMs," highlights concerns about LLMs producing biased or inaccurate text. Understanding the importance of reducing biased results, SGS data scientists have researched approaches to mitigate “hallucinations.” The first is simple; set LLM temperature to zero. Temperature is a hyper-parameter that impacts output diversity or randomness. The second is also simple, but mundane, time consuming, and difficult to scale. Provide structured text during training. A problem facing the Navy is the majority of this text is unstructured and locked away in Portable Document Format (PDF) format. SGS data scientists have applied a suite of labeling tools to develop a significantly more effective and efficient method for parsing PDFs. This novel method allows us to train a model that adaptively learns how certain publications are structured, and is an approach to enable the DON's use of GenAI.

Navigating the Seas of Uncertainty: Mitigating Risks in Large Language Models (LLMs)

The Office of the Naval Inspector General (NAVINSGEN) independently and objectively inspects, investigates, and inquires into matters of importance to the Department of the Navy. This includes handling hotline complaints received in unpredicted volumes with various priority levels based on the importance and emergency level of the case. To improve the intake process, BAH developed a Machine Learning pipeline to convert complaint forms to structural data, clean text data, and leverage Natural Language Processing (NLP) models to predict and rank the priority level for the IG intake team. The solution is backed by Databricks and lives on Naval data framework, Jupiter on Advana. In finding the optimal NLP model, the team has experimented with various transformers and techniques combined with MLFlows. This solution allows decision makers in NAVINSGEN to speed up the complaint review process by automatically suggesting and flagging critical cases.

NLP Application on Naval IG Hotline using Jupiter Platform

Snowflake Cortex is an intelligent, fully managed service that offers access to industry-leading AI models, LLMs and vector search functionality to enable organizations to quickly analyze data and build AI applications. Snowflake Container Services enables developers to effortlessly deploy, manage and scale custom containerized workloads and models for tasks such as fine-tuning open-source LLMs using secure Snowflake-managed infrastructure with GPU, in partnership with NVIDIA, instances all within the boundary of their Snowflake account.


Obfuscating GenAI Complexities

A new data-adaptive model via machine learning is investigated as a prospective capability for ocean wave height estimation based on simulated ship motion time-series. Long Short-Term Memory (LSTM) neural networks are trained and tested with physics-based Large Amplitude Motion Program (LAMP) simulation data for the ship geometry and loading conditions of the David Taylor Model Basin (DTMB) Model 5415. From this initial study, data-adaptive machine learning appears to be a suitable candidate for continued investigation of estimation of the ocean wave field based on ship motions.

Premium content

Next from NAML 2024

Enhancing Targeted Outreach for Longitudinal Surveys: Predictive Analytics for Participant Response in the Millennium Cohort Study

Similar lecture

Integrating AI at the Edge Today: Transforming Naval Strategies and Operations