United States

&quot;Neuromorphic processing mimics the processing of neural structures in the brain. Neuromorphic computer architectures enable the development of sophisticated and novel capabilities in pattern recognition, event reasoning, resilient decision-making, machine learning, and unsupervised tasking for energy-efficient mobile combat vehicles. Neuromorphic sensors with several benefits are event-based cameras. They feature little latency and low power consumption and are offered as COTS solutions. Spiking neural networks (SNN) can be trained with the event-based datasets that are produced. This paper offers a comprehensive overview of neuromorphic computing, encompassing the latest hardware and software platforms as well as some datasets in the field. This effort is based on a NISE project in which we surveyed neuromorphic computing and SNNs and collected real-time data using DAVIS cameras.
DVS cameras are a type of event-based camera -- asynchronous sensors that sample light based on the scene dynamics, rather than on a clock that has no relation to the viewed scene. Their advantages are: very high temporal resolution and low latency (both in the order of microseconds), very high dynamic range3 (140 dB vs. 60 dB of standard cameras), and low power consumption. Hence, event-based cameras have a large potential for robotics and wearable applications in challenging scenarios for standard cameras, such as where high speed and high dynamic range are critical. Spikes that indicate a change in brightness are returned by DVS cameras, which concentrate on using brightness to identify an event. SNNs and DVS cameras are complementary in that they both handle inputs by default using event-based triggers. Therefore, for algorithmic implementation, DVS cameras and neuromorphic computing hardware may result in a small form factor AI-based sensing paradigm that is both natural and incredibly efficient.
Lava is an open-source SW platform for creating AI apps for Intel Loihi and other neuromorphic hardware designs.  Lava currently supports Intel&#39;s Loihi architecture and conventional CPUs, but its compiler and runtime can be extended to support other architectures. Third-party libraries such as Nengo, ROS, YARP, and others can be added to and modified with Lava. Additional tools enable users to profile workload power and performance, visualize complex networks, and assist with the float to fixed point conversions needed by many low-precision devices, including neuromorphic hardware.
SNNs were trained in the present investigation using dynamic vision sensor (DVS)-based datasets using LAVA and other open-source tools. This is mainly because, once available, LAVA hardware will offer significant performance benefits. We examined MNIST and N-MINST, two widely used opensource datasets that serve as a benchmark for creating Navy-specific proprietary datasets and are used to train SNNs. The use of these datasets for SNN training and implementation was covered in this paper. Information on the SNN MNIST Classifier: Using a multi-layer feed-forward classifier neural network that is easily constructed and uses pre-trained weights from the LAVA repository, without the need for convolutional layers. 2) SNN testing with twenty-five MNIST dataset images 3) A classification accuracy of 88% was obtained. 2) The second portion of the SNN training phase took a long time. By adding a few more open-source spiking neural component primitives to PyTorch, users can use the most recent and stable setup created with PyTorch, as well as spiking neural network components suitable with deep learning.Using these primitives, an SNN was built and trained on the entire N-MNIST dataset. The accuracy and runtime performance were documented during five epochs. The outcomes are extensive and impressive.
Although the aforementioned event-based datasets are publicly available, none of them were designed with dod data—such as drones—in a generic format in mind. Any project that requires a drone data set must be built from the ground up. This might mean not just spending the time and employing the drones to record, but also investing in a DVS camera. In cooperation with the SAUSR range at the Naval Weapons Station, drone movement footage was recorded. We were able to set them up so we could record their plans, which they required tested. &quot;

NAML 2025

Neuromorphic Computing &amp; DVS Cameras

We are pleased to announce the Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24), which will be held in Vancouver, British Columbia at the Vancouver Convention Centre – West Building from 20-27 February, 2024.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-24 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

We expect for AAAI-24 to be an in-person conference – one author of all accepted papers will be expected to present work in person unless there are exceptional circumstances that prevent this.

In order to access the AAAI-24 event page you need to register [here](https://aaai.org/aaai-conference/registration/)

AAAI 2024

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. 

"Neuromorphic processing mimics the processing of neural structures in the brain. Neuromorphic computer architectures enable the development of sophisticated and novel capabilities in pattern recognition, event reasoning, resilient decision-making, machine learning, and unsupervised tasking for energy-efficient mobile combat vehicles. Neuromorphic sensors with several benefits are event-based cameras. They feature little latency and low power consumption and are offered as COTS solutions. Spiking neural networks (SNN) can be trained with the event-based datasets that are produced. This paper offers a comprehensive overview of neuromorphic computing, encompassing the latest hardware and software platforms as well as some datasets in the field. This effort is based on a NISE project in which we surveyed neuromorphic computing and SNNs and collected real-time data using DAVIS cameras.
DVS cameras are a type of event-based camera -- asynchronous sensors that sample light based on the scene dynamics, rather than on a clock that has no relation to the viewed scene. Their advantages are: very high temporal resolution and low latency (both in the order of microseconds), very high dynamic range3 (140 dB vs. 60 dB of standard cameras), and low power consumption. Hence, event-based cameras have a large potential for robotics and wearable applications in challenging scenarios for standard cameras, such as where high speed and high dynamic range are critical. Spikes that indicate a change in brightness are returned by DVS cameras, which concentrate on using brightness to identify an event. SNNs and DVS cameras are complementary in that they both handle inputs by default using event-based triggers. Therefore, for algorithmic implementation, DVS cameras and neuromorphic computing hardware may result in a small form factor AI-based sensing paradigm that is both natural and incredibly efficient.
Lava is an open-source SW platform for creating AI apps for Intel Loihi and other neuromorphic hardware designs.  Lava currently supports Intel's Loihi architecture and conventional CPUs, but its compiler and runtime can be extended to support other architectures. Third-party libraries such as Nengo, ROS, YARP, and others can be added to and modified with Lava. Additional tools enable users to profile workload power and performance, visualize complex networks, and assist with the float to fixed point conversions needed by many low-precision devices, including neuromorphic hardware.
SNNs were trained in the present investigation using dynamic vision sensor (DVS)-based datasets using LAVA and other open-source tools. This is mainly because, once available, LAVA hardware will offer significant performance benefits. We examined MNIST and N-MINST, two widely used opensource datasets that serve as a benchmark for creating Navy-specific proprietary datasets and are used to train SNNs. The use of these datasets for SNN training and implementation was covered in this paper. Information on the SNN MNIST Classifier: Using a multi-layer feed-forward classifier neural network that is easily constructed and uses pre-trained weights from the LAVA repository, without the need for convolutional layers. 2) SNN testing with twenty-five MNIST dataset images 3) A classification accuracy of 88% was obtained. 2) The second portion of the SNN training phase took a long time. By adding a few more open-source spiking neural component primitives to PyTorch, users can use the most recent and stable setup created with PyTorch, as well as spiking neural network components suitable with deep learning.Using these primitives, an SNN was built and trained on the entire N-MNIST dataset. The accuracy and runtime performance were documented during five epochs. The outcomes are extensive and impressive.
Although the aforementioned event-based datasets are publicly available, none of them were designed with dod data—such as drones—in a generic format in mind. Any project that requires a drone data set must be built from the ground up. This might mean not just spending the time and employing the drones to record, but also investing in a DVS camera. In cooperation with the SAUSR range at the Naval Weapons Station, drone movement footage was recorded. We were able to set them up so we could record their plans, which they required tested. "

Neuromorphic Computing & DVS Cameras

poster

### Welcome to the 9th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
 Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

The 9th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.

This project's objective is to analyze how methods of control versus reinforcement can be applied to DC motors that actuate unmanned aerial vehicles (UAVs). In a model-following approach more akin to adaptive control methods, the recursive least-squares method and other least-squares method regression are performed in this endeavor. This is compared to that of the use of reinforcement learning on a deterministic method that is also used to control output to a trajectory. A stochastic method is also an aim when the latter method is accomplished in order to determine how physics-based models perform under different probability-based models in terms of training. Examples include the optimal feedback learning for altering the control signal, in addition to exploring different types of controllers (i.e. proportional, derivative, integral, PID, etc.) for seeing how parameters influence transient outputs and inputs.

DC Motor and Altitude Control for UAVs with Adaptive/Reinforcement Learning + Deterministic AI

Given a heterogenous set of agents, we examine the use of a convolutional neural network to decide which agent should be deployed to perform a set of tasks in an environment with dynamically changing terrain. Our work takes in an initial map of the environment to make predictions of how the environment will change as an agent moves. In our simulations, we designated every cell within a grid as having water, mud, or land terrain. Our agents were specialized as either water-efficient travelers or land-efficient travelers, but all agents could move across any terrain type with corresponding speeds derived from their aptitude for that particular terrain. We used deep reinforcement learning to train the agents to explore a flood environment towards eventually reaching a target. These agents were then evaluated based on the number of steps taken to achieve success for 5,000 different scenarios. In each scenario, only the starting map of the environment and the agent that completed the task in the least number of steps was recorded and compiled into a dataset. A convolutional neural network was then trained on the dataset to predict which agent (water-efficient or land-efficient) should be chosen for a mission. Through evaluation on separate test cases, the model averaged a 90% accuracy in its prediction of which agent to send for a flood-rescue mission.

Exploratory Navigation of Dynamically Changing Flood Environments With Autonomous Delegation of Tasks

Through the application of innovative Naval GenAI foundation, based on historical fleet and vessel maintenance data – including maintenance notes and other unstructured data elements – naval personnel can use machine learning and natural language processing technologies to determine the specific inspection required as well as precise repair procedures and parts required for that necessary repair. Accordingly, the GenAI models will provide instant and data-driven precision diagnosis, along with recommended courses of action including suggested steps for inspections, repairs and swap out or overhauling of equipment. The building of a Navy GenAI model for fleet and vessel maintenance will lead to improvements in maintenance and operational availability as the model has several advantages over traditional schedule-based preventive maintenance methods. Through the amalgamated use of historical data along with maintenance notes, Sailors, maintainers and tech support personnel will be able to learn from past performance related to repairs, automate diagnostics processes, and receive trustworthy guidance regarding the ideal course of action when completing maintenance for fleets and vessels. Additonally, a use case for welding is presented.The Shipbuilders and the DoD Maritime Industrial Base (MIB) experiences delays and bottlenecks from weeks to months, with both Welding and Non-Destructive Testing documentation reviews and approvals. These delays result in 2nd and 3rd orders affecting the downrange in the planning, resourcing, and construction process. To meet the needs of our Nation, IBM has begun development, and will be deploying Minimal Viable Products (MVPs), by leveraging GenAI to deliver to the MIB:
1) User-friendly Procedure Look-up ChatBot
2) AI-Procedural Review Assistant
To accomplish this, IBM designed an innovative Welding Documentation Assistant by ingesting 3ea Technical Publications, along with 1ea MIL-Standard starting the End-to-End Process journey. Though many more Welding Specifications are available (>100ea), a strong foundation for the model at the start was a must. This enables Stakeholders to quickly understand terminology, definitions, quickly look-up the correct specification required based on the Weld DNA, to helping a user write a weld procedure. This is a very complex and time-consuming process, so this assistant will significantly reduce the time and guesswork to get to the correct answer. There will even be time savings for the most experienced with Welding knowledge to find efficiencies. Not to mention all those with less experience or perhaps are new(er) to Navy Standards.


Streamlining the complexity of Welding and Inspection Quality Documentation through GenAI and LLMs.

As part of the Intelligence Advanced Research Projects Activity (IARPA) BRIAR program, Accenture has developed, BLAU, a video-based biometric algorithms to extract accurate and reliable identity recognition and identification under non-cooperative scenarios including long-range, through atmospheric turbulence or from aerial platforms. Leveraging both image-based and temporal-based features, BLAU extracts signatures from the subject’s face, body and gait to build a robust whole-body representation.
With imagery (images or videos) captured of subjects, BLAU first performs body detection to find the individual(s) in the scene. Our body detector is capable of finding subjects that are fully (head-to-toe) and partially (head-to-waist, head-to-knees, etc.) visible. From the image crop of the body, BLAU then performs face detection to find the face. Using the body and face crops, BLAU generates biometric signatures for each modality (face, body, and gait): Face is analyzed with a quality-modulated CNN with adaptive margin loss that has been trained using embedding space augmentation, facial attribute and soft biometric expansion, facial alignment, geometric test time adaptation, among other techniques. Body is analyzed with a ResNet-based architecture that is customized with transient-appearance adversarial loss; and gait is analyzed with a pyramid spatial-temporal aggregation network to process full-motion video. The gait module learns spatio-temporal representations that can discriminate individual’s unique motion patterns.
A key module of BLAU is its fusion engine. An advanced fusion framework intelligently fuses the different biometric signatures, amplifying the strengths of some modalities while mitigating weakness with other modalities. This fusion operation is learned from training data. It works by attending to several factors (image quality, face/body pose, etc.) that are used to weight each modality adaptatively for each video. This enables BLAU to perform accurate recognition even when minimal biometric information is present, and media quality is heavily degraded.
In this presentation we will provide an overview of the BLAU pipeline, highlighting some of the innovations developed by our team, along with our academic partners, and show performance results for several use cases.


BLAU – Biometrics for Long Range Under Sensor Agnostic and Unconstrained Conditions

Individual warfighter readiness is critical to the success of the Navy mission; however, prediction of readiness status using standardized parameters in Navy warfighters is currently lacking. Data-driven machine learning applications have innovated prognostic algorithms for complex human diseases, traits, and other biological states. Despite these promising advances, relatively small sample sizes and a limited number of features per sample can pose challenges. This study addresses the knowledge gap of readiness prediction modeling using variables such self-reported physical health (e.g., pain, fatigue) and mental health (e.g., anxiety) symptoms, and selected gene variants related to the human stress response. Readiness was measured using the validated 15-item Military Readiness Scale (MRS-15). In a sample of n = 194, individuals were dichotomized into “high” (n = 90) and “low” (n = 104) readiness status using a cutoff of 77 out of 100 in the MRS-15 (equivalent to a readiness level of 77%). Initial results from the stepwise regression narrowed the predictive variables of military readiness to self-reported variables such as pain, fatigue, and anxiety, and two gene variants: one variant of the catecholamine enzyme, catechol-O-methyltransferase (rs737865), and one variant of the dopamine D2 receptor (rs1800497). Due to the limited data availability in this study, we implemented two models using random forest and LASSO-based classifiers to predict the total military readiness of a particular warfighter. A comparative analysis was run using random forest and LASSO to evaluate robustness in predicting high and low readiness status. In early comparisons of the models, preliminary results indicated that the random forest approach performed marginally better than LASSO in predicting readiness. We will present a complete summary of our results at the workshop. Overall, using machine learning techniques to develop predictions from warfighter data can lead to novel fleet readiness management tools that inform, and expedite, Navy leadership decisions.

Predicting Military Readiness In Naval Warfighters

This work addressed ways of making machine learning more robust against  adversaries manipulating its input data, as with input sensor data an adversary can at least partially control. The approach we explored was training and comparing alternative artificial-intelligence models for the same data. Our project tested this solution on real ship-tracking data from the public AIS (Automated Identification System) database available from the U.S. Coast Guard for U.S. coastal waters. We built ship tracks out of individual ship records and identified ten key features of tracks that help classify ship types, with the goal of identifying ships falsely reporting their identities. This identified some interesting errors and suspicious anomalies in the data such as impossible jumps in ship positions. We then systematically perturbed the data to various degrees to see how that affected the ship-type classifications with eight standard machine-learning methods. We were particularly interested in nonlinear effects that did not uniformly increase with the degree of perturbation, effects that varied significantly with the direction of perturbation, and in major differences in effects between the machine-learning methods. Our results did find some interesting weaknesses in the methods, and our methodology is general enough to be applied to other kinds of machine-learning tasks.

Adversary Attacks on Ship Tracking

Leadership coaching is used to support the transfer of leadership skills from the classroom to operational environments. However, it traditionally involves one-one-one development by professionally trained coaches, which requires a significant investment of resources for both training and logistics support. In this ONR-funded project, an interdisciplinary team developed a prototype AI-based leadership coach for planned use at the United States Naval Academy (USNA). The platform provides a series of individualized coaching sessions aligned with professional coaching standards, along with integrated crisis detection and user goal management, making the benefits of leadership coaching more broadly available at a fraction of the cost of a traditional coaching intervention.

Two versions of the platform are being developed concurrently, pending access to users, for both USNA midshipmen and leaders of student groups at the University of Minnesota (UMN). Both face similar challenges in identifying how to apply leadership concepts to real-world challenges, and the coach, named Nereus at USNA and Olivia at UMN, holds open-ended conversations with users helping them to understand and plan for everyday challenges. Over the course of multiple sessions, the coach works to develop actionable goals and then holds the coachee accountable in the following week. This also blurs the line between traditional and agentic AI in that the coach AI must drive the interaction and control the flow of individual sessions to meet its own goals invisibly to the coachee.

To successfully develop this prototype, core expertise from industrial-organizational psychologists was required as the traditionally favored approach to AI development by computer scientists, the collection of training data, was not feasible. Specifically, there exists no full-text database of successful coaching interactions with midshipmen to serve as exemplars for successful coaching to serve as a reinforcement data to train a custom large language model. Instead, the coach must be developed from guidance by subject matter experts in the coaching domain. This necessitates an explicitly interdisciplinary approach. Additionally, such a team is better capable of planning for and building detection models to meet secondary goals, like the automated detection of risk for imminent self-harm, and to develop appropriate procedures for the response to such events.


Developing an AI-based Leadership Coach for the US Naval Academy

A Network Science and System Dynamics Simulation Framework for Installation Resilience under Compound Extremes

For weather-informed ship route planning, optimizing for fuel efficiency entails predicting how the ship will react to the ocean environment during a voyage. Fuel consumption over a route is strongly dependent on ship resistance, which is primarily due to calm water resistance and added resistance in waves. Physics-based models of ship hydrodynamics can be employed to compute resistance as function of the hull form geometry, and ocean wave conditions. However, development of these models entails a significant amount of detailed information about the ship geometry, dimensions, and loading conditions, which are often not readily available. Additionally, there is a significant computational cost associated with the development of physics-based models. To address these challenges, this study investigates the feasibility of robust data generation for model training as part of the Machine Learning Surface ship Resistance estimation Framework (ML-SURF). This framework is focused on developing AI-enabled models to rapidly estimate calm water resistance and added resistance in waves based on a limited set of ship information.

The first step in this framework entails rapid generation of a wide range of ship geometries resulting in approximately 2,000 hull forms. Hull form geometries are generated using Rhinoceros 3D (Rhino), which is an advanced computer-aided design and graphics tool. In Rhino, this entails manipulation of the control points of a baseline parent design to generate new ship geometries based on a set of hull shaping parameters. Once the ship geometries are generated, calm water resistance and added wave resistance is computed for each respective hull form as function of large set of ocean conditions (wave height, modal period, relative wave heading), and ship speeds. Resistance predictions are generated using fast, reduced-order physics-based modeling and simulations. The resultant dataset covers a significant variety of displacement mono-hulls for a wide range of ship shaping and sizing parameters, which is a key enabler for robust model development via machine learning. As part of ML-SURF, a companion study is focusing on exploratory analysis and model training with this dat

Ship Resistance Estimation Framework with Machine Learning: Data Generation

“Harnessing Spatial AI for Advanced Geospatial Intelligence”
The rapid evolution of artificial intelligence and geospatial technology has paved the way for unprecedented advancements in various sectors, and in an era marked by complex geopolitical dynamics and continuous innovation, the U.S. Navy's strategic operations increasingly rely on advanced data analysis and intelligence capabilities. The introduction of spatial artificial intelligent within the machine learning ecosystem offers a transformative approach to processing and interpreting geospatial information. Oracle will present how spatial artificial intelligent empowers the U.S. Navy to enhance its intelligence, surveillance, and reconnaissance operations, thereby improving strategic decision-making and operational effectiveness.
Spatial AI equips the U.S. Navy with sophisticated geospatial machine learning algorithms that are designed for the detection of spatial patterns, prediction of outcomes, and classification of spatial data. By leveraging these advanced capabilities, naval personnel can efficiently and more accurately analyze vast amounts of geospatial data, which is essential for identifying potential threats, monitoring maritime activities, and supporting national security. The comprehensive suite of algorithms provided by spatial AI is designed to analyze vector data and location-based events, facilitating actionable insights. The end-to-end workflow capabilities offered by Spatial AI, encompassing data preparation, model training, and application, enabling the U.S. Navy to fully harness the power of spatial data and enhance their operational effectiveness. 
Spatial AI can be seamlessly integrated with various existing systems and processes, offering the potential to enhance the U.S. Navy’s strategic operations. For instance, by leveraging advanced spatial AI technology’s comprehensive workflow capabilities, which include training spatial machine learning models, and applying these models to various applications, the U.S. Navy can enhance the integration of AI into the existing systems. This integration allows for the consolidation of diverse geospatial data sources, enabling the U.S. Navy to uncover hidden trends and patterns that might otherwise go unnoticed. 
In practical terms, spatial AI can enhance the U.S. Navy's ability to conduct maritime domain awareness operations by providing precise and timely geospatial intelligence. This capability can be applied to monitoring illegal activities, such as smuggling and unauthorized fishing, as well as for planning and executing naval operations with greater accuracy. Additionally, spatial AI supports the U.S. Navy's efforts in disaster response and humanitarian assistance by enabling rapid assessment and analysis of affected areas.
In summary, the implementation of spatial AI within the U.S. Navy represents a significant advancement in geospatial intelligence and strategic planning. By harnessing the power of spatial AI, the U.S. Navy can improve its situational awareness, enhance international cooperation, and make well-informed decisions in an increasingly complex and interconnected world.

Premium content

Next from NAML 2025

DC Motor and Altitude Control for UAVs with Adaptive/Reinforcement Learning + Deterministic AI

Similar lecture

Shrinking Your TimeStep: Towards Low-Latency Neuromorphic Object Recognition with Spiking Neural Networks