United States

In a constantly evolving robotics ecosystem, machine learning techniques have provided essential advancements in the field of Human-Robot Interaction. From reasoning to perception, traditional hand-crafted algorithms have been replaced by learning-based methods for improved performance. However, many of the state-of-the-art artificial neural networks and transformer models require large amounts of compute and power to run. While battery technology has improved, mobile robotic platforms are still restricted to low powered onboard computers in order to maximize their field operation time. These system requirements force roboticists to use algorithms that have high power requirements, reducing their running time or to settle for systems that are less power-hungry but also less accurate. Using robots on Navy ships makes this problem especially acute because power is at an especially high premium.

To address these power and computational limitations for mobile robots, researchers have looked towards the field of neuromorphic computing. Neuromorphic computing aims to create hardware and algorithms which use biologically inspired neural networks that are distinctly different from more traditional von Neumann computing. Spiking neural networks operate on event based, timeseries data making them excellent candidates for onboard robotic sensor data. Critically, spiking neural networks use far less power because each neuron only computes when it is needed (at an event).

In this report, our goal is to use robots for shipboard maintenance for Naval operations at sea. Robots should provide assistance with completing the dirty, dull, and dangerous work that Naval personnel carry out on a day-to-day basis. In order to accomplish this mission, these robots must be able to achieve state-of-the-art performance in reasoning, perception, and action while also operating on limited, battery-based power supplies. One of the common tasks that robots will be used for onboard will be finding, storing, and retrieving various tools for sailors to use. While mobile robots have excellent algorithms for picking up objects, current state of the art algorithms for giving a tool or object to a human sailor use a thresholding approach (e.g., release when a force is &gt; ø). On any moving platform, a thresholding approach will simply not work – the robot will either drop the object if ø is too low or not let the object go if ø is too high. We designed a neuromorphic model that learned when a handoff occurred and then compared it to other learning-based approaches (e.g., K-Nearest Neighbors) and the traditional thresholding approach. These neuromorphic approaches outperform both approaches in both accuracy and speed and have a much lower power requirement. For future iterations of our work, we intend to test our neuromorphic models on neuromorphic hardware to comprehensively benchmark the power and performance improvements of our method.

NAML 2024

Neuromorphic Robot-Human Handoffs

neuromorphic

machine learning

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

To spread awareness of current machine learning research relevant to Naval applications

To connect machine learning researchers with experts in military requirements

To build and strengthen collaborations within the defense research community


<figure class="video_container">
  <iframe src="https://s3.amazonaws.com/pf-user-files-01/u-59356/uploads/2022-03-22/hk13mxo/NAML_ReceptionWithtitle.mp4" width=826  height=526  frameborder="0" allowfullscreen="true"> </iframe>
</figure>

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

NAML 2022

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

To spread awareness of current machine learning research relevant to Naval applications

To connect machine learning researchers with experts in military requirements

To build and strengthen collaborations within the defense research community



In a constantly evolving robotics ecosystem, machine learning techniques have provided essential advancements in the field of Human-Robot Interaction. From reasoning to perception, traditional hand-crafted algorithms have been replaced by learning-based methods for improved performance. However, many of the state-of-the-art artificial neural networks and transformer models require large amounts of compute and power to run. While battery technology has improved, mobile robotic platforms are still restricted to low powered onboard computers in order to maximize their field operation time. These system requirements force roboticists to use algorithms that have high power requirements, reducing their running time or to settle for systems that are less power-hungry but also less accurate. Using robots on Navy ships makes this problem especially acute because power is at an especially high premium.

To address these power and computational limitations for mobile robots, researchers have looked towards the field of neuromorphic computing. Neuromorphic computing aims to create hardware and algorithms which use biologically inspired neural networks that are distinctly different from more traditional von Neumann computing. Spiking neural networks operate on event based, timeseries data making them excellent candidates for onboard robotic sensor data. Critically, spiking neural networks use far less power because each neuron only computes when it is needed (at an event).

In this report, our goal is to use robots for shipboard maintenance for Naval operations at sea. Robots should provide assistance with completing the dirty, dull, and dangerous work that Naval personnel carry out on a day-to-day basis. In order to accomplish this mission, these robots must be able to achieve state-of-the-art performance in reasoning, perception, and action while also operating on limited, battery-based power supplies. One of the common tasks that robots will be used for onboard will be finding, storing, and retrieving various tools for sailors to use. While mobile robots have excellent algorithms for picking up objects, current state of the art algorithms for giving a tool or object to a human sailor use a thresholding approach (e.g., release when a force is > ø). On any moving platform, a thresholding approach will simply not work – the robot will either drop the object if ø is too low or not let the object go if ø is too high. We designed a neuromorphic model that learned when a handoff occurred and then compared it to other learning-based approaches (e.g., K-Nearest Neighbors) and the traditional thresholding approach. These neuromorphic approaches outperform both approaches in both accuracy and speed and have a much lower power requirement. For future iterations of our work, we intend to test our neuromorphic models on neuromorphic hardware to comprehensively benchmark the power and performance improvements of our method.

technical paper

### Welcome to the 8th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific<br><br>

**[More about NAML 2024](https://sites.google.com/go.spawar.navy.mil/naml/)**
<br><br>

Please log in to explore this workshop.

The 8th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


Currently, large-scale human movement modeling focuses on aggregated, high-level data to study migration, the spread of disease, and other patterns of behavior. For example, hourly toll booth counts indicate traffic volume. Kitware’s work on the IARPA HAYSTAC program aims to capture more fine-grained, individual human movement patterns to identify what characterizes “normal” movement while upholding public expectations for privacy. In contrast to toll booth counts, this could mean modeling the behavior of vehicles traveling along a toll road. Where do they originate from? Where do they typically go after the toll booth, and what stops do they make along a given route? How can we model this behavior without violating any individual’s privacy?

Toward these ends Kitare is developing a privacy preserving human movement simulation system designed to handle mega-cities of up to 30 million people and simulations as long as an entire year. Our MIRROR simulator uses a parallel, multi-resolution, spatio-temporal decomposition approach to achieve state-of-the-art computational efficiency and scale. In our talk we will describe the architecture of our simulator, including our novel combination of efficient mesoscopic, que-based traffic simulation for traffic modeling and route generation with a parallel kinematic movement simulator which captures the detailed, mete- by-meter movement of vehicles. Results will demonstrate the scaling behavior of our parallel simulator in comparison to existing methods and compare our simulator’s output to trajectory statistics derived from real mobility data. 

In addition, we will describe approaches our team has developed to detect anomalous behaviors in trajectory data. These include more classical “feature engineering” based methods and novel transformer-based approaches. We will share initial anomaly detection results using our methods. Further, we will introduce methods we have developed to generate anomalous trajectories that meet certain goals, such as going to an unusual place or going to a usual place at an unusual time, but are hidden. 

While HAYSTAC is focused on urban mobility modeling, the approaches we are creating can be readily applied to maritime problems. One potential extension of our work is using ship transponder data and other sources of ship tracking information to model normal ship movement patterns and detect anomalous behaviors, such as meetings for the illicit transfer of goods to evade sanctions or illegal fishing. 


Hide and Seek in Massive Simulated Trajectory Datasets

Modern radar systems with highly sophisticated cognitive mechanisms designed for responding intelligently to external stimuli can be difficult to track. Their ability to automate waveform selection and update waveform class parameters enables cognitive radars to effectively counter electronic jamming attack. This project aimed to investigate and implement an advanced machine learning algorithm to track cognitive radars. The main goal was to extract pertinent information via observing the behavior of a radar to predict its actions. A reinforcement learning model was designed to react to jamming techniques in accordance with a stochastic state-action policy. Our empirical results showed that even in a simple scenario within a space-constrained environment, accurately tracking a cognitive target at every time-step could be difficult. Utilizing the Q-learning model, we demonstrated that a trained reinforcement learning agent could accomplish its task via multiple optimal paths. The resulting Q-Table illustrated the various equally optimal sequences of actions that led the agent to reach its goal. The conclusion of this project further confirms the challenge in predicting the actions of targets equipped with cognitive capabilities. Our future work will assess the feasibility of employing artificial intelligence/machine learning techniques to estimate the final destination of a cognitive moving target.

Tracking Cognitive Radars

Many projects use security software tools for intrusion detection and the verification of integrity in system files. Typically, scans and logs are read by humans. Administering and checking logs from these tools is a repetitive, undifferentiated task. In this effort AWS worked in partnership with mission owners to analyze real-world logs from assets in-use. 

Multiple machine learning techniques for pattern recognition were explored, including clustering algorithms, and anomaly detection. Evaluation techniques were investigated on real-world logs at scale to ensure effective results. 

The benefits of a repeatable machine learning approach are increased security, lower costs from human intervention, and an improved user experience. Further by using an algorithm rather than a human in the loop, higher fidelity can be achieved in catching anomalies as its less prone to error.

Pattern Recognition in Security Logs

Accurate recognition and tracking of objects of interest, such as people in various poses, especially casualties, using AI-enabled small unmanned aerial vehicles (UAVs) have become prevalent for various applications, such as disaster relief, red force tracking, and battlefield surveillance. Successful execution of the applications requires accurately detecting small and occluded humans in various complex real-world environments. However, aerial human detection on UAV video feeds faces significant challenges due to adverse conditions, such as high-flight altitudes, aircraft vibration, and low camera resolution. Moreover, the data scarcity in UAV-based perception poses a significant problem, resulting in error-prone detectors. To tackle the adverse conditions and data scarcity issue, Archangel aims to advance the science behind UAV-based human and casualty detection by 1) creating large-scale synthetic data, 2) enhancing the realism of the synthetic data using the Sim2Real transformation, and finally, 3) augmenting the learning with hybrid sets of real and synthetic data. 

Archangel is a customer-reimbursed program sponsored by the Defense Threat Reduction Agency (DTRA). Archangel aims to develop advanced machine learning models embedded into small UAVs to recognize people in distress captured in the context of natural disasters or military conflicts by leveraging synthetic data for rapidly adapting machine learning agents. Archangel started in early 2020, and Phase 1 of the project was finished in mid-2023. The Phase 1 deliverables include 1) enhanced machine learning models for detecting humans in various poses captured by UAVs flying at relatively high altitudes and long ranges and 2) the Archangel datasets, the first UAV-based human detection dataset composed of real and synthetic subsets captured with similar imagining conditions and UAV position and pose metadata. Archangel is currently in Phase 2, focusing on identifying human casualties using UAVs in the context of military conflicts to prioritize the search and rescue of Warfighters in severe physiological conditions.
Notably, in Archangel, to enhance the synthetic data-based data augmentation for optimizing detection models, we have developed a novel approach called Progressive Transformation Learning (PTL), which gradually augments a training dataset by adding transformed synthetic images with enhanced realism. Generally, a virtual2real transformation suffers from quality degradation when a large domain gap exists between real and synthetic images. To deal with the domain gap, PTL takes a novel approach that progressively iterates the following three steps: 1) select a subset from a pool of synthetic images according to the domain gap, 2) transform the selected synthetic images to enhance realism, and 3) add the transformed synthetic images to the training set while removing them from the pool. Additionally, in PTL, accurately quantifying the domain gap is critical. To do that, we theoretically demonstrate that the feature representation space of a given object detector can be modeled as a multivariate Gaussian distribution. 
In this workshop, we will introduce Archangel and the novel data augmentation approach, PTL, developed by the Archangel team. We will also present experimental results showing substantial performance increases over the baseline, especially in the small data and the cross-domain regime.


Archangel: UAV-based Human and Casualty Detection at The Edge

Small mobile sensor platforms performing observation missions may utilize pattern recognition algorithms that are capable to locating human figures that may be found in (multi-spectral) imagery. Good human-figure detection and localization performance is desired in highly cluttered scenes in which the subjects may have random poses in various levels of concealment. Moreover, such algorithms must be capable of adapting quickly to maintain successful mission execution while maintaining cognizance of the platform’s size, weight, and power constraints. We demonstrate how an application and adaptation of a preexisting elastic shape analysis-based pattern recognition algorithm – which we call the Augmented Path Straightening Algorithm (APSA) – addresses these algorithmic requirements. Furthermore, we demonstrate how APSA has the benefit of being lightweight in terms of hardware usage and can operate on single-look scene observations. We provide a demonstration of APSA’s effectiveness in performing detections of partially concealed humans in cluttered scenes.

Lightweight and Robust Partially Concealed Human Detection in Cluttered Scenes Using Elastic Shape Analysis

3D object detection is a crucial machine learning task for robotics applications, allowing the robot to detect obstacles and threats. Detecting objects in 3D, however, is a challenging machine learning task. Sensor fusion is often used to improve performance as sensor fusion makes use of complementary information from different sensor modalities. For example, camera images have rich semantic information, while LiDAR provides spatial information. Combining these can lead to better performance than using just one sensor, but performing the fusion is difficult due to these differences in the data types. Bird’s eye view (BEV) fusion, in which the LiDAR and camera data are projected into the bird’s eye view coordinate frame and fusion is performed in that coordinate frame, has shown excellent performance in the on-road autonomous driving scenario. While autonomous vehicles use standardized configurations of sensors such as the six-camera and one lidar unit configuration used to collect the commonly used NuScenes dataset, military vehicles need varied sensor types and configurations. To make it possible to use BEV fusion on different vehicles and in more military-relevant cases, we modify the existing BEV fusion model to be able take in different types of sensors, configurations of sensors, and number of sensors. Additionally, for military use cases, additional sensor modalities may need to be added, such as infrared images to help see through vegetation. Here we demonstrate our flexible sensor-fusion approach on the task of 3D person detection in varied terrain with dense vegetation using LiDAR, RGB camera, and IR camera fusion.

Flexible multi-sensor fusion for 3D object detection 

The goal of this research is to present an application of
multispectral image fusion toward highlighting objects in still images
which are too dark, bright, or visually noisy for useful features to
be preserved. This research is currently ongoing and is intended to be
the first step towards a larger north-star goal to address a future
use case of real-time computer vision in tactical situations where
threats or objects of interest are highlighted with a silhouette
instead of a bounding box while preserving the visible image for
situational awareness. The focus of this research is on the capture,
alignment, and combination of visible and thermal images, followed by
an assessment of what the next steps are for future research to meet
the north-star goal. Preliminary experiments have shown that the image
processing does yield the intended highlight. Limitations were
discovered pertaining to processing thermal noise, as well as software
and hardware requirements for this kind of processing.

Multispectral Image Fusion as a Basis for Tactical Computer Vision

Integrating Natural Language Processing (NLP) models with unmanned aerial vehicles (UAVs) in autonomous systems represents a groundbreaking advancement towards enhancing human-machine interaction. This paper introduces an innovative approach to controlling a swarm of drones using the BERT (Bidirectional Encoder Representations from Transformers) NLP model facilitated by a novel interface that interprets voice commands into actionable directives. The cornerstone of our innovation lies in developing an Application Programming Interface (API) that seamlessly converts a stream of voice commands into precise instructions for drone swarms. This capability not only revolutionizes the ease of control but also broadens the applicability of drones in various fields, including surveillance, search and rescue operations, and agricultural monitoring.
To address the computational demands of running sophisticated NLP models like BERT on drones, our research delves into the creation of a hybrid computing platform that amalgamates Field-Programmable Gate Arrays (FPGAs) with Field-Programmable Analog Arrays (FPAAs). This combination offers unparalleled flexibility and efficiency in processing, catering to the specific needs of drone operations while executing complex NLP tasks. Furthermore, we explore the design and implementation of custom Electronic Speed Controllers (ESCs) and flight controller boards, tailoring the hardware to meet the unique requirements of our application.
A pivotal breakthrough in our project is the dynamic configuration of FPAAs using microcontrollers, a process slated for future enhancement through FPGA-based runtime-dynamic reconfiguration. This method promises significant improvements in resource utilization, power, and thermal management during operation, addressing critical challenges in drone technology.
Our work paves the way for more intuitive and effective control of drone swarms and sets the stage for significant advancements in drone computing architecture. By integrating cutting-edge NLP models, bespoke hardware solutions, and innovative computational platforms, we aim to redefine the capabilities and applications of drone technology.


Revolutionizing Drone Command Systems with Dynamic Federated Learning and Advanced NLP Integration

This project focuses on robust gait development using reinforcement learning on a 18-DOF hexapod
platform. Each actuator is equipped with a control strategy that sums position, velocity, and effort PID
control to produce motor PWM output. Due to the underlying framework, low level access to the
actuators is limited and the interface for communication is in the form of position, velocity, and torque
joint commands. The goal of this project is to create a simulation environment that can be used to train
the hexapod to walk using a soft actor-critic policy with a custom reward function. Using existing joint
feedback data from an established gait, the simulation model is enhanced to include control gains,
positional joint limits, and torque limits. Once the simulation is deemed to be high-fidelity, the soft
actor-critic policy is used to train a forward-walking gait in simulation using a tuned reward function.
The model produced by this learning event can be analyzed to determine the effectiveness of this gait
based on certain criterion: forward velocity, lateral velocity error, stance height, and balance. If the
learned gait is categorized as viable based on the aforementioned criterion, the model is then tested on
the physical robot. The learned model produces joint level commands based on joint feedback received
from the actuators.
The challenge of breaking down the gap between the simulation and real environments is explored in
this presentation. The MuJoCo environment used for simulating the robot is limited in its capabilities
for modeling individual actuators. Rather than specifying function-based control schema, a user is
given the ability to modify damping, armature, friction, and joint limit parameters. Identifying the
appropriate simulation parameters required thorough analysis of the existing control scheme aboard the
physical robot and relating it to the tunable parameters within the simulation environment. This proved
difficult due to the complex dynamics of the 18-DOF system. After much trial and error, suitable
simulation environment parameters were identified and a learned model was produced that yielded a
forward gait. The current model produces a gait with forward velocity of ~0.7 m/s, but future work
includes tuning the reward function to derive a gait with lesser forward velocity due to instability found
in testing on the physical robot.

Hexapod Gait Optimization Utilizing Reinforcement Learning

When researching robot swarms, many studies observe complex group behavior emerging from the individual agents' simple local actions. 
However, the task of learning an individual policy to produce a desired emergent behavior remains a challenging and largely unsolved problem.
We present a method of training distributed robotic swarm algorithms to produce emergent behavior. Inspired by the biological evolution of emergent behavior in animals, we use an evolutionary algorithm to train a 'population' of individual behaviors to approximate a desired group behavior. 
We perform experiments using simulations of the Georgia Tech Miniature Autonomous Blimps (GT-MABs) aerial robotics platforms conducted in the CoppeliaSim simulator. 
Additionally, we test on simulations of Anki Vector robots to display our algorithm's effectiveness on various modes of actuation.
We evaluate our algorithm on various tasks where a somewhat complex group behavior is required for success. These tasks include an Area Coverage task, a Surround Target task, and a Wall Climb task. 
We compare behaviors evolved using our algorithm against designed policies, which we create in order to exhibit the emergent behaviors we desire.

Premium content

Downloads

Next from NAML 2024

Hide and Seek in Massive Simulated Trajectory Datasets

Similar lecture

Neuromorphic Hardware Anisotropic Reservoir Trajectory Generation and execution with Robotic Arm