United States

3D object detection is a crucial machine learning task for robotics applications, allowing the robot to detect obstacles and threats. Detecting objects in 3D, however, is a challenging machine learning task. Sensor fusion is often used to improve performance as sensor fusion makes use of complementary information from different sensor modalities. For example, camera images have rich semantic information, while LiDAR provides spatial information. Combining these can lead to better performance than using just one sensor, but performing the fusion is difficult due to these differences in the data types. Bird’s eye view (BEV) fusion, in which the LiDAR and camera data are projected into the bird’s eye view coordinate frame and fusion is performed in that coordinate frame, has shown excellent performance in the on-road autonomous driving scenario. While autonomous vehicles use standardized configurations of sensors such as the six-camera and one lidar unit configuration used to collect the commonly used NuScenes dataset, military vehicles need varied sensor types and configurations. To make it possible to use BEV fusion on different vehicles and in more military-relevant cases, we modify the existing BEV fusion model to be able take in different types of sensors, configurations of sensors, and number of sensors. Additionally, for military use cases, additional sensor modalities may need to be added, such as infrared images to help see through vegetation. Here we demonstrate our flexible sensor-fusion approach on the task of 3D person detection in varied terrain with dense vegetation using LiDAR, RGB camera, and IR camera fusion.

NAML 2024

Flexible multi-sensor fusion for 3D object detection 

off-road autonomy

3d object detection

sensor fusion

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

technical paper

### Welcome to the 8th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific 

**[More about NAML 2024](https://sites.google.com/go.spawar.navy.mil/naml/)**

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this workshop.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

The 8th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


The goal of this research is to present an application of
multispectral image fusion toward highlighting objects in still images
which are too dark, bright, or visually noisy for useful features to
be preserved. This research is currently ongoing and is intended to be
the first step towards a larger north-star goal to address a future
use case of real-time computer vision in tactical situations where
threats or objects of interest are highlighted with a silhouette
instead of a bounding box while preserving the visible image for
situational awareness. The focus of this research is on the capture,
alignment, and combination of visible and thermal images, followed by
an assessment of what the next steps are for future research to meet
the north-star goal. Preliminary experiments have shown that the image
processing does yield the intended highlight. Limitations were
discovered pertaining to processing thermal noise, as well as software
and hardware requirements for this kind of processing.

Multispectral Image Fusion as a Basis for Tactical Computer Vision

Integrating Natural Language Processing (NLP) models with unmanned aerial vehicles (UAVs) in autonomous systems represents a groundbreaking advancement towards enhancing human-machine interaction. This paper introduces an innovative approach to controlling a swarm of drones using the BERT (Bidirectional Encoder Representations from Transformers) NLP model facilitated by a novel interface that interprets voice commands into actionable directives. The cornerstone of our innovation lies in developing an Application Programming Interface (API) that seamlessly converts a stream of voice commands into precise instructions for drone swarms. This capability not only revolutionizes the ease of control but also broadens the applicability of drones in various fields, including surveillance, search and rescue operations, and agricultural monitoring.
To address the computational demands of running sophisticated NLP models like BERT on drones, our research delves into the creation of a hybrid computing platform that amalgamates Field-Programmable Gate Arrays (FPGAs) with Field-Programmable Analog Arrays (FPAAs). This combination offers unparalleled flexibility and efficiency in processing, catering to the specific needs of drone operations while executing complex NLP tasks. Furthermore, we explore the design and implementation of custom Electronic Speed Controllers (ESCs) and flight controller boards, tailoring the hardware to meet the unique requirements of our application.
A pivotal breakthrough in our project is the dynamic configuration of FPAAs using microcontrollers, a process slated for future enhancement through FPGA-based runtime-dynamic reconfiguration. This method promises significant improvements in resource utilization, power, and thermal management during operation, addressing critical challenges in drone technology.
Our work paves the way for more intuitive and effective control of drone swarms and sets the stage for significant advancements in drone computing architecture. By integrating cutting-edge NLP models, bespoke hardware solutions, and innovative computational platforms, we aim to redefine the capabilities and applications of drone technology.


Revolutionizing Drone Command Systems with Dynamic Federated Learning and Advanced NLP Integration

This project focuses on robust gait development using reinforcement learning on a 18-DOF hexapod
platform. Each actuator is equipped with a control strategy that sums position, velocity, and effort PID
control to produce motor PWM output. Due to the underlying framework, low level access to the
actuators is limited and the interface for communication is in the form of position, velocity, and torque
joint commands. The goal of this project is to create a simulation environment that can be used to train
the hexapod to walk using a soft actor-critic policy with a custom reward function. Using existing joint
feedback data from an established gait, the simulation model is enhanced to include control gains,
positional joint limits, and torque limits. Once the simulation is deemed to be high-fidelity, the soft
actor-critic policy is used to train a forward-walking gait in simulation using a tuned reward function.
The model produced by this learning event can be analyzed to determine the effectiveness of this gait
based on certain criterion: forward velocity, lateral velocity error, stance height, and balance. If the
learned gait is categorized as viable based on the aforementioned criterion, the model is then tested on
the physical robot. The learned model produces joint level commands based on joint feedback received
from the actuators.
The challenge of breaking down the gap between the simulation and real environments is explored in
this presentation. The MuJoCo environment used for simulating the robot is limited in its capabilities
for modeling individual actuators. Rather than specifying function-based control schema, a user is
given the ability to modify damping, armature, friction, and joint limit parameters. Identifying the
appropriate simulation parameters required thorough analysis of the existing control scheme aboard the
physical robot and relating it to the tunable parameters within the simulation environment. This proved
difficult due to the complex dynamics of the 18-DOF system. After much trial and error, suitable
simulation environment parameters were identified and a learned model was produced that yielded a
forward gait. The current model produces a gait with forward velocity of ~0.7 m/s, but future work
includes tuning the reward function to derive a gait with lesser forward velocity due to instability found
in testing on the physical robot.

Hexapod Gait Optimization Utilizing Reinforcement Learning

When researching robot swarms, many studies observe complex group behavior emerging from the individual agents' simple local actions. 
However, the task of learning an individual policy to produce a desired emergent behavior remains a challenging and largely unsolved problem.
We present a method of training distributed robotic swarm algorithms to produce emergent behavior. Inspired by the biological evolution of emergent behavior in animals, we use an evolutionary algorithm to train a 'population' of individual behaviors to approximate a desired group behavior. 
We perform experiments using simulations of the Georgia Tech Miniature Autonomous Blimps (GT-MABs) aerial robotics platforms conducted in the CoppeliaSim simulator. 
Additionally, we test on simulations of Anki Vector robots to display our algorithm's effectiveness on various modes of actuation.
We evaluate our algorithm on various tasks where a somewhat complex group behavior is required for success. These tasks include an Area Coverage task, a Surround Target task, and a Wall Climb task. 
We compare behaviors evolved using our algorithm against designed policies, which we create in order to exhibit the emergent behaviors we desire.

Learning Emergent Behavior in Robot Swarms with NEAT

Teams of mobile [aerial, ground, or aquatic] robots have applications in delivering resources, patrolling, information-gathering, pesticide application to crops, forest fire fighting, chemical plume source localization and mapping, and search-and-rescue. Some environments contain hazards e.g., rough terrain or seas, strong winds, or adversaries capable of attacking robots. Then, the robots must coordinate their trails in the hazardous environment to (i) cooperatively achieve the team-level objective with robustness to robot failures and (ii) strike some balance between expected robot failures versus utility gained.

Herein, we pose the bi-objective, risky team orienteering problem, where (i) a team of robots are mobile within an environment, abstracted as a graph (nodes: locations, edges: spatial connections between locations); (ii) each node offers a reward to the team depending on the number of robots that visit it; (iii) the traversal of each edge in the graph imposes a risk of robot failure/destruction; and (iv) the two [often, conflicting] team objectives are to maximize the expected (a) team reward and (b) number of robots that survive the mission. We then use ant colony optimization to search for the Pareto-optimal set of robot team trail plans. A human decision-maker can then select the robot trail plans that balance, according to their values, the two [conflicting] objectives.

Bi-Objective Ant Colony Optimization of the Risky, Robot Team Orienteering Problem

Flying by Ear: Acoustic sensing for unmanned air vehicles in complex environments

In this talk, we will discuss a deep learning method for both detecting and localizing damage in metal plates. To this end, we analyze time-series data that reflects material properties; specifically, we use a square arrangement of piezoelectric transducers mounted to an aluminum sheet, one of which generates a vibrational wave that then traverses the region of interest before being received by the remaining three sensors. 
A key aspect of our model design is the incorporation of physical principles, namely, symmetries associated with both time and the square arrangement of sensors, in addition to recognizing that the relevant physics is characterized by a hierarchy of scales. That is to say, while it is well known that deep neural networks can monitor structural health, we address the question of optimal architecture by appealing to the geometric deep learning blueprint. 
In summary, we will discuss our efforts, results, and lessons learned in the context of developing a neural network to infer from material response measurements both the existence and location of damage.


Staying Alert to Fatigue: DL Enabled Detection and Localization of Damage in Metal Plates

We propose a novel method of analyzing dynamic
systems for chaotic behavior using machine learning
techniques. The study of chaos in dynamic systems often
involves analyzing visual representations of data points
generated by the systems. One method includes embedding
one-dimensional time-series data into a two-dimensional
space (a “pq diagram”) as an interim step in the test for
chaotic behavior. We study the application of a machine
learning visual recognition model —a Convolutional Neural
Network— on the pq diagrams generated in various systems
and determine that they can be used to reliably differentiate
chaotic and periodic systems from one another, and possibly
to differentiate between different types of chaotic systems
themselves.

Detection and Discrimination of Chaotic Behavior Using Visual Recognition Models

The modeling of complex chemical kinetics in modern physics simulations are increasingly hampered by the incredible computational costs required to perform them. These sorts of simulations, such as combustion models, find that as the number of involved chemical species increase, so does the percentage of total compute time required by the chemical kinetics modeling alone. The computational cost often finds that the most advanced chemistry models to be intractable on even the most advanced computing systems.

To overcome this limitation, modern machine learning solutions can be employed to replace the embedded chemical kinetics solvers found in simulation codes. As machine learning continues to advance at break-neck speeds, their integration among existing simulations can still prove tricky as the additional programming or framework overheads can make them less attractive than simply sticking to the tried-and-true solvers already in use.

By using methods to convert the machine learning models into native code that directly aligns to the simulation already in use, some or most of this use can be avoided, and the code that is generated has the opportunity to be tuned to precisely benefit the physics simulation that is intended to be helped. This method of code generation of machine learning models allows the benefits of modern ML without requiring extra libraries and programming frameworks that can sometimes prove tedious to include. This capability present a hopeful new way to integrate and improve existing chemical kinetics solvers in the on-going physics simulations that use them today.


Code Generation of Advanced Machine Learning Models for Chemical Kinetics

Mesh characteristics and mesh quality are important inputs in thermodynamic and fluid simulations. 
Currently, limited or naïve processes are used to determine the quality of an existing mesh before and after repair. 
Meshes are considered “good” based on appearance and lack of those geometric defects that impact analysis. 
There is a need for robust metrics to characterize the quality of a mesh. 
These metrics can be used to evaluate the mesh locally (for application specific analysis) and globally (to compare various versions of the same model). 
We are exploring the use of Topological Data Analysis (TDA) methods, Geometry-based algorithms, and Boolean model comparisons 
as indicators for a “good” or “bad” 3D Computer-Aided Design (CAD) model meshes. We have developed a series of robust CAD mesh metrics 
to characterize the quality of a mesh. These metrics are applied methodically to re-mesh and repair 3D CAD models, 
including increasing and decreasing the number of facets, collapse nodes, edges, faces (decimating), and watertighting.
These methods have been implemented in a semi-automated workflow, requiring a human-in-the-loop on in determining which values best suite a given applicaiton.

Mesh characteristics and mesh quality are important inputs in thermodynamic and fluid simulations. Currently, limited or naïve processes are used to determine the quality of an existing mesh before and after repair. Meshes are considered “good” based on appearance and lack of those geometric defects that impact analysis. There is a need for robust metrics to characterize the quality of a mesh. These metrics can be used to evaluate the mesh locally (for application specific analysis) and globally (to compare various versions of the same model). We are exploring the use of Topological Data Analysis (TDA) methods, Geometry-based algorithms, and Boolean model comparisons as indicators for a “good” or “bad” 3D Computer-Aided Design (CAD) model meshes. We have developed a series of robust CAD mesh metrics to characterize the quality of a mesh. These metrics are applied methodically to re-mesh and repair 3D CAD models, including increasing and decreasing the number of facets, collapse nodes, edges, faces (decimating), and watertighting. These methods have been implemented in a semi-automated workflow, requiring a human-in-the-loop on in determining which values best suite a given applicaiton.



Premium content

Downloads

Next from NAML 2024

Multispectral Image Fusion as a Basis for Tactical Computer Vision

Similar lecture

CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformer