United States

The machine “awareness” of individual entities in massive swarms is a challenge for most conventional approaches, as is the fundamental control of all entities to form these swarms. Such issues can be addressed through alternate mathematical approaches that use self-organizing topologies. One such topological approach
has been used by one author to model 20,000 entities and has shown effective control and awareness for individuals with both limited processing power and bandwidth. The control and awareness result from using an approach to information frequently used in nature - stigmergy, i.e., sparse data which is information-dense, that includes lightweight communications between all nodes allowing
them to self-organize into a MANET. In an earlier paper, one of the authors used a topological structure based upon deterministic chaos, specifically an adaptive fractal. By constraining the entities to “believe” that they exist only within the boundaries of an adaptive fractal forces their topological layout to map to topological clusters. The
invariant yet transformable features allow each node to compute the IFS (Iterative Function System) to effectively “know” the relative positions of all the other entities, and form adaptive, self-healing swarm clusters. In this paper, an in-depth mathematical description of this algorithmic topology is given, which computationally is of O(n).

NAML 2021

The Mathematics Behind Massive Swarm Self-Awareness

adaptive fractal

stigmergy

leveraging complexity science

swarms

awareness

chaos

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS 2020

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

poster

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=580 height=700 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

The annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


Acoustic source localization is a long-standing and generically challenging problem in structural acoustics (Worden et al., 2001). Localizability depends on the frequency content of the driving force and the uncertainty in the position of the source, among other factors. Naval applications, such as predictive maintenance for dozens of machines attached to a single hull, require a limited number of sensors to detect a large variety of signals. 


Smart, sparse sensor placement on a 2D plate performs better than random placements, and is fast and automatic


Active learning in semi-supervised classification seeks to select points from the pool of unlabeled data to label that will in turn help to improve the underlying classifier's accuracy. Such potential for classifier improvement is quantified in a real-valued {\it acquisition function}. We present a ``model-change'' acquisition function for batch active learning in graph-based semi-supervised learning (SSL) for multiclass classification.

Multiclass Active Learning for Graph-Based Semi-Supervised Learning

Running machine learning analytics over geographically distributed datasets is a rapidly arising problem in the world of data management policies ensuring privacy and data security. Visualizing high dimensional data using tools such as t-distributed Stochastic Neighbor Embedding (tSNE) and Uniform Manifold Approximation and Projection (UMAP) became common practice for data scientists. Both tools scale poorly in time and memory.
While recent optimizations showed successful handling of 10,000 data points, scaling beyond million points is still challenging. We introduce a novel framework: Sketch and Scale (SnS). 
It leverages a Count Sketch data structure to compress the data on the edge nodes, aggregates the reduced size sketches on the master node, and runs vanilla tSNE or UMAP on the summary, representing the densest areas, extracted from the aggregated sketch. 


TSNE & UMAP: Sketch & Scale

The headline successes of reinforcement learning on video games like StarCraft and DOTA 2 beg questions about the future role of the technology in the Department of Defense. Military simulations seemingly have much in common with video games, and the lack of a strong human behavior model is arguably a limiting factor for their use in the current environment. Training exercises require that opposing forces and adjoining allied forces act intelligently, and human pucksters with appropriate backgrounds and the funding to pay for them is not always forthcoming. The scope of analytical simulations must currently be limited due to the lack of current behavior models that react flexibly and appropriately over a wide range of circumstances. And more aggressive use cases for simulation in operational decision support might become practical with better AI to serve as a sparring partner. It is this last use case that we focus on here.

AI can potentially provide operational decision support by being a competent opponent of the human planner in simulation. The human planner would control his own forces according to his proposed plan, and the AI would do its best to defeat the plan. The planner could then choose to alter the plan and fight the AI again until he is satisfied. But is it actually practical to develop strong players using reinforcement learning?

Previously, it has been shown that reinforcement learning is capable of learning optimal attacking behavior in simple ground combat scenarios [Boron, J and Darken, C. "Developing Combat Behavior through Reinforcement Learning in Wargames and Simulations", IEEE Conference on Games (CoG) 2020]. The primary examples studied were quite small (e.g. three units attacking two on a featureless plain, with all units being the same type, defenders stationary), but the Proximal Policy Optimization algorithm was able to learn provably optimal attacking behavior, and even find qualitatively different optimal solutions for the cases of an urgent attack (highly discounted rewards) and a more cautious one (less discounting). Still, this scenario is far short of the complexity that would be required for operational use.

In the proposed presentation, we will describe our ongoing and planned efforts to investigate the ability of reinforcement learning to learn strong attacking behavior in more complex scenarios, i.e. with more units of multiple types maneuvering through heterogeneous terrain, and with an actively maneuvering opponent. Our new training simulation for the AI, while still relatively simple, supports four different unit types and five different types of terrain. Maps are hexagonal grids of any size and the number of units per side is arbitrary. The simulation supports an AI Gym standard interface and is typically trained using the Stable Baselines 3 suite of PyTorch-based reinforcement learning algorithms.

The new training simulation represents the input features of the neural network in an image like manner, permitting the use of convolutional neural networks (CNNs). The spatially invariant nature of CNN kernels allows trained policies to learn optimal attacking behaviors against a retreating adversarial force, regardless of where the units are initially placed in the simulation. This feature promotes a more dynamically trained agent and increases the likelihood of achieving optimal behaviors in highly complex scenarios.

Reinforcement Learning for Tactical Decision AI

Mindful™ software provides human-understandable competency statements in the DARPA Competency Aware Machine Learning program. MindfuL™ software models environments across ML applications compactly with hierarchical Dirichlet processes allowing it to provide environment based-competency assessments through deep learning and deep explainer models as well as competency of similar historical data.

Mindful™: Competency Aware Machine Learning

We present an ongoing investigation into the learning of the input to state relationship of a dynamical system by using nonlinear Adaptive Control Theory combined with Spiking Neural Networks (SNNs). In particular, we use a specific type of SNN named Efficient Balanced Networks (EBNs) initially proposed by Deneve, Machens et. al. that provides desirable properties such as distributed information representation, low firing rates, and high encoding precision. Neuronal spikes are fired based on a threshold that is generated by optimizing over a global cost function that penalizes output tracking error, spike frequency, and spike distribution across neurons. Our objective is to implement these algorithms on neuromorphic hardware and apply them to complex robotic control applications. Applications include online system identification, generation of oracles for policy training, adaptive model predictive control, and model-based reinforcement learning. This presentation is effectively a status update on these investigations showing results obtained to date.

Learning Dynamical Systems with Neuromorphic Efficiently Balanced Spiking Neural Networks

When learning to perform motor tasks in a simulated environment, neural networks must be allowed to explore their action space to discover new potentially viable solutions. However, in an online learning scenario with physical hardware, this exploration must be constrained by relevant safety considerations in order to avoid damage to the agent’s hardware and environment. Our aim is to address this problem by training a neural network, which we will refer to as the “safety network,” to estimate the region of attraction (ROA) of a dynamical system controlled by an “acting network.” This safety network can thereby be used to quantify the relative safety of control actions proposed by the acting network and prevent the selection of damaging actions. Here we present our preliminary development of the safety network by training a non-spiking neural network to represent the ROA of several dynamical system benchmark problems. The training of this network is predicated upon both Lyapunov theory and neural solutions to partial differential equations (PDEs). By learning to approximate the viscosity solution to a specially chosen PDE, the safety network learns to approximate a Lyapunov function whose unity level set is the boundary of the ROA. We train our safety network to solve these PDEs in a semi-supervised manner, utilizing a loss function that penalizes disagreement with the PDE’s initial and boundary conditions, as well as non-zero residual and variational terms. In future work, we intend to extend this technique to spiking neural networks to be run on neuromorphic hardware for real-time online learning.

Learning to Estimate Stability Margins in Dynamical Systems

Our NLP predictive algorithm can accurately assign and provide solutions to NAVY 311 support request tickets, reducing contact center labor and time to resolution.

Next Generation NLP and AI Methods in the Navy Contact Center

A heterogeneous, nonoverlapping domain-decomposition, explicit, finite volume method (HT-NODDE-FVM) is developed and applied to a real-time, hybrid, process-state estimator of 3D unsteady, advection-diffusion plume fields produced by moving aerial intruders. The hybrid Luenberger-naïve estimator uses field measurements taken onboard a data-driven self-guided UAV and computes the plume field in real-time while guiding the UAV to locations that provide optimal information to the estimator.

Real-Time Estimation of Threat Fields With an Autodidactic Data-Driven Sensing Aerial Vehicle

The search for neural architecture is producing many of the most exciting results in artificial intelligence. It has increasingly become apparent that task-specific neural architecture plays a crucial role for effectively solving problems. This paper presents a simple method for learning neural architecture through random mutation. This method demonstrates 1) neural architecture may be learned during the agent’s lifetime, 2) neural architecture may be constructed over a single lifetime without any initial connections or neurons, and 3) architectural modifications enable rapid adaptation to dynamic and novel task scenarios. Starting without any neurons or connections, this method constructs a neural architecture capable of high-performance on several tasks. The lifelong learning capabilities of this method are demonstrated in an environment without episodic resets, even learning with constantly changing morphology, limb disablement, and changing task goals all without losing locomotion capabilities.

Premium content

Next from NAML 2021

Smart, sparse sensor placement on a 2D plate performs better than random placements, and is fast and automatic

Similar lecture

Real-time Learning and Planning in Environments with Swarms: A Hierarchical and a Parameter-based Simulation Approach