United States

The challenge of applying artificial intelligence to autonomous network security was investigated using Graph Convolutional Network Reinforcement Learning Methods in a dynamically-scaling computer network. A deep Q-network single agent method was compared to a hybrid method, which integrated a larger agent strategizing for the entire network with individual agents working on each node. The simulation network contained &#39;real&#39; machines, to be protected, and &#39;decoy&#39; machines, to act as deception mechanisms. An attacker pivoted through the network and the agents’ defensive strategies were tracked to determine which version adapted the best. These algorithms are being explored for this problem because of the changing action space along with the ability to adapt to the evolving network topology. 


NAML 2021

Exploring Graph Convolutional Network Reinforcement Learning for Cyber Defense

cybersecurity

graph convolutional network

reinforcement learning

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

The challenge of applying artificial intelligence to autonomous network security was investigated using Graph Convolutional Network Reinforcement Learning Methods in a dynamically-scaling computer network. A deep Q-network single agent method was compared to a hybrid method, which integrated a larger agent strategizing for the entire network with individual agents working on each node. The simulation network contained 'real' machines, to be protected, and 'decoy' machines, to act as deception mechanisms. An attacker pivoted through the network and the agents’ defensive strategies were tracked to determine which version adapted the best. These algorithms are being explored for this problem because of the changing action space along with the ability to adapt to the evolving network topology. 


poster

Unlike current user emulators, D2U does more than simply replay previously-observed behavior, or call simple heuristics for stringing together patterns of actions. Instead, D2U provides the ability to generate unlimited, novel sequences of realistic behavior and orchestrate a plethora of virtual machines to enact these behaviors, thereby emulating a network of users. This ability to generate realistic, new behaviors sets D2U apart from all existing emulation technologies either in the academic literature or on the market.

D2You: Data Driven User Emulator​

Macro-meteorological models predict optical turbulence as a function of weather data. Existing models often struggle to accurately predict the rapid fluctuations in Cn2 in near-maritime environments. Seven months of Cn2 field measurements were collected along an 890 m scintillometer link over the Severn River in Annapolis, Maryland. This time series was augmented with local meteorological measurements to capture bulk-atmospheric weather measurements. The prediction accuracy of existing macro-meteorological models was analyzed in a range of conditions. Next, machine-learning techniques were applied to train new macro-meteorological models using the measured Cn2 and measured environmental parameters. Finally, the Cn2 predictions generated by the existing macro-meteorological models and new machine-learning informed models were compared for four representative days from the data set. These new models, under most conditions, demonstrated a higher overall Cn2 prediction accuracy, and were better able to track optical turbulence. Further tuning and machine learning architectural changes could further improve model performance.

Optical Turbulence Prediction from Data-driven Models

The U.S. military has been implementing simulation-based training systems in an effort to provide more effective training programs with reduced time to train. Integrating emerging technologies such as virtual, augmented, and mixed reality into their training curriculums has become the expectation. Over the past couple of years, the military has focused its attention on future threat training by developing multi-domain environments for more experienced warfighters. This includes Live, Virtual, and Constructive (LVC) assets for operations in complex, dynamic, and uncertain battlespaces.

The Department of Defense (DoD) is investing in a Joint All-Domain Command and Control (JADC2) concept that strives to connect multiple domains (air, land, sea, space, and cyber) into a single interoperable network (Congressional Research Services, 2020). Developing an interoperable Synthetic Training Environment is
challenging and once successful, the amount of data collected in a JADC2 type of environment can be overwhelming. The DoD is working across services to determine how best to manage the data. This presentation describes and illustrates an architecture for analyzing measurable data for assessing warfighter performance that can be collected during training in a natural environment. This data architecture is open, free, portable, containerized, and scalable for real-time monitoring and feedback.

The success of JADC2 training is dependent on relevant and actionable data (Rosenberg, 2020). This presentation identifies relevant data for measuring training effectiveness that provides actionable data for use in training automation, personalization, and adaptation. Research conducted by Hoke et al. (2017) and Dideriksen et al. (2018) validates the methodology of assessing training effectiveness through the combination of task performance measures and physiological signals of cognitive workload. It concludes with a discussion of how this methodology and architecture can be applied to improve training outcomes for individuals and teams for modern warfare in a multi-domain battlespace.

Automated Human-Machine Performance Assessment for the Multi-Domain Battle Space

Rip channels are a relevant area of study for Navy oceanographers because of their violent nature and relationship to the ocean floor. Images of these channels can serve as viable data when on site methods of collecting data are not available. Manually annotating rip channels is cumbersome, especially since there may be thousands of images to analyze for adequate climatology information. Although it has been shown that machine learning can help automate this process, developing a general-purpose rip channel classifier is problematic due to both lack of prior labeled data and the disparate features of litoral sites. Additionally, the subjective nature of rip channels yields complexities in general-purpose approaches, mainly due to the complexity of basic feature correlation. We present an interactive machine learning (IML) rip channel annotation application that uses online machine learning, active learning, and uncertainty modeling to provide a human-machine team for the annotation process. The application is able to assist users with annotating rip channels in their own datasets without the need for prior labeled data or parameter tuning. The interface by which this is done consists of four vertices denoting the length and span of the rip channel. The application attempts to place the vertices, the user then confirms or corrects the placements, the IML immediately learns from the verified placement, and the process continues. The IML implementation also takes advantage of user-offline time to optimize hyper parameters, perform feature selection, and reduce features to maintain scaling. Four persistent-image datasets from Oregon State University’s Coastal Imaging Laboratory were selected for gathering data from 2 locations, namely: Duck, North Carolina and Secret Harbour, Australia. The tool’s performance is evaluated based on correct vertex placement by the IML implementation. Results show that over 80% of the rip channels are correctly annotated by the IML implementation with an optimization cycle of about 4 images per dataset without the need for any prior labeled data.

Interative Machine Learning for Rip Channel Annotation

JAUST (Japan US Search Tool) is a web and mobile cognitive search solution built for the US Forces Japan that enables users to quickly search and retrieve documents in historical digital archives. Physical archives (in paper or PDF format) were digitized and text was extracted using optical character recognition (OCR) and computer vision techniques. AI models (natural language processing, large language models, graph theory) are used to extract signatures, analyze, and tag documents before ingestion into an elasticsearch database. A lucene search engine is queried using a custom REST API to display results in a Graphical User Interface (web application) built with Angular. JAUST includes free text search and filtering capability (dates, locations, memo numbers) that retrieves the documents in order of relevance, highlights search terms, and displays original PDFs. JAUST is hosted in an AWS govcloud environment and includes continuous integration/deployment.

JAUST: Japan US Search Tool

The detection and classification of passive sonar acoustics is a challenging problem faced by surface, sub surface, and naval air assets. The potential benefit of machine learning systems to assist in this task is appealing. However, little work has been conducted to develop and test machine learning models for this type of data or task. This research presents a custom Convolutional Neural Network (CNN) model designed specifically for underwater acoustic classification. This model is compared to several common CNN architectures on two datasets of hydrophone recordings of passing ships. These datasets are some of the largest datasets of ship recordings used for training CNNs to date, composed of over 4000 hours of recordings and over 400 hundred unique ships. The custom CNN shows marked improvement in performance over standard models in both multi-class and multi-label classification tasks. To the best of our knowledge, this research is the first demonstration of multi label classification on underwater soundscapes. This research demonstrates the capability of machine learning models to enhance human and unmanned systems operating in the undersea domain. This research has direct applications to both current and future Navy platforms.

Automatic detection and classification of passive sonar using Convolutional Neural Networks

Social media platforms, due to the prevalence, have become staging grounds for social warfare as antagonistic agents use social media to deceive large segments of population. Of specific concern are the coordinated campaigns orchestrated using groups of accounts to propagate false narratives. It has been observed that the coordinated campaigns by groups of social media users usually have common underlying emotions. As part of this research, we have used state-of-the-art deep learning Transformer models to extract fine grained emotions from users’ social media posts which we subsequently use to perform community detection to identify coordinated campaigns. We developed and tested our proposed methodology on a set of real-world Reddit posts. Our approach shows promise in using such a methodology to detect groups of similar users based on the emotion expressed in their social media posts. The details of our approach are described next. To start the analysis, we collected a set of Reddit public posts on a topic. Next, we used sophisticated emotion extraction to enrich the text of user social media posts. Our post emotion classification annotates social media posts and users with 27 fine-grained emotions such as optimistic, prideful, relieved, and gracious. This model can achieve state-of-the-art performance, because it uses the robust Transformer architecture that is initialized with a language learning task. Through successive layers of contextualization, the model can adapt to any natural language task. The model is first trained on a missing word task where a word in a sentence is masked and the model must predict the missing word. By doing this task on 11,000 books and the entirety of Wikipedia, the model has a strong, task-independent knowledge of the English language. Once the difficult part of teaching the model English is complete, we fine-tune the model to our task to produce a robust emotion tagging model. Once the posts are enriched and tagged with their emotion details, we use our community detection module to form communities around the extracted emotions based on social media behavior. To do this, we developed an embedding-based approach where users and emotions are translated into vector space embeddings and we use advanced mathematical operations to connect users to emotions. This allows us to identify group members in ways that go beyond plain keyword match. We extended out work to further identify influential users within a community. The process of detecting influential users is performed through a PageRank-like algorithm where connections between posts and users are leveraged to identify the key posts and users in a community. Initial experiments have shown promising results. Our emotion detector can accurately extract emotion information from posts and users. Our community detection algorithm can identify key subcommunities in a set of posts. Our influential users/post detection scores correlate strongly with Reddit social media indicators of influence.

Social Media Community Detection based on User Emotion

Bayesian optimization is a well-developed machine learning algorithm for black box optimization. In Bayesian optimization a predictive model, here a Gaussian process regression, is implemented to predict the black box function and the uncertainty of the predictive model is evaluated as predicted variance, then an acquisition function defined by the surrogate predictive model decides where to sample next. In this study we applied this technique to known ferromagnetic thin film materials such as ferromagnetic (Fe100−y Gay)1−xBx (x=0–21 & y=9–17) and (Fe100−y Gay)1−xCx (x=1–26 and y=2–18) to demonstrate optimization of structure-property relationships, specifically the dopant concentration or stoichiometry effect on magnetostriction and ferromagnetic resonance linewidth. Our results demonstrated that Bayesian optimization and regression analysis can be implemented to optimize structure-property relationships in FeGaB and FeGaC thin films. We have shown that using Bayesian optimization methods to guide experiments, the number of samples required to statistically determine the maximum or minimum was reduced by 50% compared with traditional methods. Our results suggest that Bayesian optimization can be used to save time and resources to optimize ferromagnetic films. This method is transferrable to other ferromagnetic material structure-property relationships, hence provides an accessible implementation of machine learning to magnetic materials optimization.

Application of Bayesian Optimization to Ferroic Materials Development

The objective of this work is to use machine learning to approach near real-time computation of optical propagation for high-energy lasers (HELs) under a variety of configurations and operational environments. We trained a Predictive RNN on synthetic time-series data in order to predict future states of the optical field as generated by time evolving Kolmogorov style turbulence.

Machine Learning in Optical Propagation toward Real-Time Predictive Capabilities for Navy Relevant Applications

Continuous investments in platforms capable of taking multispectral images of precipitation from space have led to consistent improvements in the accuracy of satellite-derived passive microwave (PMW) large-scale precipitation products. However, these platforms currently demonstrate a limited ability to relate observed radiometric signatures to precipitation type (convective and stratiform) and associated precipitation rate variability. Hence, PMW precipitation retrievals are prone to large systematic errors at instantaneous scales. Current research has demonstrated that deep learning neural network models (DNNs) can retrieve the convective type in precipitating systems from PMW observations with greater
accuracy than conventional methods. This study builds upon previous research in this area by presenting a Bayesian deep learning approach to extracting the information content from PMW observation vectors to classify precipitation types. A Bayesian convolutional neural network model (BCNN) is developed to retrieve the convective type in precipitating systems from PMW observations and output
the uncertainty in its predictions. A 12-month period of Global Precipitation Measurement Mission Microwave Imager (GMI) observations is used as a data corpus for model development and verification. The proposed BCNN model is shown to accurately predict precipitation types for 92% of total
precipitation volume, while outputting increased model uncertainty in regions of ambiguous or unknown data. Additionally, BCNN predictive accuracy is shown to improve when the predictions are screened via entropy filtering. Based on comparisons against currently used convective schemes and traditional DNN performance, it is concluded that the BCNN approach has the potential to address regime-specific PMW satellite precipitation biases affecting GPM operations.

Premium content

Exploring Graph Convolutional Network Reinforcement Learning for Cyber Defense

Next from NAML 2021

D2You: Data Driven User Emulator

Similar lecture

FARLAND (Framework for Advanced Reinforcement Learning for Autonomous Network Defense)

Premium content

Exploring Graph Convolutional Network Reinforcement Learning for Cyber Defense

Next from NAML 2021

D2You: Data Driven User Emulator​

Similar lecture

FARLAND (Framework for Advanced Reinforcement Learning for Autonomous Network Defense)

D2You: Data Driven User Emulator