United States

Bayesian optimization is a well-developed machine learning algorithm for black box optimization. In Bayesian optimization a predictive model, here a Gaussian process regression, is implemented to predict the black box function and the uncertainty of the predictive model is evaluated as predicted variance, then an acquisition function defined by the surrogate predictive model decides where to sample next. In this study we applied this technique to known ferromagnetic thin film materials such as ferromagnetic (Fe100−y Gay)1−xBx (x=0–21 &amp; y=9–17) and (Fe100−y Gay)1−xCx (x=1–26 and y=2–18) to demonstrate optimization of structure-property relationships, specifically the dopant concentration or stoichiometry effect on magnetostriction and ferromagnetic resonance linewidth. Our results demonstrated that Bayesian optimization and regression analysis can be implemented to optimize structure-property relationships in FeGaB and FeGaC thin films. We have shown that using Bayesian optimization methods to guide experiments, the number of samples required to statistically determine the maximum or minimum was reduced by 50% compared with traditional methods. Our results suggest that Bayesian optimization can be used to save time and resources to optimize ferromagnetic films. This method is transferrable to other ferromagnetic material structure-property relationships, hence provides an accessible implementation of machine learning to magnetic materials optimization.

NAML 2021

Application of Bayesian Optimization to Ferroic Materials Development

ferromagnetic resonance

magnetostriction

ferromagnetic materials

thin films

bayesian optimization

gaussian processes

regression

The Conference on Uncertainty in Artificial Intelligence (UAI) is one of the premier international conferences on research related to knowledge representation, learning, and reasoning in the presence of uncertainty. It is supported by the [Association for Uncertainty in Artificial Intelligence (AUAI).](https://www.auai.org/)

The 37th Conference on Uncertainty in Artificial Intelligence (UAI 2021) is held fully online. We have an exciting program consisting of keynote talks by [Susan Murphy](https://www.auai.org/uai2021/program?block=A), [Eric Horvitz](https://www.auai.org/uai2021/program?block=B), [Lenka Zdeborová](https://www.auai.org/uai2021/program?block=C), [Judea Pearl](https://www.auai.org/uai2021/program?block=D) and [Ankur Moitra](https://www.auai.org/uai2021/program?block=E), a town hall meeting, peer-reviewed papers, and three workshops. The main conference is single-track and each paper is show-cased with a live component (either a live discussion and Q&A, or a live lightning talk). All papers are also featured in video presentations and in poster sessions in gather.town. The three parallel workshops on Friday allow for smaller meetings on current topics.

The walkthrough video below shows how you can navigate this site and use the different possibilities for interaction: UAIsland2021 on gather.town for posters and social interactions, the mementor platform for virtual mentoring sessions, and the chat and Q&A systems for discussions about talks and posters.

Have a great UAI 2021! 
<br>
<br>Cassio de Campos and Marloes Maathuis
<br>UAI 2021 Program Chairs
<br>uai2021programchairs@gmail.com

<figure class="video_container">
  <iframe src="https://screencast-o-matic.com/watch/cri0lsViAGK?v=6&ff=1&title=0&controls=1" width=526  height=226  frameborder="0" allowfullscreen="true"> </iframe>
</figure>


A ticket is required to access this conference. Please use the provided link to register for the event.

https://auai.org/uai2021/registration

UAI 2021

The Conference on Uncertainty in Artificial Intelligence (UAI) is one of the premier international conferences on research related to knowledge representation, learning, and reasoning in the presence of uncertainty. It is supported by the [Association for Uncertainty in Artificial Intelligence (AUAI).](https://www.auai.org/)

Bayesian optimization is a well-developed machine learning algorithm for black box optimization. In Bayesian optimization a predictive model, here a Gaussian process regression, is implemented to predict the black box function and the uncertainty of the predictive model is evaluated as predicted variance, then an acquisition function defined by the surrogate predictive model decides where to sample next. In this study we applied this technique to known ferromagnetic thin film materials such as ferromagnetic (Fe100−y Gay)1−xBx (x=0–21 & y=9–17) and (Fe100−y Gay)1−xCx (x=1–26 and y=2–18) to demonstrate optimization of structure-property relationships, specifically the dopant concentration or stoichiometry effect on magnetostriction and ferromagnetic resonance linewidth. Our results demonstrated that Bayesian optimization and regression analysis can be implemented to optimize structure-property relationships in FeGaB and FeGaC thin films. We have shown that using Bayesian optimization methods to guide experiments, the number of samples required to statistically determine the maximum or minimum was reduced by 50% compared with traditional methods. Our results suggest that Bayesian optimization can be used to save time and resources to optimize ferromagnetic films. This method is transferrable to other ferromagnetic material structure-property relationships, hence provides an accessible implementation of machine learning to magnetic materials optimization.

poster

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

The objective of this work is to use machine learning to approach near real-time computation of optical propagation for high-energy lasers (HELs) under a variety of configurations and operational environments. We trained a Predictive RNN on synthetic time-series data in order to predict future states of the optical field as generated by time evolving Kolmogorov style turbulence.

Machine Learning in Optical Propagation toward Real-Time Predictive Capabilities for Navy Relevant Applications

Continuous investments in platforms capable of taking multispectral images of precipitation from space have led to consistent improvements in the accuracy of satellite-derived passive microwave (PMW) large-scale precipitation products. However, these platforms currently demonstrate a limited ability to relate observed radiometric signatures to precipitation type (convective and stratiform) and associated precipitation rate variability. Hence, PMW precipitation retrievals are prone to large systematic errors at instantaneous scales. Current research has demonstrated that deep learning neural network models (DNNs) can retrieve the convective type in precipitating systems from PMW observations with greater
accuracy than conventional methods. This study builds upon previous research in this area by presenting a Bayesian deep learning approach to extracting the information content from PMW observation vectors to classify precipitation types. A Bayesian convolutional neural network model (BCNN) is developed to retrieve the convective type in precipitating systems from PMW observations and output
the uncertainty in its predictions. A 12-month period of Global Precipitation Measurement Mission Microwave Imager (GMI) observations is used as a data corpus for model development and verification. The proposed BCNN model is shown to accurately predict precipitation types for 92% of total
precipitation volume, while outputting increased model uncertainty in regions of ambiguous or unknown data. Additionally, BCNN predictive accuracy is shown to improve when the predictions are screened via entropy filtering. Based on comparisons against currently used convective schemes and traditional DNN performance, it is concluded that the BCNN approach has the potential to address regime-specific PMW satellite precipitation biases affecting GPM operations.

Quantifying Uncertainty in Multispectral Satellite Image Classification using Bayesian Deep Learning

Deep Convolutional Neural Networks (CNN) are proven to be effective in a variety of classification and regression tasks including underwater soundscape classification. Underwater soundscape classification provides the potential for increased automation and performance for shipboard and autonomous UUV contact detection systems. Although CNN’s are a powerful technology, they have been shown to be vulnerable to adversarial attacks across different input domains. Adversarial attacks function by creating a change to the input that is imperceptible to a human observer but causes the classifier to incorrectly classify the input example. This type of attack is possible on underwater soundscape classification systems. A common approach to audio classification is to transform source audio into spectrogram images to use for classification. A digital attack that intercepts the spectrogram images prior to being input to the classifier is a possible vector of attack. Several established image-based adversarial example generation methods are tested for this purpose. Five methods successfully fooled the target classifier 31% of the time with an epsilon value of 0.005 and over 80% of the time when the epsilon value was increased to 0.01. A more reliable method of attack is to provide the adversarial example as audio input to the system. Such an attack is more desirable than a digital image-based attack for two reasons. An audio-based attack does not make any assumptions on the inner-workings of the classification system (i.e., using spectrogram images as features) and have more potential to be deployed in a physical space. Generating adversarial audio is generally more difficult than adversarial images. This is because humans are especially sensitive to small disturbances in audio that could be caused by delays due to generation time or artifacts from the generation process. This work aims to overcome these obstacles by using a training method that is similar to the training methods used by Generative Adversarial Networks (GAN). Various GAN architectures have been used to create realistic source audio. The training method differs from those used in training GANs by only training the generator network, not both the generator and discriminator. Instead, the target classifier is used in as a static discriminator. Use of the target network as a discriminator makes this a black box attack on the network. Additionally, proposed usage of perceptual evaluation metrics in the loss function, the quality of the generated audio perturbation is increased and the probability of detection by a human observer is reduced. A generative approach decreases the time to create a single adversarial example compared to other audio generation techniques, which rely on an iterative optimization approach. Improved generation speeds further decrease the likelihood of detection by a human observer and increases the potential for the attack to be deployed in a physical space.

Adversarial Attacks on Underwater Soundscape Classifiers

Publiciy available program acquisition, contracting and award data commonly include many variables per observation. In some cases, the number of variables is greater than the number of observations. This poster summarizes a method to efficiently select multivariate regression predictor variables using the R VSURF random forest package to identify important varaibles prior to applying traditional fitting methods.

Modeling Defense Acqusitions: simplifying regression predictor selection with random forests

Machine learning algorithms embedded in deep level NN (Neural Networks) require large amounts of data that are often unavailable in many existing radar receivers. The proposed NN approach provides a condensed form for training, testing, and processing radar data. The test setup uses synthetically generated pulses from a time of arrival generator. It then derives a rate of time interval change between pulses. These slopes are used for NN training and testing. A total of 20 synthetic signals were used with an 80/20 split for training and testing. The results provide a perspective on the performance with actual emitters, demonstrating an ability to distinguish the Pulse Repetition Interval (PRI) type between single Electromagnetic (EM) wave generators. Synthetic results also show an ability to differentiate between single EM wave generators and multiple EM wave generators. Future work will expand to distinguish between PRI types from multiple generators received simultaneously.

Peak to Peak NN: Condensing Electromagnetic Wave Characteristics for Neural Networks

The US Submarine Navy relies on SONAR for detection and classification of contacts to support navigation and other mission related tasks. The field of deep learning (DL) could be useful in this problem space given that it is the current state of the art approach for image and speech recognition. Supervised learning however is a data hungry process and many Navy problems have limited labeled data. An underwater acoustic classification problem that is likely to have enough labeled samples to investigate the application of DL is recognizing marine mammals from individual clicks. The objective of this project is to apply DL to underwater acoustic transient classification and determine how much training data is required for a DL classifier to compete with an existing baseline approach. This is a collaborative effort between the Naval Undersea Warfare Center (NUWC) Division, Newport, RI (NUWCDIVNPT) and the Naval Surface Warfare Center Carderock Division (NSWCCD). The NUWC research team was able to use a one dimensional convolutional neural network (1D CNN) to ingest raw time-series data previously isolated by a power law detector and achieve classification accuracy as high as 93.7% with minimal optimization of hyper-parameters. As expected, classification accuracy increased as more training data were used. Further analysis into the performance of the network raises concerns on generalization because classification accuracy is not consistent across class or by recording location of sample data. Findings from fiscal year 2020’s efforts highlight the importance of scrutinizing classifiers that show overall good performance, especially in deep learning applications where intractability prevents intuitive awareness of knee cases.

deepBioNet: Acoustic Transient Classifier

Passive identification of underwater sound sources is critical for a variety of applications including surveys of marine mammals and maritime domain awareness. Currently, the Navy can collect large sets of underwater acoustic signals. The cost and challenge lies in correctly labeling the data, which is time consuming and requires subject matter experts. The advent of machine learning makes the labeling process ripe for automation into a semi-supervised workflow. We begin with a labeled set of recordings released by the Scripps Institution of Oceanography (DCLDE 2015 and 2018) from hydrophones placed offshore in Southern California and US East Coast. In DCLDE 2015, five species were identified by experts across three sites with seven hydrophones, providing substantial variability in context and acoustic content. The DCLCE 2018 data is similar, though with nine different species identified across a variety of East Coast sites. Here the intersection of these two data sets is the presence of the Risso’s Dolphin, allowing us to test a classifier for this class on both different backgrounds and with different groups of dolphins that generated the vocalizations. Given the variety of approaches that currently exist for classification, we explore the effect on labeling performance of three fundamentally different models: transfer learning with AlexNet (image processing and classification), an SVM with precomputed acoustic features, and Temporal Convolutional Networks (TCN) with processed and raw timeseries data. Specifically, we explore generalization of these methods (robustness to different backgrounds or placement contexts) when trained in one environment and tested in another. Additionally we studied background suppression strategies and the effect of the sample distribution used to train the model. We found that generalization of the TCN and SVM model was dependent on mitigating the background and providing the correct blend of labeled training data. AlexNet transfer learning performance was found to be consistent despite changes in class balancing and the three background treatments tested.

Automated classification for multiple species of echolocating whales

NAVWAR's Digital Transformation Learning Continuum Program supports the Commander's strategic goal to rapidly build, strengthen and exercise data analytics capabilities across the NAVWAR HQ and affiliated PEO workforces. The Program delivers NAVWAR-mandated training for all hands (Foundations for Data Citizens) and Supervisors (Foundations of Data Analytics). Additionally, the Program offers a direct, "hands-on" workforce experience with advanced analytics through a theory-to-practice Workshop (2-Day), Laboratory (Case Study) and Incubator (Competency/PEO-led Applied Analytics Projects) development progression.


NAVWAR Digital Transformation Learning Continuu

Source localization and characterization are important tasks of passive sonar systems. The variability of the ocean environment alters the propagation of the acoustic signal and therefore affects the overall accuracy of passive sonar if not accounted for. To achieve more accurate results, this study presents a deep learning approach to seabed classification using the noise from surface ships on a single hydrophone. Ship noise spectrograms are used as input for a convolutional neural network to classify between 34 types of seabeds. This work aims to determine the optimal time duration and frequency band of the ship noise spectrogram to predict seabed type with 3-layer convolutional neural networks. The performance using both 1-dimensional and 2-dimensional convolutional kernels is compared. The networks are trained on synthetic data created with a broadband source model for the ship noise and a range-independent, normal mode propagation model to simulate propagation in the shallow ocean for the 34 seabed classes and a wide range of ship speed and closest point of approach. Training is augmented with a broadband model for the wind noise. The neural networks are verified using five-fold cross validation, and then tested using data from a 2017 seabed characterization experiment. Seabed class predictions are compared to measure the impact of altering the time window and frequency band of the input spectrograms. Finding the optimal spectrogram size for use with these networks can improve the overall accuracy and efficiency of using surface ships for seabed classification and, thus, improve passive sonar. [Work Supported by the College of Physical and Mathematical Sciences, Brigham Young University]


Finding Optimal Spectrogram Length 
for CNN Seabed Classification from Surface Ship Noise

Digital maps are important in many naval applications for situational awareness and decision making. Attempts have been made to automate digital map generation, but these regularly fall short in quality and usefulness. For example, digital maps often contain labels that can get occluded at certain zoom levels or as new layers are added. Algorithms that optimize label placement are computationally complex, and automated solutions must rely on heuristic or statistical methods to choose how the labels are distributed. Completely automated solutions are often inferior to maps designed by trained experts and still require extensive editing by an analyst. To address this challenge, we describe an interactive map generation approach that leverages cognitive models and interactive machine learning to improve label placements over purely manual or automated solutions. 

Interactive machine learning (IML) creates a continuous loop of online feedback to improve a machine learning model. These approaches start with little to no data and iteratively improve through interactions with the user. This is useful for digital maps, where optimal label placements often depend on features that cannot always be clearly defined. Using IML, analysts can adjust labels in a map view, supplying the algorithm with feedback the algorithm can use to identify the relevant features and improve its recommendations over time. Due to digital maps having a large feature space, IML may require a great deal of analyst feedback before the generated maps are considered acceptable.

We propose using a cognitive model to train an IML algorithm, decreasing the amount of required time and effort needed from real users. Cognitive architectures, such as ACT-R, are commonly used in human-computer interaction research to simulate human behavior and to generate synthetic training data for machine learning. Cognitive architectures provide a framework for modeling behavior using established cognitive models of perception and memory. This is relevant in determining label placements, because an analyst's decision of where a label should be placed is based on perceptual information (i.e., the location and density of map colors and textures) and heuristic rules about when labels should be moved. A cognitive architecture can simulate an analyst's decision processes about label placements using cues such as saliency, spatial distribution, and color density. Using defined procedural rules, the architecture simulates an analyst iteratively identifying poorly placed labels or cluttered areas and then moving labels as needed. By using a cognitive model to simulate an analyst making label adjustments on a digital map, the IML algorithm can be initialized with some basic information about features important for generating good label placements, without taking an analyst's time. In this way, the system generates better initial recommendations, while still continuing to be fine-tuned by analysts going forward.

Premium content

Application of Bayesian Optimization to Ferroic Materials Development

Next from NAML 2021

Machine Learning in Optical Propagation toward Real-Time Predictive Capabilities for Navy Relevant Applications

Similar lecture

Nonmyopic Approach to Cost-Constrained Bayesian Optimization