United States

In the pursuit of Artificial General Intelligence (AGI), Machine Learning (ML) has achieved immense progress by exploiting parallel classical computing. Because of this success, our reliance on these techniques in autonomous systems has exponentially grown. However, throughout the scientific community there is growing concern of our increasingly rapid approach towards the theoretical limits of classical computation, better known as the end of Moore&#39;s Law. To continue these advancements, the adaptation and exploration of novel computational platforms is absolutely necessary. Biologically accurate, energy efficient neuromorphic systems that mimic the incredible efficiency of the human mind and the theoretically powerful optimization potential of fully entangled quantum systems are particularly promising arenas for enabling future advances. In this work, we compare these two flavors of novel substrate in detail by applying them to an identical challenge, and benchmark their performances in power consumption, time to solution, and reconstruction error against a classical CPU approach.
Our focus is on a subset of Hopfield networks, specifically, sparse attractor networks in which a uniform applied field is used to globally suppress activity. This encourages solutions consisting of a minimum number of active elements. We solve this sparse coding problem using the biologically inspired Locally Competitive Algorithm (LCA) on the D-Wave quantum annealer and the Intel Loihi neuromorphic spiking processor in two distinct ways, and compare the solutions to those found classically using Orthogonal Matching Pursuit (OMP). The Fashion-MNIST data set was chosen and dimensionally-reduced by sparse Principal Component Analysis (sPCA) under the constraint that a graph-based clustering metric (Henze-Penrose) remain unchanged. This enables the problem to be mapped identically to both devices. A sign flipped second data set was appended to the original to give each class a mean zero distribution, effectively creating an environment where the data could not be linearly separated. An early-in-time normalization technique for Loihi is presented along with analysis of optimal parameter selection and unsupervised dictionary learning using a local Hebbian rule for all variations. Studies are ongoing, but preliminary results suggest each computational substrate requires casting the NP-Hard optimization problem in a slightly different manner to best capture the individual strengths of the substrates, and the new Loihi method allows for a more realistic comparison with the D-Wave, while also drastically reducing the time needed to compute the solution with statistically improved reconstructions. Both substrates trail the classical approach to this problem in terms of reconstruction error, but D-Wave is the only one performing true binary sparse coding and Loihi provides immense energy savings along with viable solutions for both the full simulation and, especially, the early in time normalization variation.
The development of these substrates, and their use in expanding the machine learning parameter space, has multiple applications to Naval operations. For NP-Hard problems, our development of practical applications for quantum annealers lends itself to centralized decision making. For processing at the tactical edge, our progress in characterizing rapid neuromorphic solutions at low power are important in determining performance envelopes.


NAML 2021

Brain Inspired Quantum and Neuromorphic Machine Learning

brain

quantum

neuromorphic

[![](https://assets.underline.io/uploads/markdown_image/1/image/985c8a40b04e56675221c8549975de3c.png)](https://drive.google.com/file/d/1B70nPIeeeqdG4UU70NlYXuKVOHLRuJEo/view?usp=sharing)

### ** History of DRC**
The Device Research Conference (DRC) began in the pre-transistor era with the 1942 establishment of the Conference on Electron Device Research (CEDR). Following William Shockley’s 1948 CEDR presentation of the transistor device, the Solid-State Device Research Conference (SSDRC) was established, with invitation-only attendance.

In the ensuing decades, many exciting firsts were presented at SSDRC, including the use of impurity diffusion in silicon and germanium, the first practical MOSFET, and the first lattice-matched heterojunction.

In 1969, under the leadership or Herbert Kroemer and Cal Quate, the CEDR and SSDRC merged into DRC. In 1970, attendance was opened to all, and the first DRAM (dynamic random-access memory), room temperature lasers, and first CCD (charge-coupled device) were reported—all in the same 1970 DRC meeting!

1976 saw the first official coordination of tandem meetings between DRC and the Electronic Materials Conference (EMC) to bring together the electronic materials and device communities—a custom which persists to this day. In 1991, DRC began publishing its digest with IEEE; they can all be found [here on IEEE Xplore](https://ieeexplore.ieee.org/xpl/conhome/1000199/all-proceedings). Throughout this period, DRC featured reports of many novel and record-setting devices, including the HEMT (High Electron Mobility Transistor), VCSEL (Vertical Cavity Surface Emitting Laser), HBT (Heterojunction Bipolar Transistor), and nanoscale MOSFETs.

In recent years, DRC has continued to be the world’s premier forum to present innovative semiconductor device research with work on organic thin film transistors, graphene and carbon nanotube transistors, and 2D materials all featuring prominently in the 1990s through to the present day. In 2017, we celebrated DRC’s 75th anniversary with special sessions and an [article in the Journal of the Electron Device Society](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8168249&isnumber=8232486). DRC technical topics continue to evolve with emerging device trends such as memory devices, quantum devices, optoelectronics/photonics, and wide bandgap power devices to reflect the exciting work being done by a diverse community of device engineers around the globe.

Today, as we look forward to the next 75+ years of device innovation, DRC continues to lead the way in showcasing top-caliber research, facilitating vibrant discussion and debate and acting as a platform to nurture the next generation of device engineers. We welcome you to join us at the next DRC!

To learn more about DRC's history, please see [Aaron Franklin, Debdeep Jena and Deji Akinwande, "75 Years of the Device Research Conference—A History Worth Repeating," in IEEE Journal of the Electron Devices Society, vol. 6, pp. 116-120, 2018. doi: 10.1109/JEDS.2017.2780778](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8168249&isnumber=8232486).

### **Topics**
* Biological Devices
* Devices for Extreme Conditions
* Heterogeneously Integrated Devices
* Memory Devices
* Modeling and Simulation of Devices
* Nanoscale and Vacuum Devices
* Optoelectronic and Optical Devices
* Other Emerging Devices
* Power Devices
* Quantum Devices
* RF and Terahertz Devices
* Spintronic and Magnetic Devices
* Thin-film Devices
* Wide-bandgap Devices
 
### **Invited Speakers**
* Patrick Fay, University of Notre Dame
* Susanne Hoffmann-Eifert, Peter Grünberg Institute
* Sanghyeon Kim, Korea Advanced Institute of Science and Technology
* Sriram Krishnamoorthy, University of California, Santa Barbara
* Mario Lanza, King Abdullah University of Science and Technology
* Xiuling Li, The University of Texas at Austin
* Marko Radosavljevic, Intel Corporation
* Siddharth Rajan, The Ohio State University
* Giordano Scappucci, Delft University of Technology
* Fisher Yu, University of Arkansas
* Wenjuan Zhu, University of Illinois at Urbana-Champaign




Registration is required to attend this event. If you have not already done so, please use the link below to register for a ticket to the conference:

https://www.mrs.org/drc-2022/registration

Registration Is Required

DRC 2022

DRC continues to be the world’s premier forum to present innovative semiconductor device research with work on organic thin film transistors, graphene and carbon nanotube transistors, and 2D materials all featuring prominently in the 1990s through to the present day. In 2017, we celebrated DRC’s 75th anniversary with special sessions and an article in the Journal of the Electron Device Society. 
DRC technical topics continue to evolve with emerging device trends such as memory devices, quantum devices, optoelectronics/photonics, and wide bandgap power devices to reflect the exciting work being done by a diverse community of device engineers around the globe.

In the pursuit of Artificial General Intelligence (AGI), Machine Learning (ML) has achieved immense progress by exploiting parallel classical computing. Because of this success, our reliance on these techniques in autonomous systems has exponentially grown. However, throughout the scientific community there is growing concern of our increasingly rapid approach towards the theoretical limits of classical computation, better known as the end of Moore's Law. To continue these advancements, the adaptation and exploration of novel computational platforms is absolutely necessary. Biologically accurate, energy efficient neuromorphic systems that mimic the incredible efficiency of the human mind and the theoretically powerful optimization potential of fully entangled quantum systems are particularly promising arenas for enabling future advances. In this work, we compare these two flavors of novel substrate in detail by applying them to an identical challenge, and benchmark their performances in power consumption, time to solution, and reconstruction error against a classical CPU approach.
Our focus is on a subset of Hopfield networks, specifically, sparse attractor networks in which a uniform applied field is used to globally suppress activity. This encourages solutions consisting of a minimum number of active elements. We solve this sparse coding problem using the biologically inspired Locally Competitive Algorithm (LCA) on the D-Wave quantum annealer and the Intel Loihi neuromorphic spiking processor in two distinct ways, and compare the solutions to those found classically using Orthogonal Matching Pursuit (OMP). The Fashion-MNIST data set was chosen and dimensionally-reduced by sparse Principal Component Analysis (sPCA) under the constraint that a graph-based clustering metric (Henze-Penrose) remain unchanged. This enables the problem to be mapped identically to both devices. A sign flipped second data set was appended to the original to give each class a mean zero distribution, effectively creating an environment where the data could not be linearly separated. An early-in-time normalization technique for Loihi is presented along with analysis of optimal parameter selection and unsupervised dictionary learning using a local Hebbian rule for all variations. Studies are ongoing, but preliminary results suggest each computational substrate requires casting the NP-Hard optimization problem in a slightly different manner to best capture the individual strengths of the substrates, and the new Loihi method allows for a more realistic comparison with the D-Wave, while also drastically reducing the time needed to compute the solution with statistically improved reconstructions. Both substrates trail the classical approach to this problem in terms of reconstruction error, but D-Wave is the only one performing true binary sparse coding and Loihi provides immense energy savings along with viable solutions for both the full simulation and, especially, the early in time normalization variation.
The development of these substrates, and their use in expanding the machine learning parameter space, has multiple applications to Naval operations. For NP-Hard problems, our development of practical applications for quantum annealers lends itself to centralized decision making. For processing at the tactical edge, our progress in characterizing rapid neuromorphic solutions at low power are important in determining performance envelopes.


poster

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

MetaTagger is an accredited GOTS ML test and evaluation toolset designed to streamline ground-truth data labelling and performance testing of imagery and Full Motion Video (FMV) based models. MetaTagger accelerates model testing to increase performance transparency, user trust, and model adoption. A smart collection of tools enable fast image labeling for training data purposes. Our custom-built tool called AutoTagger uses a Single Object Tracking (SOT) algorithm to automate object detection, labeling, and creation of ground truth datasets in less than half the time of manual approaches. Once Ground truth is created, FVM inference can be evaluated in minutes by simply uploading inference to the application and letting the MetaTagger algorithm work in the background. Once evaluation is completed, a filterable reporting tool allows custom reports to be generated and exported for future analysis. MetaTagger is the one-stop-shop for testing and evaluation of object detection and object classification algorithms.

MetaTagger - Computer Vision Testing & Evaluation Toolset

We approach the problem of image captioning with Cycle-Consistent Generative Adversarial Networks (CycleGANs). Due to CycleGANs' ability to learn functions to map between multiple domains and use duality to strengthen each individual mapping with the usage of a cycle consistency loss, these models show great promise in their ability to learn both image captioning and image synthesis and to create a better image captioning framework. Historically, cycle consistency loss was based on the premise that the input should undergo little to no change when mapped to another domain and then back to its original; however, image captioning presents a unique challenge to this concept due to the many-to-many nature of the mapping from images to captions and vice-versa. We have combined the state-of-the-art image synthesis model from Hao Zhang, StackGAN, with a modified version of Kelvin Xu's Show, Attend, and Tell image captioning model in a CycleGAN architecture to create a robust image captioning framework that is able to create diverse captions for each input image. Due to the duality of CycleGAN, we will also explore our model's capability to successfully generate relevant images based on an image description to test its ability to perform the opposite task and observe CycleGAN's ability at learning mappings between two domains from different mediums. We will be discussing our model's performance on the following datasets: Oxford Flowers, Caltech-UCSD Birds, and Microsoft Common Objects in Context (MSCOCO) datasets.

TextCycleGAN

With the vast amounts of unstructured data that is accrued through surveillance, body-cameras, mobile phones, humanitarian-aid and disaster relief, etc., there is a need to perform data synthesis to structured outputs through automated methods. Recent advances in machine learning have enabled compression of GBs/TBs of data sequences into short, compact, informal summaries as keyframes and/or video thumbnails and the capability to generate text that describes an overall image or full motion video has also rapidly increased. However, generating structured text beyond informal summaries, remains a relatively unsolved area of Natural Language Generation. This work will address this gap in the data summarization and document generation problem. This research would be the final component to an (non-yet existent) end-to-end system that could ingest data, summarize it, and generate a structured report.

Automating Situational Reporting

The presentation illustrates where machine learning applications built using transfer learning can be attacked, the means for carrying out the attack and some mitigations that can be employed. The elements in building and deploying a machine learning application are reviewed, considering both data and processes. Mitigations to these attacks are discussed with the engineering tradeoffs between security and accuracy. Finally, the methods by which an attacker could get access to the machine learning system are reviewed.

Threats to Transfer Learning Applications

The challenge of applying artificial intelligence to autonomous network security was investigated using Graph Convolutional Network Reinforcement Learning Methods in a dynamically-scaling computer network. A deep Q-network single agent method was compared to a hybrid method, which integrated a larger agent strategizing for the entire network with individual agents working on each node. The simulation network contained 'real' machines, to be protected, and 'decoy' machines, to act as deception mechanisms. An attacker pivoted through the network and the agents’ defensive strategies were tracked to determine which version adapted the best. These algorithms are being explored for this problem because of the changing action space along with the ability to adapt to the evolving network topology. 


Exploring Graph Convolutional Network Reinforcement Learning for Cyber Defense

Unlike current user emulators, D2U does more than simply replay previously-observed behavior, or call simple heuristics for stringing together patterns of actions. Instead, D2U provides the ability to generate unlimited, novel sequences of realistic behavior and orchestrate a plethora of virtual machines to enact these behaviors, thereby emulating a network of users. This ability to generate realistic, new behaviors sets D2U apart from all existing emulation technologies either in the academic literature or on the market.

D2You: Data Driven User Emulator​

Macro-meteorological models predict optical turbulence as a function of weather data. Existing models often struggle to accurately predict the rapid fluctuations in Cn2 in near-maritime environments. Seven months of Cn2 field measurements were collected along an 890 m scintillometer link over the Severn River in Annapolis, Maryland. This time series was augmented with local meteorological measurements to capture bulk-atmospheric weather measurements. The prediction accuracy of existing macro-meteorological models was analyzed in a range of conditions. Next, machine-learning techniques were applied to train new macro-meteorological models using the measured Cn2 and measured environmental parameters. Finally, the Cn2 predictions generated by the existing macro-meteorological models and new machine-learning informed models were compared for four representative days from the data set. These new models, under most conditions, demonstrated a higher overall Cn2 prediction accuracy, and were better able to track optical turbulence. Further tuning and machine learning architectural changes could further improve model performance.

Optical Turbulence Prediction from Data-driven Models

The U.S. military has been implementing simulation-based training systems in an effort to provide more effective training programs with reduced time to train. Integrating emerging technologies such as virtual, augmented, and mixed reality into their training curriculums has become the expectation. Over the past couple of years, the military has focused its attention on future threat training by developing multi-domain environments for more experienced warfighters. This includes Live, Virtual, and Constructive (LVC) assets for operations in complex, dynamic, and uncertain battlespaces.

The Department of Defense (DoD) is investing in a Joint All-Domain Command and Control (JADC2) concept that strives to connect multiple domains (air, land, sea, space, and cyber) into a single interoperable network (Congressional Research Services, 2020). Developing an interoperable Synthetic Training Environment is
challenging and once successful, the amount of data collected in a JADC2 type of environment can be overwhelming. The DoD is working across services to determine how best to manage the data. This presentation describes and illustrates an architecture for analyzing measurable data for assessing warfighter performance that can be collected during training in a natural environment. This data architecture is open, free, portable, containerized, and scalable for real-time monitoring and feedback.

The success of JADC2 training is dependent on relevant and actionable data (Rosenberg, 2020). This presentation identifies relevant data for measuring training effectiveness that provides actionable data for use in training automation, personalization, and adaptation. Research conducted by Hoke et al. (2017) and Dideriksen et al. (2018) validates the methodology of assessing training effectiveness through the combination of task performance measures and physiological signals of cognitive workload. It concludes with a discussion of how this methodology and architecture can be applied to improve training outcomes for individuals and teams for modern warfare in a multi-domain battlespace.

Automated Human-Machine Performance Assessment for the Multi-Domain Battle Space

Rip channels are a relevant area of study for Navy oceanographers because of their violent nature and relationship to the ocean floor. Images of these channels can serve as viable data when on site methods of collecting data are not available. Manually annotating rip channels is cumbersome, especially since there may be thousands of images to analyze for adequate climatology information. Although it has been shown that machine learning can help automate this process, developing a general-purpose rip channel classifier is problematic due to both lack of prior labeled data and the disparate features of litoral sites. Additionally, the subjective nature of rip channels yields complexities in general-purpose approaches, mainly due to the complexity of basic feature correlation. We present an interactive machine learning (IML) rip channel annotation application that uses online machine learning, active learning, and uncertainty modeling to provide a human-machine team for the annotation process. The application is able to assist users with annotating rip channels in their own datasets without the need for prior labeled data or parameter tuning. The interface by which this is done consists of four vertices denoting the length and span of the rip channel. The application attempts to place the vertices, the user then confirms or corrects the placements, the IML immediately learns from the verified placement, and the process continues. The IML implementation also takes advantage of user-offline time to optimize hyper parameters, perform feature selection, and reduce features to maintain scaling. Four persistent-image datasets from Oregon State University’s Coastal Imaging Laboratory were selected for gathering data from 2 locations, namely: Duck, North Carolina and Secret Harbour, Australia. The tool’s performance is evaluated based on correct vertex placement by the IML implementation. Results show that over 80% of the rip channels are correctly annotated by the IML implementation with an optimization cycle of about 4 images per dataset without the need for any prior labeled data.

Interative Machine Learning for Rip Channel Annotation

JAUST (Japan US Search Tool) is a web and mobile cognitive search solution built for the US Forces Japan that enables users to quickly search and retrieve documents in historical digital archives. Physical archives (in paper or PDF format) were digitized and text was extracted using optical character recognition (OCR) and computer vision techniques. AI models (natural language processing, large language models, graph theory) are used to extract signatures, analyze, and tag documents before ingestion into an elasticsearch database. A lucene search engine is queried using a custom REST API to display results in a Graphical User Interface (web application) built with Angular. JAUST includes free text search and filtering capability (dates, locations, memo numbers) that retrieves the documents in order of relevance, highlights search terms, and displays original PDFs. JAUST is hosted in an AWS govcloud environment and includes continuous integration/deployment.

Premium content

Brain Inspired Quantum and Neuromorphic Machine Learning

Next from NAML 2021

MetaTagger - Computer Vision Testing & Evaluation Toolset

Similar lecture

An Experimentally Validated, Universal Memristor Model Enabling Temporal Neuromorphic Computation