United States

Knowledge graphs, or node-edge graphs representing entities and relationships are fundamental to a wide variety of cutting-edge Natural Language Processing (NLP) tasks including search, question-answering, salience determination, slot-filling, and event schema induction. Such tasks enable advanced analytics that allow users to derive new, meaningful insights about their data and are particularly useful in the realm of Big Data. Google’s Knowledge Graph, for example, forms the backbone of its search engine, enabling Google to make more intelligent inferences about results relevant to user queries. Knowledge graphs and their associated NLP tasks are widely applicable to large Navy text corpora including Navy missile standards documents, instrument logs, and cybersecurity and safety policy documents. 

The goal of this project is to build a Natural Language Processing (NLP) pipeline capable of ingesting a corpus of documents and automatically creating a Knowledge Graph containing the entities and relationships in the corpus. We are constructing an end-to-end pipeline composed of six modules: Coreference Resolution, Open Information Extraction, Canonicalization, Entity Linking, a Neo4j Graph Database, and a Graphical User Interface (GUI). 

We first perform Coreference Resolution (AllenNLP) on the corpus to create a single term for all expressions that refer to the same entity. We then extract all subject-predicate-object triplets using Open Information Extraction models by Stanford, AllenNLP, and the University of Washington and IIT Delhi; currently the user can choose a single model or a combination of models. Next we map each name to a canonical form, by clustering entities via cosine similarity, Jaccard similarity, or inverse document frequency. We then link our entities to existing entities with the same names in a pre-constructed knowledge base (KB), in effect ingesting all knowledge contained in the KB. Finally, we load the processed subject-predicate-object triplets into a Neo4j graph database, accessible to users via our GUI. 
The GUI provides an interface through which users can input text data, either by uploading files or typing raw text, choose pipeline modules to include and models and thresholds to use, query the database, run advanced analytics, produce interactive graph visualizations, and save data and visualizations. 

Through the aforementioned six modules, our project is an end-to-end pipeline of state-of-the-art methods for converting unstructured text data into a structured Knowledge Graph. 


NAML 2021

Automated Knowledge Graphs

graph database

neo4j

automation

natural language processing

knowledge graph

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=1280 height=706 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

poster

There is a critical need within the military to be able to identify outbreaks of communicable diseases early in order to be able to contain, mitigate, or track their spread. This is especially true when soldiers and personnel have to be kept in close quarters on a ship or in barracks. Wearable health monitoring devices provide the capability to record a variety of real time physiological time series data including, but not limited to: respiratory rate, respiratory rate variability, heart rate, steps, calorie intake and state of mind (or mental activity level). We hypothesize that these physiological measures may be used to predict the health status of individuals over the course of an acute respiratory infection (ARI) from healthy, infected, pre-symptomatic, symptomatic, and recovered. To test this hypothesis, we are using data collected using Spire HealthTag devices that were deployed during a community cohort contact tracing study conducted by the University of Maryland during the 2018-2019 influenza season. The objective of this study is to use the wearable data to predict the onset of acute respiratory infections due to viruses such as parainfluenza, coronavirus, adenovirus, influenza, and rhinovirus. We are using methods in time series analysis, topological data analysis, machine learning, and anomaly detection to develop predictors that link physiological variables collected from the Spire HealthTag device to disease onset based on perceived symptom scores and confirmed infections. A model being developed and its performance in predicting the onset of ARIs will be presented.

Learning Objectives

1. Understand the potential of wearable health data to predict the onset of illness.
2. Describe the model and our analytic approach.
3. Discuss this potential health monitoring system’s use in the field.

Predicting of Health Status from Wearable Physiological Monitoring Data (PreMonData)

PMW/ISEA teams are dealing with thousands of trouble tickets on a daily basis – currently the process is reliant on sailors or help desk self-assessing issue severity​. CASREP Prediction Model was develop to predict which Trouble Tickets are most likely to escalate into a CASREP or will take longer to resolve. This allows immediate identification of high-risk issues as soon as the trouble ticket is submitted. Enables team to prioritize response actions and resources, and potentially reduce system downtime. CASREP Prediction Model uses NAVWAR Help Desk ITSM data and five different AI/ML algorithms to work together to boost accuracy. 

The Trouble Ticket Duration model builds upon the framework of the CASREP Prediction Model but with the focus on predicting how long until a trouble ticket will be resolved. The Fleet will be potentially able to forecast downtime and adjust operational plans to compensate effectively. The PMW/ISEA teams are potentially able to use the forecast to prioritize tasking in resolving these trouble tickets.



Using Machine Learning to Predict CASREP Occurrence and Trouble Ticket Duration

Create synchronized data to train a machine learning algorithm to esimate turbulence in image data.

Atmospheric Turbulence Estimation from Degraded Image Data

We present several algorithms for vehicle routing problems that involve coordinated operations with a ground vehicle and a collection of drones.

Algorithms for drone-assisted mission planning problems

Patent pending for this big data collection and cleaning process. The sample data includes engineered features through various Navy databases. The data is then modeled in several predictive analytics algorithms, to include error metrics, training, and test sets. This then produces a list of submarine equipment needed to deploy that are forecasted to go obsolete in the next five years. These visual analytics are used by system leads and analysts to procure or prevent any shortfall. In turn, this encourages proactive triggers and money-savings for the fleet.

Forecasting Obsolescence Model Optimization

Recognizing that “garbage in” leads to “garbage out,” many organizations that employ machine learning and other types of data analytics strive for a zero-defect data ingestion process. While these efforts are critical, they are not infallible. Therefore, programs should adopt a multi-layered approach to risk management that includes mitigation plans for addressing issues that occur after faulty data has corrupted data analytic efforts and potentially influenced program-level decisions.

The impetus for what would become the Quality Risk Management Framework was a planned change to an analytics environment’s data ingestion process. The authors realized that the upcoming changes would likely increase the risks relating to data management. The authors also realized that these risks might not be discovered until after that faulty data had been used in subsequent analysis and decision support efforts, including those that may impact the safety of personnel. With these potential issues in mind, the authors performed a literature review in the hopes of finding an existing framework that addressed their concerns. While the review did not uncover a suitable framework, it did uncover elements that the authors would later incorporate into their Quality Risk Management Framework.

The authors’ resulting framework takes the form of a matrix where the columns relate to six quality dimensions and the rows relate to the six phases of the intelligence lifecycle. Using this matrix, the authors were able to identify 36 unique areas where quality risks could arise. Then, the authors developed mitigation plans for each area of potential danger. With these mitigation plans established, the authors are confident their program will be able to rapidly respond to the discovery of a quality issue regardless of when the program discovers it.

What if we were wrong? – Creating and Applying a Quality Risk Management Framework

Over the years, the use of data throughout the V-22 Osprey program steadily grew. Initially, automated software analyzed flight data to inform maintenance decisions. A few years later, analysts mined maintenance data to inform program-level decisions. Then, a few years after that, analysts leveraged artificial intelligence and machine learning on dozens of different V-22 data sets to gain deep insights on aircraft readiness. While each of the three program areas described above was successful, their roadmaps and maturation plans were independent until now. This presentation describes how overlaying a decision framework on the “big picture” enabled system-level optimization of the formerly siloed initiatives.

In the 1950s, John Boyd, an instructor at the Air Force’s Fighter Weapons School, realized that the pilot who made decisions faster than his opponent would win the aerial dogfight. Studying this further, Boyd realized that decision-making was a continual progression through four activities: observe, orient, decide, and act. Boyd subsequently named this decision-making process the OODA Loop. When the authors of this presentation took a system-level view of the V-22 maintenance support infrastructure, they discovered those same four elements were also critical factors in their program's success. They also realized, as did Boyd, that their goal should be to get through the decision-making cycle as quickly as possible.

With this system-level view and a goal of optimizing the OODA Loop, the authors examined the existing technologies and capabilities supporting each of the four decision-making activities. Next, they identified what was necessary to accelerate the program’s progression through the OODA Loop. Then, they set out to fill those gaps. As these new capabilities are introduced, the program’s decision-making system will become increasingly optimized. As a result, the V-22 program will gain maximum benefit from its machine learning and other data analytic efforts.

Applying a Decision Framework to Increase Machine Learning Impact on the V-22 Osprey Program

Naval Officers at the Office of Naval Research Reserve Component (ONR-RC) have developed a prototype taxonomy for Artificial Intelligence (AI), focused on Naval and DoD audiences and applications. Our taxonomy establishes 4 pillars of AI: Data, Mechanism, Function, and Naval Task. Sub-taxonomies within these pillars are based on authoritative resources from organizations such as NASA and the Association for Computing Machinery (ACM). We posit that any AI system can be decomposed and classified according our structure, which mitigates problems of mutual exclusivity that other taxonomies suffer. Organizations like ONR can use taxonomy to organize and shape their AI research portfolio. Furthermore, we hypothesize that concrete proposals for future AI systems can be developed by constructing their Data, Mechanism, Function, and Task components. This hypothesis is currently being tested by students at the Naval War College, who will write such proposals for the ongoing course AI for Strategic Leaders. We offer this prototype taxonomy for feedback and criticism, so that we may continue to improve its utility for wider DoD audiences.

Taxonomy of Artificial Intelligence

The objective of Self-Localization and Intelligent Mapping (SLIM) research is to expand the Navy's AUV capabilities through artificial intelligence (AI) to static undersea sensors and/or dynamic undersea groups, improve autonomous perception and data fusion, and generate world models from individual sensing while localizing the sensors and UUV swarm within the model. In the current study, we present the performance of the proposed method in a networked robotic scenario using robot operating system (ROS) Gazebo for simulation of physical environment.

A Novel Hybrid Method for Self-Localization and Intelligent Mapping of AUVs

NSWC Corona is developing the Perceptive Engine with Natural Language Processing (peNLP) application with the vision of enabling the digital transformation of documentation and processes, and bringing advanced data analytic capabilities to analysts’ fingertips. Its intuitive interface will include: displaying all documentation in one searchable wiki-like location with insightful metadata, tracing of concepts through related files, comparing and contrasting within the corpus to reduce redundancy and discover conflicts, and a better overall understanding of the documents through in-depth analyses and pattern recognition algorithms (i.e. clustering, changing word meaning over time, metrics) for future integration across different data sources (i.e. logistics, specifications, maintenance, RMA, funding). 

The peNLP application has made rapid advancements in its user interface in integrating NLP capabilities with automated Python scripts. With both “Document Search” and “Document Knowledge Graph” views, peNLP utilizes in-house, Corona-developed Python scripts to derive relevant metadata for documentation (i.e. top keywords, action items, date, section). Use of the BM25 search algorithm allows in-depth search capabilities of not only the documentation metadata but also of sections of interest within the documents. Additional information for users is provided by advanced NLP capabilities such as abstractive and extractive summarization as a quick read for end users, topic modeling, hypothesis testing between requirements and documents, Named Entity Recognition (NER) and syntax parsing for identifying roles and responsibilities, and textual entailment using AllenNLP for understanding overlap of similar language between different documents. All of peNLP’s information is integrated together through its extensive Document Knowledge Graph, which not only behaves as a visual aid in finding connections between documents but also serves as the backing database for tracing concepts and requirements through corpora.


Premium content

Automated Knowledge Graphs

Next from NAML 2021

Predicting of Health Status from Wearable Physiological Monitoring Data (PreMonData)

Similar lecture

Next Generation NLP and AI Methods in the Navy Contact Center