Spain

We consider the issue of AI (Artificial Intelligence) evaluation in the light of risk management for the proper marketing of machinery. When a physical device is driven by AI, the question arises as to how to manage the risks associated with the autonomous functions. Indeed, the risk of physical damage to persons or property in the event of a ``wrong&#39;&#39; decision must be properly assessed. The study presents the advances needed to carry out relevant risk analyses for the deployment of safe intelligent machines, and offers good practice that leverages the knowledge in AI evaluation to optimize the safety of these machines.

ECAI 2020

Risk assessment of artificial intelligence in autonomous machines

### Welcome to the 10th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.
#### On-Demand Access

* This is an on-demand site and all the recorded content from the workshop is available here. 
* Access to the recordings is exclusively available to registered participants.
* Each registrant has received an email notification, with instructions on how to access the content.

[![](https://assets.underline.io/markdown_image/1/image/60cd12aa376c191d3467ea2ee60944ea.png)](https://naml2026.org/)

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations
<br><br>

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Please log in using the email address you registered with.** If you have any issues logging in, please contact us at <naml2026@underline.io>.

Let’s celebrate the past decade by investing in the next. 🚀


Please log in to explore this event.

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Explore the latest advancements in machine learning tailored to Naval applications 🔎**

With on-demand access to NAML 2026, you can watch all lectures & browse poster presentations at your convenience, while also engaging directly with presenters via the Underline platform - ask questions, exchange ideas, and stay connected.

**If you still haven’t registered, you can do so now.**
👇
**Take advantage of a special one-time offer: full access for just $99, with complimentary registration for students, government, and military employees.**

**Get access here: https://naml2026-ondemand.paperform.co/**

Please check your spam folder, or contact our support team at **<naml2026@underline.io>** if you registered, but still haven't received an access email.

Let’s celebrate the past decade by investing in the next. 🚀

Please register!

NAML 2026

The 10th Annual Workshop on Naval Applications of Machine Learning (NAML 2026) was held in San Diego, California, March 2-5, 2026. NAML is a forum for interdisciplinary experts from government, industry, and academia to discuss results of ongoing research on theory and applications of machine learning to address some of the most challenging problems faced by our naval forces. 



We consider the issue of AI (Artificial Intelligence) evaluation in the light of risk management for the proper marketing of machinery. When a physical device is driven by AI, the question arises as to how to manage the risks associated with the autonomous functions. Indeed, the risk of physical damage to persons or property in the event of a ``wrong'' decision must be properly assessed. The study presents the advances needed to carry out relevant risk analyses for the deployment of safe intelligent machines, and offers good practice that leverages the knowledge in AI evaluation to optimize the safety of these machines.

technical paper

ECAI2020 featured a full programme of technical papers, as well as many AI-related events of interest for researchers, students and all attendants who are interested on contemporary AI, including:




**An extensive programme of workshops, tutorials, and invited speakers**

**The Prestigious Applications of Intelligent Systems conference (PAIS)**

**The Frontiers in AI track sessions**

**The Lab To Market event**

**The Women in AI time**

**The EU Challenges forum**

… and many more!

ECAI 2020 featured a full programme of technical papers, as well as many AI-related events of interest for researchers, students and all attendants who are interested on contemporary AI.

This paper argues that at a certain point in research toward AGI, the problem may become well-enough theorized that a clear roadmap exists for achieving it, such that a Manhattan Project- like effort could greatly shorten the time to completion. If state actors perceive that this threshold has been crossed, their incentives around openness and international cooperation may shift rather suddenly, with serious implications for AI risks and the stability of international AI governance regimes. The paper characterizes how such a ‘runway’ period would be qualitatively different from preceding stages of AI research, and accordingly proposes a research program aimed at assessing how close the field of AI is to such a threshold—that is, it calls for the formulation of a ‘roadmap to the roadmap.’

Roadmap to a Roadmap: How Could We Tell When AGI is a ‘Manhattan Project’ Away?

The widespread use of experimental benchmarks in AIresearch  has  created  new  competition  and  collaboration  dynamics that are still poorly understood. In this paper we provide an innovative methodology to explore this dynamics and analyse the way different entrants in these competitions, from academia to tech giants,behave and react depending on their own or others’ achievements.We perform an analysis of over twenty popular benchmarks in AI,linking their underlying research papers. We identify links between researchers and institutions (i.e., communities) beyond the standard co-authorship relations, and we explore a series of hypotheses about their behaviour as well as some aggregated results in terms of activity, breakthroughs and efficiency. As a result, we detect and characterise the dynamics of research communities at different levels of abstraction, including organisation, affiliation, trajectories, results and activity.

The Scientometrics of AI Benchmarks: Unveiling the Underlying Mechanics of AI Research

The performance of the machine learning algorithms can be empirically evaluated via test campaigns: conventional testing approaches allow to obtain, from annotated databases, a first estimate of the operating environment of artificial intelligence systems. However, as those data are hardly I .i.d, additional scores could be useful to offer complementary performance assessment typically for intelligent systems performing critical functions. This paper proposes an approach to generate, without annotation phase, realistic corner cases to complete the exploration of the operating environment. The results of the proof of concept provided for optical character recognition highlight the interest of test data generation and automatic scenario selection methods, including GAN and adversarial approaches, to address this evaluation need.

Generating corner cases for crash-testing deep networks

"The field of Artificial Intelligence (AI) and, in particular, the Machine Learning area, counts on a wide range of performance metrics and benchmark data sets to assess the problem-solving effec- tiveness of its solutions. However, the appearance of research centres, projects or institutions addressing AI solutions from a multidisci- plinary and multi-stakeholder perspective suggests a new approach to assessment comprising ethical guidelines, reports or tools and frame- works to help both academia and business to move towards a re- sponsible conceptualisation of AI. They all highlight the relevance of three key aspects: (i) enhancing cooperation among the different stakeholders involved in the design, deployment and use of AI; (ii) promoting multidisciplinary dialogue, including different domains of expertise in this process; and (iii) fostering public engagement to maximise a trusted relation with new technologies and practitioners. In this paper, we introduce the Observatory on Society and Artifi- cial Intelligence (OSAI), an initiative grew out of the project AI4EU aimed at stimulating reflection on a broad spectrum of issues of AI (ethical, legal, social, economic and cultural). In particular, we de- scribe our work in progress around OSAI and suggest how this and similar initiatives can promote a wider appraisal of progress in AI. This will give us the opportunity to present our vision and our modus operandi to enhance the implementation of these three fundamental dimensions.
"

Progressing Towards Responsible AI

This work presents an operational definition that facilitates a common understanding of AI and its measurability, paving the way for a transparent and comparable monitoring activity in the context of AI Watch, the European Commission’s knowledge service to monitor the development, uptake and impact of artificial intelligence (AI) for Europe. The operational definition is constituted by a taxonomy and a list of related keywords. The method that we propose is iterative, considering that AI is a dynamic field, so its definition should be updated over time to capture the rapid AI evolution. The method consists of the following steps: (i) qualitative analysis of AI definitions and subdomains emanating from reports with academic, industrial and policy perspectives, (ii) selection of definition as starting point, taxonomy formation, and identification of representative keywords in AI with a natural language processing method, and (iii) taxonomy and keywords validation. This results in a unique taxonomy that represents and interconnects all the AI subdomains from political, research and industrial perspective and enables the efficient mapping of the AI landscape of economic agents across different technological areas.

Setting the Boundaries of the AI Landscape: An Operational Definition for the European Commission’s AI Watch

This short paper aims at introducing some labelling concepts developed and used by the authors. Indeed, developing a conceptual framework for interdisciplinary research in history is a much-needed strategy in order to ensure that historians use all vestiges from the past regardless their origin or support for the construction of historical discourse. Fixing the semantics of historical knowledge is the first unavoidable step to build a new scenario in which NLP tasks enable more efficient data gathering and exploitation.

The Semantics of Historical Knowledge. Labelling Strategies for Interdisciplinary and Digital Research in History

Presentation is about our chatbot creation platform and its dialog management techniques. The framework enables creating chatbots without coding using intent-slot model. It contains intent detection and dialog act classifier modules to understand user utterances more accurately. Also the framework includes and analytics tool to monitor and cluster past conversation data to discover topics and training sentences.

Domain-Independent, Task-Oriented Chatbot Creation and Conversation Policy Management Framework

"Static word embeddings, like word2vec or GloVe, are often assessed when solving syntactic and semantic analogies. Among the latter, we are interested in relations that one would find in lexical-semantic knowledge bases like WordNet, also covered in analogy test sets for English. This paper describes the creation of a new test for assessing Portuguese word embeddings, dubbed TALES, with an exclusive focus on lexical-semantic relations, acquired from lexical resources in Portuguese.
It further reports on the performance of methods previously used for solving analogies, with pre-trained Portuguese word embeddings, when applied to the created dataset, an experiment that revealed that TALES is challenging to solve.
Results achieved are briefly discussed, with conclusions that may be useful for developing new approaches for this problem, possibly new embeddings, as well as future versions of TALES."

 TALES: Test Set of Portuguese Lexical-Semantic Relations for Assessing Word Embeddings

"In this presentation we will show the application of the Thematic Theory and the Thematic Progression Theory to automatically annotate discourse features, which enable the inference of the implicit conceptual schemas of texts.
"

A new approach for extracting the conceptual schema of texts based on the linguistic Thematic Progression theory

Open information extraction (OpenIE) is a task of extracting structured information from unstructured texts independently of the domain. Recent advances have applied Deep Learning for Natural Language tasks improving the state-of-the-art, even though those methods usually require a large and high-quality corpus. The construction of an OpenIE dataset is a tedious and error-prone task, and one technique employed concerns the extractions from rule-based techniques and manual validation of those extraction triples. As low-resource languages usually lack available datasets for the application of high-performance Deep Learning techniques, our intuition is that a low-resource model based-on multilingual information can learn generalizations across languages and benefits from cross-lingual data. Moreover, we would like to interpret the set of generalized information gathered from multilingual learning to increase the Open IE classification task. In this paper, we introduce TabOIEC, a multilingual classifier based on generic morpho-syntactic features. Our classifier carries a glass-box method which can provide interpretation about some of the classifier decisions. We evaluate our approach through a small corpus of Open IE extractions for the English, Spanish, and Portuguese languages. Our results consider that for all languages our approach improves F1 measures, particularly for monolinguality. Experiments on Zero-shot learning provide evidence that our TabOIEC generalizes the classifier on other languages than that trained, although there is a shy transfer learning among them. Experiments on multilinguality do reduce the cost of training, however, in our experiments were difficult to provide appropriate generalizations. 

Downloads

Next from ECAI 2020

Roadmap to a Roadmap: How Could We Tell When AGI is a ‘Manhattan Project’ Away?

Similar lecture

Operationalizing AI Risk Tolerance: Framework-Agnostic Taxonomy, Calibration, and Composite Risk Scoring

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ECAI 2020

Roadmap to a Roadmap: How Could We Tell When AGI is a ‘Manhattan Project’ Away?

Similar lecture

Operationalizing AI Risk Tolerance: Framework-Agnostic Taxonomy, Calibration, and Composite Risk Scoring

Downloads