Ireland

Robots operating in everyday environments need to effectively perceive, model, and infer semantic properties of objects. Existing knowledge reasoning frameworks only model binary relations between an object&#39;s class label and its semantic properties, unable to collectively reason about object properties detected by different perception algorithms and grounded in diverse sensory modalities. We bridge the gap between multimodal perception and knowledge reasoning by introducing an n-ary representation that models complex, inter-related object properties. To tackle the problem of collecting n-ary semantic knowledge at scale, we propose a transformer neural network that directly generalizes knowledge from observations of object instances. The learned model can reason at different levels of abstraction, effectively predicting unknown properties of objects in different environmental contexts given different amounts of observed information. We quantitatively validate our approach against five prior methods on LINK, a unique dataset we contribute that contains 1457 situated object instances with 15 multimodal properties types and 200 total properties. Additionally, we apply our work to a mobile manipulation robot, demonstrating its ability to leverage n-ary reasoning to retrieve objects and actively detect object properties.

ACL 2022

Learning Instance-Level N-Ary Semantic Knowledge At Scale For Robots Operating in Everyday Environments

robotics

commonsense reasoning

knowledge representation

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

The AAMAS conference series was initiated in 2002 in Bologna, Italy as a joint event comprising the 6th International Conference on Autonomous Agents (AA), the 5th International Conference on Multiagent Systems (ICMAS), and the 9th International Workshop on Agent Theories, Architectures, and Languages (ATAL).

Subsequent AAMAS conferences have been held in Melbourne, Australia (July 2003), New York City, NY, USA (July 2004), Utrecht, The Netherlands (July 2005), Hakodate, Japan (May 2006), Honolulu, Hawaii, USA (May 2007), Estoril, Portugal (May 2008), Budapest, Hungary (May 2009), Toronto, Canada (May 2010), Taipei, Taiwan (May 2011), Valencia, Spain (June 2012), Minnesota, USA (May 2013), Paris, France (May 2014), Istanbul, Turkey (May 2015), Singapore (May 2016), São Paulo (2017) and Stockholm, Sweden (2018), Montreal (May 2019), Auckland (May 2020, Virtual), London (May 2021, Virtual).

A ticket is required to attend this event, please register using the link below:

https://aamas2022-conference.auckland.ac.nz/attending/registration/

Registration Is Required

AAMAS 2022

AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the largest and most influential conference in the area of agents and multiagent systems. The aim of the conference is to bring together researchers and practitioners in all areas of agent technology and to provide a single, high-profile, internationally renowned forum for research in the theory and practice of autonomous agents and multiagent systems. AAMAS is the flagship conference of the non-profit International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

Robots operating in everyday environments need to effectively perceive, model, and infer semantic properties of objects. Existing knowledge reasoning frameworks only model binary relations between an object's class label and its semantic properties, unable to collectively reason about object properties detected by different perception algorithms and grounded in diverse sensory modalities. We bridge the gap between multimodal perception and knowledge reasoning by introducing an n-ary representation that models complex, inter-related object properties. To tackle the problem of collecting n-ary semantic knowledge at scale, we propose a transformer neural network that directly generalizes knowledge from observations of object instances. The learned model can reason at different levels of abstraction, effectively predicting unknown properties of objects in different environmental contexts given different amounts of observed information. We quantitatively validate our approach against five prior methods on LINK, a unique dataset we contribute that contains 1457 situated object instances with 15 multimodal properties types and 200 total properties. Additionally, we apply our work to a mobile manipulation robot, demonstrating its ability to leverage n-ary reasoning to retrieve objects and actively detect object properties.

workshop paper

# Welcome everyone to ACL 2022!

The 60th Annual Meeting of the Association for Computational Linguistics is taking place May 22-27, 2022 as a hybrid event, in Dublin and online. We are happy to welcome all of you to this anniversary edition with an almost 50-50 in-person and virtual participation. 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, three amazing keynote events and two new initiatives of invited talks: Spotlight Talks for Young Rising Stars (STIRS) and The Next Big Idea Talks. Posters (including Findings of ACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop will have an oral session and a poster session as part of Poster Session 1. The program also features eight Tutorials and 28 Workshops. 

 
We wish you a wonderful conference! 
[**The ACL 2022 Organizing Committee**](https://www.2022.aclweb.org/organisers)
 
[**Conference Handbook**](https://drive.google.com/file/d/1_BUCMfhMVrjG9E2e71aHdHeE28KSje0l/view?usp=sharing) 
[**Mini Handbook**](https://drive.google.com/file/d/1qlBKl0wzmlVF1oCeMQl3BahLd9nLP5Ce/view?usp=sharing) 
[**Posters and Demo guides**](https://drive.google.com/file/d/1UucMAoCNncIOaH1rMMDa0owuG9qgvJTG/view?usp=sharing)

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing (NLP). 

The ability to recognize analogies is fundamental to human cognition. Existing benchmarks to test word analogy do not reveal the underneath process of analogical reasoning of neural models. Holding the belief that models capable of reasoning should be right for the right reasons, we propose a first-of-its-kind Explainable Knowledge-intensive Analogical Reasoning benchmark (E-KAR). Our benchmark consists of 1,655 (in Chinese) and 1,251 (in English) problems sourced from the Civil Service Exams, which require intensive background knowledge to solve. More importantly, we design a free-text explanation scheme to explain whether an analogy should be drawn, and manually annotate them for each and every question and candidate answer. Empirical results suggest that this benchmark is very challenging for some state-of-the-art models for both explanation generation and analogical question answering tasks, which invites further research in this area.

E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning

Answering questions about why events happen in narratives requires commonsense knowledge that is external to the narrative. What aspects of this knowledge is accessible to large models? What aspects can be made accessible via exter- nal commonsense resources? We study these in the context of answering Why questions in the TellMeWhy dataset using COMET as a source of relevant commonsense relations. We analyze the relative improvements over a base T5 model when (a) increasing the model size, (b) injecting knowledge from COMET as part of the task input, and (c) asking the model to generate COMET relation type as an explanation in addition to its answer. Results show that the larger model, as expected, yields substantial improvements over the base. Interestingly, we find that the question specific COMET relations can provide substantial improvements for both base and large models, with additional possible gains when asking the model to also generate COMET relation type. So, we augment a large model with noisy hints from COMET and find that this improves performance on the TellMeWhy task. We also develop a simple ontology of knowledge types and analyze the relative cover- age of the different models on these categories. Together, these findings suggest potential for methods that can automatically select and inject commonsense from relevant sources.

Analyzing the Contribution of Commonsense Knowledge for Why-Question Answering

Large LMs such as GPT-3 are powerful, but can commit mistakes that are obvious to hu- mans. For example, GPT-3 would mistakenly interpret "What word is similar to good?" to mean a homonym, while the user intended a synonym. Our goal is to effectively correct such errors via user interactions with the system but without retraining, which will be prohibitively costly. We pair GPT-3 with a growing memory of recorded cases where the model misunderstood the user’s intents, along with user feedback for clarification. Such a memory allows our system to produce enhanced prompts for any new query based on the user feedback for error correction on similar cases in the past. On four tasks (two lexical tasks, two advanced ethical reasoning tasks), we show how a (simulated) user can interactively teach a deployed GPT-3, substantially in- creasing its accuracy over the queries with different kinds of misunderstandings by the GPT- 3. Our approach is a step towards the low-cost utility enhancement for very large pre-trained LMs.

Memory-assisted prompt editing to improve GPT-3 after deployment

How grounded is document-grounded conversational AI?

Artificial Intelligence services are increasingly being used to provide accessibility for
consumers with sensory disabilities in information and communication technologies.
However, these AI-powered are only as good as the data they’ve been fed, which
often reflects historical or current biases either in the collection of data or in the data
itself. In general, people with diverse visual and auditory disabilities are not wellrepresented in these datasets because although many people have some disability,
the number with any one disability is usually a small percentage of the population.
These disparities in representation can lead to disproportionate impacts of AIpowered services on people with disabilities, and though laws to protect people with
disabilities exist, the evaluation of AI-based systems and the models they rely on, to
comply with those laws in both the letter and spirit of the law are important, unsolved
problems. We advocate for guidelines for an evaluation of the models used by AIpowered services that considers their fairness, accountability, transparency, in terms
of use-cases, inclusiveness, and legal requirements for accessibility. These
accessibility guidelines can be used by architects or developers of AI-powered
services to ensure the services and underlying models are fair by being available to
consumers with diverse sensory abilities, accountable by complying with legal
requirements, and transparent by complying with both social and user expectations.

Accessibility Guidelines for AI-powered Services and Models

Humans quite frequently interact with conversational agents. The rapid advancement in generative language modeling through neural networks has helped advance the creation of intelligent conversational agents. Researchers typically evaluate the output of their models through crowdsourced judgments, but there are no established best practices for conducting such studies. We look closely at the practices of evaluation of NLG output, and discuss implications of human cognitive biases on experiment design and the resulting data.

Human Centered Approach to Conversational AI - On the Evaluation of NLG systems

The Role of Text Generation in Fighting Hostile Posts

Social, Secure, Scalable, and Efficient Federated Learning

The success of artificial neural networks in language processing tasks has underscored the need to understand how they accomplish their behavior, and, in particular, how that behavior is supported by their internal vector representations. Probing classifiers, which are often used to address this question, rely on a strong, and arguably questionable, assumption: if a classifier can decode a particular piece of information from a vector, we can conclude that that piece of information affects the model's behavior. My talk will present two methods that go beyond this paradigm. The first method, AlterRep, modifies a contextualized representation such that it encodes a particular value of a linguistic feature (e.g., whether it’s in an embedded clause), and examines how that perturbation affects the model’s behavior. Using this method, I will show that syntactic information can be decoded from all layers of BERT, but only plays a functional role in a subset of those (the middle layers). The second, DISCOVER, aims to identify linear compositional structure in the vector: it tests the hypothesis that the vector can be fully decomposed into a sum of symbol-representing vectors (filler-role bindings). This hypothesis is borne out to a remarkable extent for networks trained on highly compositional tasks, and to a lesser extent for BERT’s embeddings.

Beyond Probing Classifiers: Deconstructing the Function and Structure of Vector Representations

Artificial intelligence in the form of natural language processing has a pivotal role in increasing the communication rate for users of Augmentative and Alternative Communication (AAC) technologies. However, communication rates remain prohibitively slow. Research at Dundee University has highlighted the importance of participatory design, focusing on language development and the role that NLP can play to support different aspects of communication. With reference to how different AI technologies can impact communication, the need for a paradigm shift in the design of interfaces will be demonstrated.

Learning Instance-Level N-Ary Semantic Knowledge At Scale For Robots Operating in Everyday Environments

Downloads

Next from ACL 2022

E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning

Similar lecture

Multi-Agent Heterogeneous Digital Twin Framework with Dynamic Responsibility Allocation for Complex Task Simulation