Ireland

Artificial Intelligence services are increasingly being used to provide accessibility for
consumers with sensory disabilities in information and communication technologies.
However, these AI-powered are only as good as the data they’ve been fed, which
often reflects historical or current biases either in the collection of data or in the data
itself. In general, people with diverse visual and auditory disabilities are not wellrepresented in these datasets because although many people have some disability,
the number with any one disability is usually a small percentage of the population.
These disparities in representation can lead to disproportionate impacts of AIpowered services on people with disabilities, and though laws to protect people with
disabilities exist, the evaluation of AI-based systems and the models they rely on, to
comply with those laws in both the letter and spirit of the law are important, unsolved
problems. We advocate for guidelines for an evaluation of the models used by AIpowered services that considers their fairness, accountability, transparency, in terms
of use-cases, inclusiveness, and legal requirements for accessibility. These
accessibility guidelines can be used by architects or developers of AI-powered
services to ensure the services and underlying models are fair by being available to
consumers with diverse sensory abilities, accountable by complying with legal
requirements, and transparent by complying with both social and user expectations.

ACL 2022

Accessibility Guidelines for AI-powered Services and Models

# Welcome everyone to ACL 2022!

The 60th Annual Meeting of the Association for Computational Linguistics is taking place May 22-27, 2022 as a hybrid event, in Dublin and online. We are happy to welcome all of you to this anniversary edition with an almost 50-50 in-person and virtual participation. 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, three amazing keynote events and two new initiatives of invited talks: Spotlight Talks for Young Rising Stars (STIRS) and The Next Big Idea Talks. Posters (including Findings of ACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop will have an oral session and a poster session as part of Poster Session 1. The program also features eight Tutorials and 28 Workshops. 

 
We wish you a wonderful conference! 
[**The ACL 2022 Organizing Committee**](https://www.2022.aclweb.org/organisers)
 
[**Conference Handbook**](https://drive.google.com/file/d/1_BUCMfhMVrjG9E2e71aHdHeE28KSje0l/view?usp=sharing) 
[**Mini Handbook**](https://drive.google.com/file/d/1qlBKl0wzmlVF1oCeMQl3BahLd9nLP5Ce/view?usp=sharing) 
[**Posters and Demo guides**](https://drive.google.com/file/d/1UucMAoCNncIOaH1rMMDa0owuG9qgvJTG/view?usp=sharing)

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing (NLP). 

keynote

Humans quite frequently interact with conversational agents. The rapid advancement in generative language modeling through neural networks has helped advance the creation of intelligent conversational agents. Researchers typically evaluate the output of their models through crowdsourced judgments, but there are no established best practices for conducting such studies. We look closely at the practices of evaluation of NLG output, and discuss implications of human cognitive biases on experiment design and the resulting data.

Human Centered Approach to Conversational AI - On the Evaluation of NLG systems

The Role of Text Generation in Fighting Hostile Posts

Social, Secure, Scalable, and Efficient Federated Learning

The success of artificial neural networks in language processing tasks has underscored the need to understand how they accomplish their behavior, and, in particular, how that behavior is supported by their internal vector representations. Probing classifiers, which are often used to address this question, rely on a strong, and arguably questionable, assumption: if a classifier can decode a particular piece of information from a vector, we can conclude that that piece of information affects the model's behavior. My talk will present two methods that go beyond this paradigm. The first method, AlterRep, modifies a contextualized representation such that it encodes a particular value of a linguistic feature (e.g., whether it’s in an embedded clause), and examines how that perturbation affects the model’s behavior. Using this method, I will show that syntactic information can be decoded from all layers of BERT, but only plays a functional role in a subset of those (the middle layers). The second, DISCOVER, aims to identify linear compositional structure in the vector: it tests the hypothesis that the vector can be fully decomposed into a sum of symbol-representing vectors (filler-role bindings). This hypothesis is borne out to a remarkable extent for networks trained on highly compositional tasks, and to a lesser extent for BERT’s embeddings.

Beyond Probing Classifiers: Deconstructing the Function and Structure of Vector Representations

Artificial intelligence in the form of natural language processing has a pivotal role in increasing the communication rate for users of Augmentative and Alternative Communication (AAC) technologies. However, communication rates remain prohibitively slow. Research at Dundee University has highlighted the importance of participatory design, focusing on language development and the role that NLP can play to support different aspects of communication. With reference to how different AI technologies can impact communication, the need for a paradigm shift in the design of interfaces will be demonstrated.

Participatory Interface Design for Intelligent Augmentative and Alternative Communication Technologies

Human evaluation of modern high-quality machine translation systems is a difficult problem, and there is increasing evidence that inadequate evaluation procedures can lead to erroneous conclusions. While there has been considerable research on human evaluation, the field still lacks a commonly accepted standard procedure. As a step toward this goal, we propose an evaluation methodology grounded in explicit error analysis, based on the Multidimensional Quality Metrics (MQM) framework. We carry out the largest MQM research study to date, scoring the outputs of top systems from the WMT 2020 shared task in two language pairs using annotations provided by professional translators with access to full document context. We analyze the resulting data extensively, finding among other results a substantially different ranking of evaluated systems from the one established by the WMT crowd workers, exhibiting a clear preference for human over machine output. Surprisingly, we also find that automatic metrics based on pre-trained embeddings can outperform human crowd workers. We further discuss the impact of this study on both the WMT metric task, and the general MT task. We will close the talk by showcasing research that benefits from the new evaluation methodology: Minimum Bayes Risk Decoding with neural metrics significantly outperforms beam search decoding in expert-based human evaluations while the previous human evaluation standards using crowd-workers set both decoding strategies on par with each other.

A journey of MT research: Why it is crucial to work on human evolution

Most work on scholarly document processing assumes that the information processed is trustworthy and factually correct. However, this is not always the case. There are two core challenges, which should be addressed: 1) ensuring that scientific publications are credible -- e.g. that claims are not made without supporting evidence, and that all relevant supporting evidence is provided; and 2) that scientific findings are not misrepresented, distorted or outright misreported when communicated by journalists or the general public. In this talk, I will present some first steps towards addressing these problems, discussing our research on exaggeration detection of scientific claims and on scientific fact checking.

Automatically Detecting Scientific Misinformation

In this talk I will give an overview of our work on fact extraction and verification. I will first introduce the first fact extraction and verification shared task on unstructured information (text) from Wikipedia and how we constructed the first large-scale dataset for it. I will then discuss how we incorporated structured inference to achieve improved robustness while maintaining state-of-the-art accuracy in fact verification. Following this, I will describe our work on correcting factual errors in claims. I will conclude this talk by presenting our recent dataset and shared task on fact extraction and verification on both structured (tables) and unstructured (tables) information.

Downloads

Next from ACL 2022

Human Centered Approach to Conversational AI - On the Evaluation of NLG systems

Similar lecture

Will Big Models Solve Dialog Systems?

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2022

Human Centered Approach to Conversational AI - On the Evaluation of NLG systems

Similar lecture

Will Big Models Solve Dialog Systems?

Downloads