United Arab Emirates

Most vision-and-language pretraining research focuses on English tasks. However, the creation of multilingual multimodal evaluation datasets (e.g. Multi30K, xGQA, XVNLI, and MaRVL) poses a new challenge in finding high-quality training data that is both multilingual and multimodal. We investigate whether machine translating English multimodal data can be an effective proxy for the lack of readily available multilingual data. We call this framework TD-MML: Translated Data for Multilingual Multimodal Learning, and it can be applied to any multimodal dataset and model. We apply it to both pretraining and fine-tuning data with a state-of-the-art model. In order to prevent models from learning from low-quality translated text, we propose two metrics for automatically removing such translations from the resulting datasets. In experiments on five tasks across 20 languages in the IGLUE benchmark, we show that translated data can provide a useful signal for multilingual multimodal learning, both at pretraining and fine-tuning.

EMNLP 2022

Multilingual Multimodal Learning with Machine Translated Text

vision-and-language pretraining

multilingual multimodal learning

machine translation

## Welcome to EMNLP 2022!
I am delighted to welcome you to EMNLP 2022! I believe this conference has been complicated beyond any precedent. Over the past year, it’s been thrilling to see the organization team approach each new puzzle with creativity and enthusiasm. We hope that those participating in Abu Dhabi as well as those joining remotely will leave the conference feeling newly inspired by the program and newly connected to our ever-growing community. Following EMNLP 2021 and major NLP conferences since, EMNLP 2022 is “hybrid,” serving both virtual and in-person participants.

Our key innovations for EMNLP 2022 include:

* EMNLP 2022 is “hybrid” in a second sense, as well: we allowed both direct and rolling review paper submissions, building on the pilot experiment of EMNLP 2021, which considered a small number of ARR submissions. 
* Familiar from NAACL but new to EMNLP, we’ve added an industry track.
* During the conference, “portals” will link virtual poster sessions to in-person conference participants during poster sessions each day.
* The first ACL-family conference in the United Arab Emirates.

 *Message from Noah A. Smith, University of Washington and Allen Institute for AI, Seattle, Washington, USA* 
***EMNLP 2022 General Chair***
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/9eec7d4a287ee18c278b08229290aa83.png)](https://drive.google.com/file/d/1OlPv6QBeo62VVTughj2jkiLeyHd1WnUt/view)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/a3db7a768409f05192210d98601edb25.png)](https://emnlp2022.rocket.chat/)

To access this site you need to register. Please register [here](https://2022.emnlp.org/registration/).

Register here

Welcome!
EMNLP 2022 will take place in Abu Dhabi from December 7th to December 11th, 2022. And it will be held in hybrid mode, both online and offline.

workshop paper

Social media has increasingly played a key role in emergency response: first responders can use public posts to better react to ongoing crisis events and deploy the necessary resources where they are most needed. Timeline extraction and abstractive summarization are critical technical tasks to leverage large numbers of social media posts about events. Unfortunately, there are few datasets for benchmarking technical approaches for those tasks. This paper presents \datasetname{}, the largest dataset of local crisis event timelines available to date. \datasetname{} contains 1,000 crisis event timelines across four domains: wildfires, local fires, traffic, and storms. We built \datasetname{} using a semi-automated cluster-then-refine approach to collect data from the public Twitter stream. 
Our initial experiments indicate a significant gap between the performance of strong baselines compared to the human performance on both tasks.
Our dataset, code, and models are publicly available (https://github.com/CrisisLTLSum/CrisisTimelines).

CrisisLTLSum: A Benchmark for Local Crisis Event Timeline Extraction and Summarization

A deployed question answering (QA) model can easily fail when the test data has a distribution shift compared to the training data. Robustness tuning (RT) methods have been widely studied to enhance model robustness against distribution shifts before model deployment. However, can we improve a model after deployment? To answer this question, we evaluate test-time adaptation (TTA) to improve a model after deployment. We first introduce ColdQA, a unified evaluation benchmark for robust QA against text corruption and changes in language and domain. We then evaluate previous TTA methods on ColdQA and compare them to RT methods. We also propose a novel TTA method called online imitation learning (OIL). Through extensive experiments, we find that TTA is comparable to RT methods, and applying TTA after RT can significantly boost the performance on ColdQA. Our proposed OIL improves TTA to be more robust to variation in hyper-parameters and test distributions over time.

Robust Question Answering against Distribution Shifts with Test-Time Adaption: An Empirical Study

Timely and effective response to humanitarian crises requires quick and accurate analysis of large amounts of text data - a process that can highly benefit from expert-assisted NLP systems trained on validated and annotated data in the humanitarian response domain. To enable creation of such NLP systems, we introduce and release HumSet, a novel and rich multilingual dataset of humanitarian response documents annotated by experts in the humanitarian response community. The dataset provides documents in three languages (English, French, Spanish) and covers a variety of humanitarian crises from 2018 to 2021 across the globe. For each document, HumSet provides selected snippets (entries) as well as assigned classes to each entry annotated using common humanitarian information analysis frameworks. HumSet also provides novel and challenging entry extraction and multi-label entry classification tasks. In this paper, we take a first step towards approaching these tasks and conduct a set of experiments on Pre-trained Language Models (PLM) to establish strong baselines for future research in this domain. The dataset is available at https://blog.thedeep.io/humset/.

HUMSET: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crisis Response

JamPatoisNLI provides the first dataset for natural language inference in a creole language, Jamaican Patois. Many of the most-spoken low-resource languages are creoles. These languages commonly have a lexicon derived from a major world language and a distinctive grammar reflecting the languages of the original speakers and the process of language birth by creolization. This gives them a distinctive place in exploring the effectiveness of transfer from large monolingual or multilingual pretrained models. While our work, along with previous work, shows that transfer from these models to low-resource languages that are unrelated to languages in their training set is not very effective, we would expect stronger results from transfer to creoles. Indeed, our experiments show considerably better results from few-shot learning of JamPatoisNLI than for such unrelated languages, and help us begin to understand how the unique relationship between creoles and their high-resource base languages affect cross-lingual transfer. JamPatoisNLI, which consists of naturally-occurring premises and expert-written hypotheses, is a step towards steering research into a traditionally underserved language and a useful benchmark for understanding cross-lingual NLP.

JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset

Zero-resource cross-lingual transfer approaches aim to apply supervised models
from a source language to unlabelled target languages. In this paper we perform
an in-depth study of the two main techniques employed so far for cross-lingual
zero-resource sequence labelling, based either on data or model transfer.
Although previous research has proposed translation and annotation projection
(data-based cross-lingual transfer) as an effective technique for cross-lingual
sequence labelling, in this paper we experimentally demonstrate that high
capacity multilingual language models applied in a zero-shot (model-based
cross-lingual transfer) setting consistently outperform data-based
cross-lingual transfer approaches. A detailed analysis of our results suggests
that this might be due to important differences in language use. More
specifically, machine translation often generates a textual signal which is
different to what the models are exposed to when using gold standard data,
which affects both the fine-tuning and evaluation processes. Our results also
indicate that data-based cross-lingual transfer approaches remain a competitive
option when high-capacity multilingual language models are not available.

Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings

This paper deals with the problem of incremental dialect identification. Our goal is to
reliably determine the dialect before the full utterance is given as input. The major part of the previous research on dialect identification has been model-centric, focusing on performance. We address a new question: How much input is needed to identify a dialect? Our approach is a data-centric analysis that results in general criteria for finding the shortest input needed to make a plausible guess. Working
with three sets of language dialects (Swiss German, Indo-Aryan and Arabic languages), we show that it is possible to generalize across dialects and datasets with two input shortening criteria: model confidence and minimal input length (adjusted for the input type).

Early Guessing for Dialect Identification

We propose EntityCS, a method that focuses on Entity-level Code-Switching to capture fine-grained cross-lingual semantics without corrupting syntax. We construct and release an EntityCS corpus with 93 languages based on English Wikipedia and Wikidata, and design novel masking strategies for entity prediction. XLM trained on the constructed EntityCS corpus with the proposed masking strategies shows consistent improvement on entity-centric downstream tasks.

EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switching

Causal inference is becoming an increasingly important topic in deep learning, with the potential to help with critical deep learning problems such as model robustness, interpretability, and fairness. In addition, causality is naturally widely used in various disciplines of science, to discover causal relationships among variables and estimate causal effects of interest. In this tutorial, we introduce the fundamentals of causal discovery and causal effect estimation to the natural language processing (NLP) audience, provide an overview of causal perspectives to NLP problems, and aim to inspire novel approaches to NLP further. This tutorial is inclusive to a variety of audiences and is expected to facilitate the community’s developments in formulating and addressing new, important NLP problems in light of emerging causal principles and methodologies.

Tutorial on Causal Inference for Natural Language Processing

State-of-the-art language models in NLP perform best when fine-tuned even on small datasets, but due to their increasing size, fine-tuning and downstream usage have become extremely compute-intensive. Being able to efficiently and effectively fine-tune the largest pre-trained models is thus key in order to reap the benefits of the latest advances in NLP. In this tutorial, we provide a comprehensive overview of parameter-efficient fine-tuning methods. We highlight their similarities and differences by presenting them in a unified view. We explore the benefits and usage scenarios of a neglected property of such parameter-efficient models—modularity—such as composition of modules to deal with previously unseen data conditions. We finally highlight how both properties—parameter efficiency and modularity—can be useful in the real-world setting of adapting pre-trained models to under-represented languages and domains with scarce annotated data for several downstream applications. 

Modular and Parameter-Efficient Fine-Tuning for NLP Models

The autoregressive mechanism makes it convenient to train sequence generation models, but the generation is slow since autoregressive models can only generate the target sequence word-by-word. Non-autoregressive (NAR) models generate all tokens in parallel by removing the sequential dependencies within the target sequence. While NAR models have received much attention in sequence generation tasks, it is non-trivial for NAR models to obtain satisfactory performance, where the major challenge is to capture the multi-modal data distribution. In this tutorial, we will provide a comprehensive introduction to non-autoregressive sequence generation in four sections. First, we will briefly introduce the background of sequence generation, give the motivation of NAR generation and the multi-modality challenge faced by NAR models. Second, we will introduce efforts to improve NAR generation. We focus on non-autoregressive translation in this paper, which includes many representative methods from different aspects: training objectives, data distillation, latent modeling, upsampling, improved decoding approaches, etc. Third, we will introduce the application of NAR models on a wide range of sequence generation tasks, including language modeling, speech recognition, text-to-speech, etc. Finally, we will conclude this tutorial by summarizing the strengths and challenges of NAR models, and discussing current concerns and future directions of NAR generation. 


Downloads

Next from EMNLP 2022

CrisisLTLSum: A Benchmark for Local Crisis Event Timeline Extraction and Summarization

Similar lecture

Multilingual Multimodal Learning with Machine Translated Text