Dominican Republic

Despite excellent performance on tasks such as question answering, Transformer-based architectures remain sensitive to syntactic and contextual ambiguities. Question Paraphrasing (QP) offers a promising solution as a means to augment existing datasets. The main challenges of current QP models include lack of training data and difficulty in generating diverse and natural questions. In this paper, we present ConQuest, a framework for generating synthetic datasets for contextual question paraphrasing. To this end, ConQuest first employs an answer-aware question generation(QG) model to create a question-pair dataset and then uses this data to train a contextualized question paraphrasing model. We extensively evaluate ConQuestand to show its ability to produce more diverse and fluent question pairs than existing approaches. Our contextual paraphrase model also establishes a strong baseline for end-to-end contextual paraphrasing. Further, We find that context can improve BLEU-1 score on contextual compression and expansion by 4.3 and 11.2 respectively, compared to a non-contextual model.

EMNLP 2021

ConQuest: Contextual Question Paraphrasing through Answer-Aware Synthetic Question Generation

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible. The experience for on-site participants would closely approximate a normal pre-COVID *ACL conference, with 5-6 thematically organized parallel sessions and live Q/A and interactive discussion immediately after the talks. Presentations by virtual participants will be equitably interleaved with those of on-site participants, projected on the auditorium screens as if on-site, and also followed immediately by live Q/A and interactive discussion at a time during reasonable waking hours for the virtual presenter. For all participants, on-site and virtual, who are unable to attend a session due to either time-zone issues or because they are participating in another session live, talk recordings and slides will be available online at a minimum after the live presentation (and in many cases before as well), and questions may be submitted in advance on session-specific discussion boards and answered live in session with the usual visual aids if desired.

<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/ceby/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

Please Note: The EMNLP registration system is not currently connected to the underline site as we are still in the process of building out EMNLP 2021. You will receive access instructions from underline the week of November 1st. 

Access is given only to EMNLP upon registration, if you have not registered please do so [here](https://2021.emnlp.org/registration).

Registered attendees will receive access the week of November 1st.

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible.

workshop paper

We studied structure augmentation on documents for keyphrase generation.

Structure-Augmented Keyphrase Generation

Auxiliary information from multiple sources has been demonstrated to be effective in zero-shot fine-grained entity typing (ZFET). However, there lacks a comprehensive understanding about how to make better use of the existing information sources and how they affect the performance of ZFET. In this paper, we empirically study three kinds of auxiliary information: {context consistency}, {type hierarchy} and {background knowledge} (e.g., prototypes and descriptions) of types, and propose a multi-source fusion model (MSF) targeting these sources. The performance obtains up to 11.42% and 22.84% absolute gains over state-of-the-art baselines on BBN and Wiki respectively with regard to macro F1 scores. More importantly, we further discuss the characteristics, merits and demerits of each information source and provide an intuitive understanding of the complementarity among them.

An Empirical Study on Multiple Information Sources for Zero-Shot Fine-Grained Entity Typing

Due to unstructured sentences and some misspellings and errors, finding the named entities in a noisy environment such as social media takes much more effort than formal languages. The ParsTwiNER corpus, gathered from Persian Twitter, contains about 250k tokens, based on standard instructions such as MUC-6 or CoNLL 2003. Using Cohen's Kappa coefficient, the consistency of annotators is 0.95, a high score. We demonstrate that some state-of-the-art models were degraded on this corpus and trained a model using parallel transfer learning based on BERT architecture. The proposed model works well in informal Persian, as well as in formal Persian. The corpus and trained models are publicly available at https://github.com/overfit-ir/parstwiner and https://huggingface.co/overfit/twiner-bert-base-mtl, respectively.

ParsTwiNER: A Corpus for Named Entity Recognition at Informal Persian

We present DreamDrug, a crowdsourced dataset for detecting mentions of drugs in
noisy user-generated item listings from darknet markets. Our dataset contains nearly 15,000 manually annotated drug entities in over 3,500 item listings scraped from the darknet market platform "DreamMarket" in 2017. We also train and evaluate baseline models for detecting these entities, using contextual language
models fine-tuned in a few-shot setting and on the full dataset, and examine the effect of pretraining on in-domain unannotated corpora.

DreamDrug - A crowdsourced NER dataset for detecting drugs in darknet markets

Code-mixed text generation systems have found applications in many downstream tasks, including speech recognition, translation and dialogue. A paradigm of these generation systems relies on well-defined grammatical theories of code-mixing, and there is a lack of comparison of these theories. We present a large-scale human evaluation of two popular grammatical theories, Matrix-Embedded Language (ML) and Equivalence Constraint (EC). We compare them against three heuristic-based models and quantitatively demonstrate the effectiveness of the two grammatical theories.

Comparing Grammatical Theories of Code-Mixing

The benchmark performance of cross-database semantic parsing has climbed steadily in recent years, catalyzed by the wide adoption of pre-trained language models. Yet existing work have shown that state-of-the-art cross-database semantic parsers struggle to generalize to novel user utterances, databases and query structures. To obtain transparent details on the strengths and limitation of these models, we propose a diagnostic testing approach based on controlled synthesis of canonical natural language and SQL pairs. Inspired by the "Checklist" paper (Ribeiro et al. 2020), we characterize a set of essential capabilities for cross-database semantic parsing models, and detailed the method for synthesizing the corresponding test data. We evaluated a variety of high performing models using the proposed approach, and identified several non-obvious weaknesses across models (e.g. unable to correctly select many columns). Our dataset and code are released as a test suite at: http://github.com/hclent/BehaviorCheckingSemPar.

Improving Punctuation Restoration for Speech Transcripts via External Data

Fake news causes significant damage to society. To deal with these fake news, several studies on building detection models and arranging datasets have been conducted.
Most of the fake news datasets depend on a specific time period. Consequently, the detection models trained on such a dataset have difficulty detecting novel fake news generated by political changes and social changes; they may possibly result in biased output from the input, including specific person names and organizational names. We refer to this problem as Diachronic Bias because it is caused by the creation date of news in each dataset. In this study, we confirm the bias, especially proper nouns including person names, from the deviation of phrase appearances in each dataset. Based on these findings, we propose masking methods using Wikidata to mitigate the influence of person names and validate whether they make fake news detection models robust through experiments with in-domain and out-of-domain data.

Mitigation of Diachronic Bias in Fake News Detection Dataset

This work takes a critical look at the evaluation of user-generated content (UGC) automatic translation and the well-known specificities of which raise many challenges for MT. We introduce the PMUMT, a new data set for UGC translation evaluation. Using this data set, we conduct several experiments to measure the impact of different kinds of UGC specificities on translation quality, more precisely than previously possible.

Understanding the Impact of UGC Specificities on Translation Quality

This work explores the capacities of character-based Neural Machine Translation to translate noisy User-Generated Content (UGC). Within a strict zero-shot scenario, we first study the detrimental impact on translation performance of various UGC phenomena on an annotated dataset we developed, and then show that such models are indeed incapable of handling unknown characters, for which we propose a simple but efficient solution.

Noisy User-Generated Content Translation at the Character Level: Revisiting Open-Vocabulary Capabilities and Robustness of Char-Based Models

Twitter data has become established as a valuable source of data for various application scenarios in the past years. For many such applications, it is necessary to know where Twitter posts (tweets) were sent from or what location they refer to. Researchers have frequently used exact coordinates provided in a small percentage of tweets, but Twitter removed the option to share these coordinates in mid-2019. Moreover, there is reason to suspect that a large share of the provided coordinates did not correspond to GPS coordinates of the user even before that.

In this paper, we explain the situation and the 2019 policy change and shed light on the various options of still obtaining location information from tweets. We provide usage statistics including changes over time, and analyze what the removal of exact coordinates means for various common research tasks performed with Twitter data. Finally, we make suggestions for future research requiring geolocated tweets.

Content not yet available

ConQuest: Contextual Question Paraphrasing through Answer-Aware Synthetic Question Generation

Next from EMNLP 2021

Structure-Augmented Keyphrase Generation

Similar lecture

SituatedQA: Incorporating Extra-Linguistic Contexts into QA