Dominican Republic

Online conversations include more than just text. Increasingly, image-based responses such as memes and animated gifs serve as culturally recognized and often humorous responses in conversation. However, while NLP has broadened to multimodal models, conversational dialog systems have largely focused only on generating text replies. Here, we introduce a new dataset of 1.56M text-gif conversation turns and introduce a new multimodal conversational model Pepe the King Prawn for selecting gif-based replies. We demonstrate that our model produces relevant and high-quality gif responses and, in a large randomized control trial of multiple models replying to real users, we show that our model replies with gifs that are significantly better received by the community.

EMNLP 2021

An animated picture says at least a thousand words: Selecting Gif-based Replies in Multimodal Dialog

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible. The experience for on-site participants would closely approximate a normal pre-COVID *ACL conference, with 5-6 thematically organized parallel sessions and live Q/A and interactive discussion immediately after the talks. Presentations by virtual participants will be equitably interleaved with those of on-site participants, projected on the auditorium screens as if on-site, and also followed immediately by live Q/A and interactive discussion at a time during reasonable waking hours for the virtual presenter. For all participants, on-site and virtual, who are unable to attend a session due to either time-zone issues or because they are participating in another session live, talk recordings and slides will be available online at a minimum after the live presentation (and in many cases before as well), and questions may be submitted in advance on session-specific discussion boards and answered live in session with the usual visual aids if desired.

<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/ceby/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

Please Note: The EMNLP registration system is not currently connected to the underline site as we are still in the process of building out EMNLP 2021. You will receive access instructions from underline the week of November 1st. 

Access is given only to EMNLP upon registration, if you have not registered please do so [here](https://2021.emnlp.org/registration).

Registered attendees will receive access the week of November 1st.

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible.

workshop paper

Conventional entity typing approaches are based on independent classification paradigms, which make them difficult to recognize inter-dependent, long-tailed and fine-grained entity types. In this paper, we argue that the implicitly entailed extrinsic and intrinsic dependencies between labels can provide critical knowledge to tackle the above challenges. To this end, we propose Label Reasoning Network(LRN), which sequentially reasons fine-grained entity labels by discovering and exploiting label dependencies knowledge entailed in the data. Specifically, LRN utilizes an auto-regressive network to conduct deductive reasoning and a bipartite attribute graph to conduct inductive reasoning between labels, which can effectively model, learn and reason complex label dependencies in a sequence-to- set, end-to-end manner. Experiments show that LRN achieves the state-of-the-art performance on standard ultra fine-grained entity typing benchmarks, and can also resolve the long tail label problem effectively.

Fine-grained Entity Typing via Label Reasoning

Distantly supervised relation extraction (RE) automatically aligns unstructured text with relation instances in a knowledge base (KB). Due to the incompleteness of current KBs, sentences implying certain relations may be annotated as N/A instances, which causes the socalled false negative (FN) problem. Current RE methods usually overlook this problem, inducing improper biases in both training and testing procedures. To address this issue, we propose a two-stage approach. First, it finds out possible FN samples by heuristically leveraging the memory mechanism of deep neural networks. Then, it aligns those unlabeled data with the training data into a unified feature space by adversarial training to assign pseudo labels and further utilize the information contained in them. Experiments on two wildlyused benchmark datasets demonstrate the effectiveness of our approach.

Knowing False Negatives: An Adversarial Training Method for Distantly Supervised Relation Extraction

Natural language processing systems such as dialogue agents should be able to reason about other people's beliefs, intentions and desires. This capability, called theory of mind (ToM), is crucial, as it allows a model to predict and interpret the needs of users based on their mental states. A recent line of research evaluates the ToM capability of existing memory-augmented neural models through question-answering. These models perform poorly on false belief tasks where beliefs differ from reality, especially when the dataset contains distracting sentences. In this paper, we propose a new temporally informed approach for improving the ToM capability of memory-augmented neural models. Our model incorporates priors about the entities' minds and tracks their mental states as they evolve over time through an extended passage. It then responds to queries through textual time travel---i.e., by accessing the stored memory of an earlier time step. We evaluate our model on ToM datasets and find that this approach improves performance, particularly by correcting the predicted mental states to match the false belief.

Textual Time Travel: A Temporally Informed Approach to Theory of Mind

Jupyter notebook allows data scientists to write machine learning code together with its documentation in cells. 
In this paper, we propose a new task of code documentation generation (CDG) for computational notebooks. In contrast to the previous CDG tasks which focus on generating documentation for single code snippets, in a computational notebook, one documentation in a markdown cell often corresponds to multiple code cells, 
and these code cells have an inherent structure. We proposed a new model HAConvGNN that uses a hierarchical attention mechanism to consider the relevant code cells and the relevant code tokens information when generating the documentation. Tested on a new corpus constructed from well-documented Kaggle notebooks, we show that our model outperforms other baseline models.

HAConvGNN: Hierarchical Attention Based Convolutional Graph Neural Network for Code Documentation Generation in Jupyter Notebooks

Large pre-trained language models have repeatedly shown their ability to produce fluent text. Yet even when starting from a prompt, generation can continue in many plausible directions. Current decoding methods with the goal of controlling generation, e.g., to ensure specific words are included, either require additional models or fine-tuning, or work poorly when the task at hand is semantically unconstrained, e.g., story generation. In this work, we present a plug-and-play decoding method for controlled language generation that is so simple and intuitive, it can be described in a single sentence: given a topic or keyword, we add a shift to the probability distribution over our vocabulary towards semantically similar words. We show how annealing this distribution can be used to impose hard constraints on language generation, something no other plug-and-play method is currently able to do with SOTA language generators. Despite the simplicity of this approach, we see it works incredibly well in practice: decoding from GPT-2 leads to diverse and fluent sentences while guaranteeing the appearance of given guide words. We perform two user studies, revealing that (1) our method outperforms competing methods in human evaluations; and (2) forcing the guide words to appear in the generated text has no impact on the fluency of the generated text.

A Plug-and-Play Method for Controlled Text Generation

Most progress in machine learning has been measured according to gains in top line metrics. However, top-line metrics are often not correlated with other design objectives that we would like our models to fulfill. How can we explain how deep neural networks arrive at decisions and audit when there are generalization failures? This talk explores some open challenges in deep neural network interpretability and will discuss desirable properties methods should fulfill in order to build trust between humans and algorithm.

Through the Looking-Glass: Understanding Model Behavior

Where next? Towards multi-text consumption via three inspired research lines

I survey work on linking Large Language Models to brain imaging data. I argue 'interpretability' is crucial for using this work to reveal more about how the human brain processes language. I then survey some of our contributions to interpretability methods, and discuss the relevance of these methods for the big questions from cognitive neuroscience.

Language, Brains & Interpretability

Human explanations are responses to "why P rather than Q?" where P is an observed event and Q an imagined, counterfactual event that did not occur. Since people assign human-like traits to AI models, they expect explanations of models' behavior to follow the same conceptual framework used to explain human behavior. The consequence of not meeting this expectation is that people using AI technology do not have agency and hence one of the XAI objectives is not achieved. NLP researchers are accepting the perspective of the social and behavioral sciences as evident by the number of papers that have been published in the last and this year on contrastive explanations. In this talk, I will present a categorization of methods for producing contrastive explanations of NLP models and go into details of one such method.

Contrastive Explanations of NLP Models

Research in financial economics utilizing natural language processing has modeled how U.S. companies compete across product markets, how they innovate, and how supply chains are organized. This presentation will summarize these models, how they have evolved to the present day, and new work in progress. A key technical foundation is that these concepts of market structure, which were developed by economic theorists, have relatedness and overlap as their very intellectual foundation. Hence NLP methods (new and old) that focus on textual relatedness make up an ideal toolkit for analyzing these economic concepts, which are also central to many applications aimed at understanding and predicting business success.
One of the most compelling properties of an NLP approach is that market structures can be updated and redrawn every year at low cost as textual corpora are updated. This observation indicates the use of NLP to measure innovation. This has led to many insights on how companies manage growth, whether and when they go public, and how they manage the risk of disruption. This work continues to evolve with a focus on innovation, disaggregating complex product portfolios, and to build NLP measures for private firms through their websites. Among other economic findings, this work reveals that U.S. companies are becoming increasingly complex and broad in their product offerings, a result that is not seen using standard numerical databases in the public domain. Work using these same methods has also been deployed in related settings such as modeling systemic risk.

An animated picture says at least a thousand words: Selecting Gif-based Replies in Multimodal Dialog

Downloads

Next from EMNLP 2021

Fine-grained Entity Typing via Label Reasoning

Similar lecture

A Large-Scale Dataset for Empathetic Response Generation